Enhanced Labeling of Issue Reports (with F^3T)
share
Standard automatic methods for recognizing problematic code can be greatly improved via the incremental application of human+artificial expertise. In this approach, call F^3T, AI tools explore software to find commits that they guess is most problematic. Humans the apply their expertise to check that guess (perhaps resulting in the AI updating the support vectors within their SVM learner). We recommend this human+AI partnership, for several reasons. When a new domain is encountered, F^3T can learn better ways to label which comments refer to real problems. Further, in studies with 9 open source software projects, F^3T's incremental application of human+artificial intelligence is at least an order of magnitude cheaper to use than existing methods. Lastly, F^3T is very effective. For the data sets explored here, when compared to standard methods, F^3T improved P_opt(20) and G-scores performance by 26% and 48% on median value.
READ FULL TEXT