Evaluating software defect prediction performance: an updated benchmarking study

by   Libo Li, et al.

Accurately predicting faulty software units helps practitioners target faulty units and prioritize their efforts to maintain software quality. Prior studies use machine-learning models to detect faulty software code. We revisit past studies and point out potential improvements. Our new study proposes a revised benchmarking configuration. The configuration considers many new dimensions, such as class distribution sampling, evaluation metrics, and testing procedures. The new study also includes new datasets and models. Our findings suggest that predictive accuracy is generally good. However, predictive power is heavily influenced by the evaluation metrics and testing procedure (frequentist or Bayesian approach). The classifier results depend on the software project. While it is difficult to choose the best classifier, researchers should consider different dimensions to overcome potential bias.



There are no comments yet.


page 1

page 2

page 3

page 4


Configuration Testing: Testing Configuration Values as Code and with Code

This paper proposes configuration testing--evaluating configuration valu...

Software Effort Estimation Accuracy Prediction of Machine Learning Techniques: A Systematic Performance Evaluation

Software effort estimation accuracy is a key factor in effective plannin...

Does class size matter? An in-depth assessment of the effect of class size in software defect prediction

In the past 20 years, defect prediction studies have generally acknowled...

Moving from Cross-Project Defect Prediction to Heterogeneous Defect Prediction: A Partial Replication Study

Software defect prediction heavily relies on the metrics collected from ...

Software Defect Prediction Based On Deep Learning Models: Performance Study

In recent years, defect prediction, one of the major software engineerin...

Correlation and Prediction of Evaluation Metrics in Information Retrieval

Because researchers typically do not have the time or space to present m...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.