A Simulation Study of Bandit Algorithms to Address External Validity of Software Fault Prediction

Various software fault prediction models and techniques for building algorithms have been proposed. Many studies have compared and evaluated them to identify the most effective ones. However, in most cases, such models and techniques do not have the best performance on every dataset. This is because there is diversity of software development datasets, and therefore, there is a risk that the selected model or technique shows bad performance on a certain dataset. To avoid selecting a low accuracy model, we apply bandit algorithms to predict faults. Consider a case where player has 100 coins to bet on several slot machines. Ordinary usage of software fault prediction is analogous to the player betting all 100 coins in one slot machine. In contrast, bandit algorithms bet one coin on each machine (i.e., use prediction models) step-by-step to seek the best machine. In the experiment, we developed an artificial dataset that includes 100 modules, 15 of which include faults. Then, we developed various artificial fault prediction models and selected them dynamically using bandit algorithms. The Thomson sampling algorithm showed the best or second-best prediction performance compared with using only one prediction model.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/13/2023

Do the Defect Prediction Models Really Work?

You may develop a potential prediction model, but how can I trust your m...
research
04/29/2021

Machine Learning Techniques for Software Quality Assurance: A Survey

Over the last years, machine learning techniques have been applied to mo...
research
04/26/2021

Revisiting the size effect in software fault prediction models

BACKGROUND: In object oriented (OO) software systems, class size has bee...
research
02/24/2021

Practitioners' Perceptions of the Goals and Visual Explanations of Defect Prediction Models

Software defect prediction models are classifiers that are constructed f...
research
10/28/2019

The Study of Machine Learning Models in Predicting the Intention of Adolescents to Smoke Cigarettes

The use of electronic cigarette (e-cigarette) is increasing among adoles...
research
04/01/2021

Exploring the relationship between performance metrics and cost saving potential of defect prediction models

Performance metrics are a core component of the evaluation of any machin...
research
05/18/2015

DopeLearning: A Computational Approach to Rap Lyrics Generation

Writing rap lyrics requires both creativity to construct a meaningful, i...

Please sign up or login with your details

Forgot password? Click here to reset