Using Machine Learning to Generate Test Oracles: A Systematic Literature Review

07/02/2021
by   Afonso Fontes, et al.
0

Machine learning may enable the automated generation of test oracles. We have characterized emerging research in this area through a systematic literature review examining oracle types, researcher goals, the ML techniques applied, how the generation process was assessed, and the open research challenges in this emerging field. Based on a sample of 22 relevant studies, we observed that ML algorithms generated test verdict, metamorphic relation, and - most commonly - expected output oracles. Almost all studies employ a supervised or semi-supervised approach, trained on labeled system executions or code metadata - including neural networks, support vector machines, adaptive boosting, and decision trees. Oracles are evaluated using the mutation score, correct classifications, accuracy, and ROC. Work-to-date show great promise, but there are significant open challenges regarding the requirements imposed on training data, the complexity of modeled functions, the ML algorithms employed - and how they are applied - the benchmarks used by researchers, and replicability of the studies. We hope that our findings will serve as a roadmap and inspiration for researchers in this field.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/21/2022

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

Context: Machine learning (ML) may enable effective automated test gener...
research
06/25/2021

Test Case Selection and Prioritization Using Machine Learning: A Systematic Literature Review

Regression testing is an essential activity to assure that software code...
research
05/22/2023

Systematic Literature Review on Application of Machine Learning in Continuous Integration

This research conducted a systematic review of the literature on machine...
research
03/02/2020

A review of machine learning applications in wildfire science and management

Artificial intelligence has been applied in wildfire science and managem...
research
03/06/2021

Applying Machine Learning in Self-Adaptive Systems: A Systematic Literature Review

Recently, we witness a rapid increase in the use of machine learning in ...
research
12/18/2020

Machine learning applications using diffusion tensor imaging of human brain: A PubMed literature review

We performed a PubMed search to find 148 papers published between Januar...
research
02/20/2023

Railway Virtual Coupling: A Survey of Emerging Control Techniques

This paper provides a systematic review of emerging control techniques u...

Please sign up or login with your details

Forgot password? Click here to reset