A Systematic Literature Review of Test Breakage Prevention and Repair Techniques

by   Javaria Imtiaz, et al.

Context: When an application evolves, some of the developed test cases break. Discarding broken test cases causes a significant waste of effort and leads to test suites that are less effective and have lower coverage. Test repair approaches evolve test suites along with applications by repairing the broken test cases. Objective: Numerous studies are published on test repair approaches every year. It is important to summarise and consolidate the existing knowledge in the area to provide directions to researchers and practitioners. This research work provides a systematic literature review in the area of test case repair and breakage prevention, aiming to guide researchers and practitioners in the field of software testing. Method: We followed the standard protocol for conducting a systematic literature review. First, research goals were defined using the Goal Question Metric (GQM). Then we formulate research questions corresponding to each goal. Finally, metrics are extracted from the included papers. Based on the defined selection criteria a final set of 41 primary studies are included for analysis. Results: The selection process resulted in 5 journal papers, and 36 conference papers. We present a taxonomy that lists the causes of test case breakages extracted from the literature. We found that only four proposed test repair tools are publicly available. Most studies evaluated their approaches on open-source case studies. Conclusion: There is significant room for future research on test repair techniques. Despite the positive trend of evaluating approaches on large scale open-source studies, there is a clear lack of results from studies done in a real industrial context. Few tools are publicly available which lowers the potential of adaption by industry practitioners.



There are no comments yet.


page 16

page 22

page 23

page 25

page 30


NLP-assisted software testing: a systematic review

Context: To reduce manual effort of extracting test cases from natural-l...

A Systematic Literature Review of Automated Techniques for Functional GUI Testing of Mobile Applications

Context. Multiple automated techniques have been proposed and developed ...

Exploring ML testing in practice – Lessons learned from an interactive rapid review with Axis Communications

There is a growing interest in industry and academia in machine learning...

Test Case Selection and Prioritization Using Machine Learning: A Systematic Literature Review

Regression testing is an essential activity to assure that software code...

Predicting Software Effort from Use Case Points: A Systematic Review

Context: Predicting software project effort from Use Case Points (UCP) m...

Code and Structure Editing for Teaching: A Case Study in using Bibliometrics to Guide Computer Science Research

Structure or projectional editors are a well-studied concept among resea...

Data-driven surrogate modelling and benchmarking for process equipment

A suite of computational fluid dynamics (CFD) simulations geared towards...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.