On the use of test smells for prediction of flaky tests

08/26/2021
by   B. H. P. Camara, et al.
0

Regression testing is an important phase to deliver software with quality. However, flaky tests hamper the evaluation of test results and can increase costs. This is because a flaky test may pass or fail non-deterministically and to identify properly the flakiness of a test requires rerunning the test suite multiple times. To cope with this challenge, approaches have been proposed based on prediction models and machine learning. Existing approaches based on the use of the test case vocabulary may be context-sensitive and prone to overfitting, presenting low performance when executed in a cross-project scenario. To overcome these limitations, we investigate the use of test smells as predictors of flaky tests. We conducted an empirical study to understand if test smells have good performance as a classifier to predict the flakiness in the cross-project context, and analyzed the information gain of each test smell. We also compared the test smell-based approach with the vocabulary-based one. As a result, we obtained a classifier that had a reasonable performance (Random Forest, 0.83) to predict the flakiness in the testing phase. This classifier presented better performance than vocabulary-based model for cross-project prediction. The Assertion Roulette and Sleepy Test test smell types are the ones associated with the best information gain values.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/23/2021

What is the Vocabulary of Flaky Tests? An Extended Replication

Software systems have been continuously evolved and delivered with high ...
research
06/14/2023

Explainable Software Defect Prediction from Cross Company Project Metrics Using Machine Learning

Predicting the number of defects in a project is critical for project te...
research
02/24/2022

Investigating the Use of One-Class Support Vector Machine for Software Defect Prediction

Early software defect identification is considered an important step tow...
research
11/02/2018

Too Trivial To Test? An Inverse View on Defect Prediction to Identify Methods with Low Fault Risk

Background. Test resources are usually limited and therefore it is often...
research
03/24/2021

Data Balancing Improves Self-Admitted Technical Debt Detection

A high imbalance exists between technical debt and non-technical debt so...
research
04/13/2021

Feature-Oriented Defect Prediction: Scenarios, Metrics, and Classifiers

Several software defect prediction techniques have been developed over t...
research
11/05/2021

Discerning Legitimate Failures From False Alerts: A Study of Chromium's Continuous Integration

Flakiness is a major concern in Software testing. Flaky tests pass and f...

Please sign up or login with your details

Forgot password? Click here to reset