An Empirical Analysis of UI-based Flaky Tests

03/03/2021
by   Alan Romano, et al.
0

Flaky tests have gained attention from the research community in recent years and with good reason. These tests lead to wasted time and resources, and they reduce the reliability of the test suites and build systems they affect. However, most of the existing work on flaky tests focus exclusively on traditional unit tests. This work ignores UI tests that have larger input spaces and more diverse running conditions than traditional unit tests. In addition, UI tests tend to be more complex and resource-heavy, making them unsuited for detection techniques involving rerunning test suites multiple times. In this paper, we perform a study on flaky UI tests. We analyze 235 flaky UI test samples found in 62 projects from both web and Android environments. We identify the common underlying root causes of flakiness in the UI tests, the strategies used to manifest the flaky behavior, and the fixing strategies used to remedy flaky UI tests. The findings made in this work can provide a foundation for the development of detection and prevention techniques for flakiness arising in UI tests.

READ FULL TEXT

page 4

page 6

research
07/03/2022

An Empirical Study of Flaky Tests in JavaScript

Flaky tests (tests with non-deterministic outcomes) can be problematic f...
research
05/14/2020

Intermittently Failing Tests in the Embedded Systems Domain

Software testing is sometimes plagued with intermittently failing tests ...
research
05/08/2023

FlaPy: Mining Flaky Python Tests at Scale

Flaky tests obstruct software development, and studying and proposing mi...
research
12/19/2018

Carving Parameterized Unit Tests

We present a method to automatically extract ("carve") parameterized uni...
research
06/27/2022

Analisis Kepuasan Pengguna Aplikasi Bintang Cash Credit Menggunakan Metode End User Computing Satisfaction (EUCS)

The use of android application technology has advanced rapidly in recent...
research
07/26/2023

Empirical likelihood test for community structure in networks

Network data, characterized by interconnected nodes and edges, is pervas...
research
02/23/2022

Exploring Classic Quantitative Strategies

The goal of this paper is to debunk and dispel the magic behind the blac...

Please sign up or login with your details

Forgot password? Click here to reset