DeepAI AI Chat
Log In Sign Up

Scripted GUI Testing of Android Apps: A Study on Diffusion, Evolution and Fragility

by   Riccardo Coppola, et al.

Background. Evidence suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI test fragility, i.e. GUI test classes failing due to minor modifications in the GUI, without the application functionalities being altered. Aims. The objective of our study is to examine the diffusion of GUI testing on Android, and the amount of changes required to keep test classes up to date, and in particular the changes due to GUI test fragility. We define metrics to characterize the modifications and evolution of test classes and test methods, and proxies to estimate fragility-induced changes. Method. To perform our experiments, we selected six widely used open-source tools for scripted GUI testing of mobile applications previously described in the literature. We have mined the repositories on GitHub that used those tools, and computed our set of metrics. Results. We found that none of the considered GUI testing frameworks achieved a major diffusion among the open-source Android projects available on GitHub. For projects with GUI tests, we found that test suites have to be modified often, specifically 5%-10% of developers' modified LOCs belong to tests, and that a relevant portion (60% on average) of such modifications are induced by fragility. Conclusions. Fragility of GUI test classes constitute a relevant concern, possibly being an obstacle for developers to adopt automated scripted GUI tests. This first evaluation and measure of fragility of Android scripted GUI testing can constitute a benchmark for developers, and the basis for the definition of a taxonomy of fragility causes, and actionable guidelines to mitigate the issue.


page 1

page 2

page 3

page 4


Espresso vs. EyeAutomate: Comparison of Two Generations of Android GUI Testing

Context: Albeit different approaches exist for automated GUI testing of ...

How do Developers Test Android Applications?

Enabling fully automated testing of mobile applications has recently bec...

To the Attention of Mobile Software Developers: Guess What, Test your App!

Software testing is an important phase in the software development life-...

Fragility of Layout-Based and Visual GUI Test Scripts: An Assessment Study on a Hybrid Mobile Application

Context: Albeit different approaches exist for automated GUI testing of ...

Automation of Android Applications Testing Using Machine Learning Activities Classification

Mobile applications are being used every day by more than half of the wo...

Concurrency-related Flaky Test Detection in Android apps

Validation of Android apps via testing is difficult owing to the presenc...

Obfuscation Resilient Search through Executable Classification

Android applications are usually obfuscated before release, making it di...