Scripted GUI Testing of Android Apps: A Study on Diffusion, Evolution and Fragility

11/09/2017
by   Riccardo Coppola, et al.
0

Background. Evidence suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI test fragility, i.e. GUI test classes failing due to minor modifications in the GUI, without the application functionalities being altered. Aims. The objective of our study is to examine the diffusion of GUI testing on Android, and the amount of changes required to keep test classes up to date, and in particular the changes due to GUI test fragility. We define metrics to characterize the modifications and evolution of test classes and test methods, and proxies to estimate fragility-induced changes. Method. To perform our experiments, we selected six widely used open-source tools for scripted GUI testing of mobile applications previously described in the literature. We have mined the repositories on GitHub that used those tools, and computed our set of metrics. Results. We found that none of the considered GUI testing frameworks achieved a major diffusion among the open-source Android projects available on GitHub. For projects with GUI tests, we found that test suites have to be modified often, specifically 5%-10% of developers' modified LOCs belong to tests, and that a relevant portion (60% on average) of such modifications are induced by fragility. Conclusions. Fragility of GUI test classes constitute a relevant concern, possibly being an obstacle for developers to adopt automated scripted GUI tests. This first evaluation and measure of fragility of Android scripted GUI testing can constitute a benchmark for developers, and the basis for the definition of a taxonomy of fragility causes, and actionable guidelines to mitigate the issue.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/18/2019

Espresso vs. EyeAutomate: Comparison of Two Generations of Android GUI Testing

Context: Albeit different approaches exist for automated GUI testing of ...
research
01/19/2018

How do Developers Test Android Applications?

Enabling fully automated testing of mobile applications has recently bec...
research
02/07/2019

To the Attention of Mobile Software Developers: Guess What, Test your App!

Software testing is an important phase in the software development life-...
research
07/18/2019

Fragility of Layout-Based and Visual GUI Test Scripts: An Assessment Study on a Hybrid Mobile Application

Context: Albeit different approaches exist for automated GUI testing of ...
research
09/04/2017

Automation of Android Applications Testing Using Machine Learning Activities Classification

Mobile applications are being used every day by more than half of the wo...
research
05/30/2015

An Open Source Testing Tool for Evaluating Handwriting Input Methods

This paper presents an open source tool for testing the recognition accu...
research
06/06/2018

Obfuscation Resilient Search through Executable Classification

Android applications are usually obfuscated before release, making it di...

Please sign up or login with your details

Forgot password? Click here to reset