DeepAI AI Chat
Log In Sign Up

Layout and Image Recognition Driving Cross-Platform Automated Mobile Testing

by   Shengcheng Yu, et al.

The fragmentation problem has extended from Android to different platforms, such as iOS, mobile web, and even mini-programs within some applications (app). In such a situation, recording and replaying test scripts is a popular automated mobile app testing approaches. But such approach encounters severe problems when crossing platforms. Different versions of the same app need to be developed to support different platforms relying on different platform supports. Therefore, mobile app developers need to develop and maintain test scripts for multiple platforms aimed at completely the same test requirements, greatly increasing testing costs. However, we discover that developers adopt highly similar user interface layouts for versions of the same app on different platforms. Such a phenomenon inspires us to replay test scripts from the perspective of similar UI layouts. We propose an image-driven mobile app testing framework, utilizing Widget Feature Matching and Layout Characterization Matching. We use computer vision technologies to perform UI feature comparison and layout hierarchy extraction on app screenshots to obtain UI structures with rich contextual information, including coordinates, relative relationship, etc. Based on acquired UI structures, we can form a platform-independent test script, and then locate the target widgets under test. Thus, the proposed framework non-intrusively replays test scripts according to a novel platform-independent test script model. We also design and implement a tool named LIT to devote the proposed framework into practice, based on which, we conduct an empirical study to evaluate the effectiveness and usability of the proposed testing framework. Results show that the overall replay accuracy reaches around 63.39 state-of-the-art approaches) and 21.83 state-of-the-art approaches).


NiCro: Purely Vision-based, Non-intrusive Cross-Device and Cross-Platform GUI Testing

To ensure app compatibility and smoothness of user experience across div...

Automated Mobile App Test Script Intent Generation via Image and Code Understanding

Testing is the most direct and effective technique to ensure software qu...

Universally Adaptive Cross-Platform Reinforcement Learning Testing via GUI Image Understanding

With the rapid development of the Internet, more and more applications (...

Continuous, Evolutionary and Large-Scale: A New Perspective for Automated Mobile App Testing

Mobile app development involves a unique set of challenges including dev...

A Semi-Supervised and Inductive Embedding Model for Churn Prediction of Large-Scale Mobile Games

Mobile gaming has emerged as a promising market with billion-dollar reve...

Espresso vs. EyeAutomate: Comparison of Two Generations of Android GUI Testing

Context: Albeit different approaches exist for automated GUI testing of ...

Fragility of Layout-Based and Visual GUI Test Scripts: An Assessment Study on a Hybrid Mobile Application

Context: Albeit different approaches exist for automated GUI testing of ...