Log In Sign Up

What do all these Buttons do? Statically Mining Android User Interfaces at Scale

by   Konstantin Kuznetsov, et al.

We introduce FRONTMATTER: a tool to automatically mine both user interface models and behavior of Android apps at a large scale with high precision. Given an app, FRONTMATTER statically extracts all declared screens, the user interface elements, their textual and graphical features, as well as Android APIs invoked by interacting with them. Executed on tens of thousands of real-world apps, FRONTMATTER opens the door for comprehensive mining of mobile user interfaces, jumpstarting empirical research at a large scale, addressing questions such as "How many travel apps require registration?", "Which apps do not follow accessibility guidelines?", "Does the user interface correspond to the description?", and many more. FRONTMATTER and the mined dataset are available under an open-source license.


Do we agree on user interface aesthetics of Android apps?

Context: Visual aesthetics is increasingly seen as an essential factor i...

Modeling Mobile Interface Tappability Using Crowdsourcing and Deep Learning

Tapping is an immensely important gesture in mobile touchscreen interfac...

Siri, Write the Next Method

Code completion is one of the killer features of Integrated Development ...

Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels

Many accessibility features available on mobile platforms require applic...

pix2code: Generating Code from a Graphical User Interface Screenshot

Transforming a graphical user interface screenshot created by a designer...

Empirically Assessing Opportunities for Prefetching and Caching in Mobile Apps

Network latency in mobile software has a large impact on user experience...

Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps

It is common practice for developers of user-facing software to transfor...