AidUI: Toward Automated Recognition of Dark Patterns in User Interfaces

03/12/2023
by   SM Hasan Mansur, et al.
0

Past studies have illustrated the prevalence of UI dark patterns, or user interfaces that can lead end-users toward (unknowingly) taking actions that they may not have intended. Such deceptive UI designs can result in adverse effects on end users, such as oversharing personal information or financial loss. While significant research progress has been made toward the development of dark pattern taxonomies, developers and users currently lack guidance to help recognize, avoid, and navigate these often subtle design motifs. However, automated recognition of dark patterns is a challenging task, as the instantiation of a single type of pattern can take many forms, leading to significant variability. In this paper, we take the first step toward understanding the extent to which common UI dark patterns can be automatically recognized in modern software applications. To do this, we introduce AidUI, a novel automated approach that uses computer vision and natural language processing techniques to recognize a set of visual and textual cues in application screenshots that signify the presence of ten unique UI dark patterns, allowing for their detection, classification, and localization. To evaluate our approach, we have constructed ContextDP, the current largest dataset of fully-localized UI dark patterns that spans 175 mobile and 83 web UI screenshots containing 301 dark pattern instances. The results of our evaluation illustrate that achieves an overall precision of 0.66, recall of 0.67, F1-score of 0.65 in detecting dark pattern instances, reports few false positives, and is able to localize detected patterns with an IoU score of  0.84. Furthermore, a significant subset of our studied dark patterns can be detected quite reliably (F1 score of over 0.82), and future research directions may allow for improved detection of additional patterns.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 9

research
08/11/2023

Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile Applications

Mobile apps bring us many conveniences, such as online shopping and comm...
research
03/10/2021

ReportAGE: Automatically extracting the exact age of Twitter users based on self-reports in tweets

Advancing the utility of social media data for research applications req...
research
08/29/2020

SocCogCom at SemEval-2020 Task 11: Characterizing and Detecting Propaganda using Sentence-Level Emotional Salience Features

This paper describes a system developed for detecting propaganda techniq...
research
10/27/2022

Fully-attentive and interpretable: vision and video vision transformers for pain detection

Pain is a serious and costly issue globally, but to be treated, it must ...
research
03/12/2019

Taxonomies in DUI Design Patterns: A Systematic Approach for Removing Overlaps Among Design Patterns and Creating a Clear Hierarchy

Recently a library of design patterns for designing distributed user int...
research
01/19/2021

Situation and Behavior Understanding by Trope Detection on Films

The human ability of deep cognitive skills are crucial for the developme...
research
05/22/2023

Defending Against the Dark Arts: Recognising Dark Patterns in Social Media

Interest in unethical user interfaces has grown in HCI over recent years...

Please sign up or login with your details

Forgot password? Click here to reset