Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content

10/08/2021
by   Alan Lundgard, et al.
0

Natural language descriptions sometimes accompany visualizations to better communicate and contextualize their insights, and to improve their accessibility for readers with disabilities. However, it is difficult to evaluate the usefulness of these descriptions, and how effectively they improve access to meaningful information, because we have little understanding of the semantic content they convey, and how different readers receive this content. In response, we introduce a conceptual model for the semantic content conveyed by natural language descriptions of visualizations. Developed through a grounded theory analysis of 2,147 sentences, our model spans four levels of semantic content: enumerating visualization construction properties (e.g., marks and encodings); reporting statistical concepts and relations (e.g., extrema and correlations); identifying perceptual and cognitive phenomena (e.g., complex trends and patterns); and elucidating domain-specific insights (e.g., social and political context). To demonstrate how our model can be applied to evaluate the effectiveness of visualization descriptions, we conduct a mixed-methods evaluation with 30 blind and 90 sighted readers, and find that these reader groups differ significantly on which semantic content they rank as most useful. Together, our model and findings suggest that access to meaningful information is strongly reader-specific, and that research in automatic visualization captioning should orient toward descriptions that more richly communicate overall trends and statistics, sensitive to reader preferences. Our work further opens a space of research on natural language as a data interface coequal with visualization.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 8

research
07/17/2023

"Customization is Key": Four Characteristics of Textual Affordances for Accessible Data Visualization

Current best practices recommend using textual descriptions to make data...
research
10/18/2020

Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model

Information visualizations such as bar charts and line charts are very p...
research
07/03/2016

Visualizing Natural Language Descriptions: A Survey

A natural language interface exploits the conceptual simplicity and natu...
research
05/10/2022

Rich Screen Reader Experiences for Accessible Data Visualization

Current web accessibility guidelines ask visualization designers to supp...
research
11/28/2019

Words of Estimative Correlation: Studying Verbalizations of Scatterplots

Multimodal approaches where interactive visualization and natural langua...
research
11/19/2018

A Map of Knowledge

Knowledge representation has gained in relevance as data from the ubiqui...
research
07/26/2019

Cooperative image captioning

When describing images with natural language, the descriptions can be ma...

Please sign up or login with your details

Forgot password? Click here to reset