DeepAI
Log In Sign Up

Communicating Visualizations without Visuals: Investigation of Visualization Alternative Text for People with Visual Impairments

08/08/2021
by   Crescentia Jung, et al.
University of Wisconsin-Madison
0

Alternative text is critical in communicating graphics to people who are blind or have low vision. Especially for graphics that contain rich information, such as visualizations, poorly written or an absence of alternative texts can worsen the information access inequality for people with visual impairments. In this work, we consolidate existing guidelines and survey current practices to inspect to what extent current practices and recommendations are aligned. Then, to gain more insight into what people want in visualization alternative texts, we interviewed 22 people with visual impairments regarding their experience with visualizations and their information needs in alternative texts. The study findings suggest that participants actively try to construct an image of visualizations in their head while listening to alternative texts and wish to carry out visualization tasks (e.g., retrieve specific values) as sighted viewers would. The study also provides ample support for the need to reference the underlying data instead of visual elements to reduce users' cognitive burden. Informed by the study, we provide a set of recommendations to compose an informative alternative text.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

01/12/2021

Vis Ex Machina: An Analysis of Trust in Human versus Algorithmically Generated Visualization Recommendations

More visualization systems are simplifying the data analysis process by ...
01/30/2023

Understanding Visual Arts Experiences of Blind People

Visual arts play an important role in cultural life and provide access t...
09/11/2019

Sociotechnical Considerations for Accessible Visualization Design

Accessibility–the process of designing for people with disabilities (PWD...
09/04/2020

Literal Encoding: Text is a first-class data encoding

Digital humanities are rooted in text analysis. However, most visualizat...

1 Introduction

Visualizations are a powerful medium to summarize complex data effectively, as they offload the cognitive burden of a reader by leveraging visual perception [13]. The visualization community has devised theories and principles to design effective visualizations so that readers can extract insights without much mental effort. As a result, visualizations are pervasive across the media, scientific communities, and government agencies to present data. In addition to efficient data communication, visualizations also provide credibility to the content presented in articles [46] or can complement the content [21].

However, people with visual impairments have been largely underserved in receiving the benefit of “using vision to think” that visualizations offer. Not being able to understand visualizations may result in aggravating information inequality of people with visual impairments. A few assistive technologies have been proposed to bridge this gap over the past years, attempting to make visualizations accessible to a broader audience. For example, popular approaches include tactile visualizations (e.g., [89, 36]) that use embossed surfaces to present visualizations and data sonification (e.g., [64, 90]) that maps the data to various dimensions (e.g., pitch, volume) of sounds. However, integrating these technologies into the online environment, in which we frequently consume data, is a major challenge.

Communicating visualizations with text can be a practical way to convey the information in visualizations, especially in web environments. For example, HTML supports many ways to add invisible text fields near graphics to allow people with visual impairments to access the description of graphics through screen readers. These descriptions used to replace graphics are known as alternative text (alt text). Alt text serves as a textual substitution of a graphic that describes the visual components within it. This helps people to construct the graphic mentally when they have no access to the graphic. Several organizations provide guidelines to create alt text for visualizations. However, the guidelines do not offer rationales and empirical evidence for their recommendations. Thus, it remains unclear whether the suggested ways of generating alt text align with what people with visual impairments need or how effective the recommendations are.

In this work, we investigate how to formulate alt text for visualizations to best serve people with visual impairments. As a first step, we surveyed current guidelines on generating visualization alt text. Next, we reviewed current practices to understand to what extent people follow the guidelines and what other approaches people use when formulating alt texts. As we could not find many examples of alt texts generated for visualizations from major news outlets, we collected alt texts from academic papers. We focused on alt texts written by people who have expertise in visualizations and graphics (IEEE VIS & TVCG), have knowledge of accessibility (ACM ASSETS), or are often prompted to write alt texts for visualizations (ACM CHI as the submission system encourages authors to submit alt texts for figures). We found that people use diverse strategies, ranging from providing brief explanations of the charts to listing all data points. The observed practices are only partially aligned with the surveyed guidelines.

To evaluate the surveyed guidelines and further probe information needs for visualization alt texts, we interviewed 22 participants with visual impairments. In the study, participants examined visualizations by reading the alt text formulated with several strategies through their screen readers. Then, participants were asked to share their preferences and opinions on the strategies and provide insights toward the ideal visualization alt texts. Our findings support many of the surveyed guidelines, such as the need to provide information about the chart type, axes, access to data points, and description of data trends. However, contrary to guidelines, some participants want to enrich their mental picture of the visualization by knowing visual attributes used in visualization, such as color encoding. This is in line with another finding of the study – that participants wish to “visualize” visualizations in their head while listening to alt texts. This finding implies that alt texts should describe the necessary visual components to fill the gap in their imagination and ease the construction of an image of the visualization. Additionally, our study findings provide evidence to guide how an author should generate alt texts for their visualizations, including which component to mention first, how an author should describe the trend in the visualization, and what language tone the author should maintain.

Our contribution is three-fold:

  1. [topsep=-0mm,itemsep=0mm,partopsep=0ex,parsep=0.1ex]

  2. We survey and summarize existing guidelines for the generation of visualization alt text. We also analyze current practices to demonstrate how aligned they are with the guidelines and identify other commonly used strategies.

  3. We report findings from an interview study with 22 people with visual impairments that provide insights into the properties of good alt texts and into the role of visualizations for people with visual impairments to support people with visual impairments in the context of reading online news.

  4. We propose actionable recommendations for generating informative alt texts along with rationales and suggest system-level improvements (e.g., structures of HTML, multimodal integration) informed by the study to better support visualization interpretation for people with visual impairments.

2 Background

People with visual impairments are a large population that includes people who are blind and have low vision. According to the World Health Organization, about 285 million people in the world have visual impairments, of whom 39 million are entirely blind [58]. Due to their vision loss, this population faces difficulties in understanding graphics, which hinders their education and employment opportunities.

With the advances of the Internet and computer technology, numerous physical readings and learning materials have been digitized into web pages or other digital formats (e.g., pdf, doc). To enable people with visual impairments to access digital information, screen readers have been developed. Screen readers are software programs that scan the text on a screen and communicate the information to people with visual impairments via speech (some also have braille output via a braille display) [30]. Users control which components in the page the screen reader communicates through several key combinations on the keyboard or a touch gesture on a smartphone. Most mainstream computer or smartphone systems have embedded screen readers. Some examples include VoiceOver for Mac and iOS [5], Narrator for Windows 10 [51], and TalkBack for Android [39]. There are also standalone screen reader softwares, such as JAWS [68] and NVDA [2].

While making text more accessible, screen readers cannot interpret images for people with visual impairments. Alternative text is the major source that screen readers rely on to communicate graphics. Alternative text, also known as “Alt text,” was first introduced to the HTML 2.0 specification in 1995 [8]. Alt text is an invisible text block that can be inserted as an alt attribute of the <img> tag. The original purpose was to inform web viewers of an image’s content when the image could not show properly. Nowadays, alt text is commonly used by people with visual impairments to access image content via screen readers. Adding alt text to images has thus become one of the most important principles for web accessibility [85]. Similarly, the longdesc attribute in an <img> tag is used to provide a more lengthy explanation about non-text elements [1]. URLs, a location in the page referred by id or class, or text can be part of longdesc. Even though longdesc is deprecated in HTML5, WCAG still recommends it to be provided for complex graphics.

For other graphic formats such as SVG elements, the <title> and <desc> tags can also be used to convey the contents of visualizations in an easily accessible textual form. However, compatibility issues with some screen readers have been reported [83, 71]. Beyond web pages, alt text features have been extended to various digital document formats (e.g., docx, pptx, pdf). Commercial content-editing and viewing software, such as Microsoft Word and Adobe Acrobat, also enables users to add alt text to images, thus making digital documents more accessible to people with visual impairments [50, 3].

2.1 Visualization accessibility for visually impaired people

Several strategies have been explored for communicating visualizations to people with visual impairments by relying on senses other than vision. One popular approach is sonification [16, 80, 64], which seeks to encode the data of a chart into various dimensions (e.g., pitch, volume, pan) of an audio signal. Several studies have shown that audio charts can communicate patterns in the data effectively [16, 64]. Data sonification has been used on various visualizations such as line graphs, bar graphs, and maps  [90, 22, 40]. Tactile visualizations realized via haptic feedback, braille display, or embossed prints are another popular approach to enhance accessibility for people with visual impairments [82, 25, 28, 26, 27, 59, 89, 31, 41, 24]. Yet, the low-resolution nature of tactile visualizations limits their use for more complex visualizations [27]. Another recent exploration suggests that olfaction can be use to perceive data [60]. To overcome the limitation of one modality, multiple sensory inputs can also be combined to complement each other [37, 34, 73, 49, 81, 32, 47].

Summarizing visualizations in textual form is another way to convey visual information to people with visual impairments [23, 53, 35]. EvoGraphs is a jQuery plugin that enables creating visualizations that can be read by screen readers [69]. EvoGraphs automatically formulates an alt text that contains each data point and some representative values of the data (e.g., max, min, mean). However, these systems often focus on one or two basic chart types, such as bar and line charts. While not aiming at supporting people with impairment specifically, several techniques have been proposed to automatically generate captions for visualizations [29, 19, 52, 42, 43, 15, 57, 86, 12]. A more thorough survey on accessible visualizations can be found [45].

2.2 Study on alternative text for images

Unlike visualization alt texts, methods to generate alt texts for images (i.e., image description111Conventionally, the community concerning alternative texts for images refers the alt texts more generally as image descriptions or image captions) have been explored extensively. As presented in [72]

, alt text generation can be categorized in three groups: human-generated (e.g., 

[9]), computer-generated (e.g., [76, 88]), and hybrid approaches (e.g., [54, 65, 66]). While many efforts have been put into investigating formulating and evaluating image descriptions, a large proportion of images found online lack alt text descriptions [33].

Recommendations for alt text formulation depend on the purpose of the image, presented objects, individuals, active motions, location, colors, and emotions [61]. Studies in image alt text often focus on social network contexts where people share their photos actively [6, 55, 79, 91]. Updated guidelines for social networking sites suggest containing elements such as the number of people and facial expressions [87]. Recent research demonstrates that people with visual impairments have different needs based on the context where they encounter the images (e.g., social network, e-commerce) [72].

3 Phase 1: Guidelines for Visualization alt texts

To understand the current standard, we collected the existing guidelines for generating visualization alt texts. Since the goal of alt texts is to replace the graphics, most guidelines address how to describe visual components in visualizations to support people to mentally construct visualizations.

3.1 Guideline collection

We collected guidelines via Google search using relevant keywords such as visualization accessibility, visualization alternative text, etc., resulting in 31 postings. We excluded accessibility guidelines that were not explicitly related to blind or low vision individuals (e.g., guidelines for color blind people), guidelines not focused on formulating alt texts, or guidelines created by citing other guidelines. This process resulted in four sets of guidelines, namely the WCAG guidelines [18], the Penn State’s accessibility guidelines [14], the Diagram Center’s guidelines [70], and CFPB’s guidelines [78]. The Web Content Accessibility Guidelines (WCAG) 2.1 is a widely used collection of recommendations for increasing the accessibility of Web content [84]. WCAG provided guidelines for visualization alt text through the W3C Web Accessibility Initiative [18]. Accessibility at Penn State complemented the WCAG guidelines with specific examples of a few different chart types [14]. The Diagram Center is a research development center that provides specific guidelines for various types of graphics such as charts [70]

. Finally, CFPB Design System is an open-source resource for teams at the Consumer Financial Protection Bureau (CFPB) that helps teams produce accessible products 

[78].

3.2 Findings

Source Summary
WCAG [18] Complex images, which includes graphs, charts, and maps, must include a two-part text alternative. The first part is a short description that identifies the complex image. The second part is a long description that is a textual representation of the essential information. The WCAG 2.0 Guideline 1.1.1 outlines that all non-text contents must have a text alternative to present the equivalent purpose.
Accessibility at Penn State [14] If the data in a complex image is essential, a text description of the image must be provided. In appropriate cases, a numeric table representing the chart data will provide additional accessibility.
Diagram Center [70] Charts and graphs must be converted into accessible tables; a brief description and a summary, if needed, should be provided. Additional information such as the title and axis labels should be included as well. However, visual attributes such as color are not necessary to include unless there is an explicit need for them.
CFPB [78] The alt texts should include one sentence of what the chart is and the chart type. There should also be a link to a CSV or another machine-readable data format with the raw data. Moreover, the data must have descriptive column labels. Take into consideration that screen readers do not let users skip or speed up while reading alt texts.
Table 1: Brief summary of each guideline source.

Table 1 summarizes the four guidelines. These guidelines offer recommendations for generating alt texts to visualizations, including the structure of alt text and which components to include.

Structure of alt text. WCAG recommends two-part alt texts for complex visualizations, with the first part containing a short description of the chart and the second part a more detailed description [18]. WCAG suggests not to include the long description in an alt text. Instead, they propose various ways to incorporate long descriptions, for example, by specifying a link to the long description with an <a> tag adjacent to the chart or using the longdesc attribute. According to WCAG, the short description should also indicate how/where to access the long description.

Concerning length, different guidelines prescribe varying principles. While WCAG [18] encourages longer descriptions as visualizations contain substantial amount of information, other guidelines [70, 14, 78] suggest one or two sentences. For example, CFPB asserts that alt texts must be short yet descriptive since screen readers may not allow users to skip or speed up while reading alt text [78].

Pid Age G. Edu. Occupation Diagnosis Onset Age (year) Light Perception Screen Readers Years Used
P1 22 M H.S. Customer service Retinal detachment 0 N NVDA 5
P2 38 F M.S. Accessibility tester Leber hereditary optic neuropathy 0 N JAWS 20
P3 24 M B.S. Student Juvenile macular degeneration 6 Y VoiceOver 10
P4 22 F H.S. Student Astrocytoma resulting in optic nerve compression 9 N JAWS 11
P5 46 F M.A. Unemployed Albinism, Glaucoma 0 Y JAWS 21
P6 31 F B.A. Writer Retinopathy of prematurity 0 N JAWS 20
P7 26 M B.A. Unemployed Retinopathy of prematurity 5 N JAWS 14
P8 35 F B.A. Customer service Leber hereditary optic neuropathy 0 N JAWS 20
P9 23 F H.S. Student Retinopathy of prematurity 0 N JAWS 12
P10 26 F M.A. Policy analyst Retinopathy of prematurity 0 N JAWS 6
P11 20 M A.A. Developer Leber hereditary optic neuropathy 0 Y VoiceOver 10
P12 31 M B.S. Usher Glaucoma 0 Y JAWS 10
P13 32 F H.S. Information specialist Glaucoma 0 Y JAWS, NVDA 7
P14 25 M H.S. Student Glaucoma and Peters Anomaly 4 N JAWS, NVDA 7
P15 25 F B.S. Student Microphthalmia 0 N VoiceOver 16
P16 26 F J.D. Claims specialist Leber’s congenital amaurosis 0 Y JAWS, NVDA 18
P17 24 F M.A. Speaking employment Leber’s congenital amaurosis 0 Y JAWS 12
P18 31 F B.A. Unemployed Optic Atrophy 0 Y JAWS 21
P19 27 M B.S. Customer service Optic Atrophy 8 N VoiceOver 17
P20 30 F H.S. Student Retinopathy of prematurity 0 Y JAWS 25
P21 29 F M.A. Teacher Genetic mutation 0 Y JAWS 5
P22 37 F M.S. Technology specialist Retinopathy of prematurity 0 N JAWS 25
Table 2: Demographics of participants. Pid=Participant ID. G=Gender (M=Male, F=Female). Edu=Education (H.S.=High School, B.S.=Bachelors of Science, B.A.=Bachelors of Arts, M.A.=Masters of Arts, M.S.=Masters of Science, J.D.=Doctor of Jurisprudence, A.A.=Associates).

Components in alt text. Alt text should provide a meaningful and informative description of a visualization that is sufficient for a reader to understand its content [78]. To support this goal, CFPB and the Diagram Center’s guidelines encourage alt texts to include a one-sentence summary of the chart, chart type, and axis labels [78, 70]. WCAG also suggests mentioning all visually presented scales and values [18]. According to the Diagram Center’s guidelines, visual attributes (e.g., the color of a bar or line types) don’t need to be explained unless there is a special need [70]. Some guidelines also encourage to include a summary of the data trends [70, 78, 18].

Data tables. For complex visualizations, the conventional alt text provided via the alt attribute in HTML may not suffice to provide the information needed. In addition to a text summary, a data table can further enhance accessibility to a visualization. Both CFPB’s and Diagram Center’s guidelines suggest including a link to the data represented in the visualization in a format accessible to screen readers, e.g., CSV or other machine-readable formats [78, 70]. Also, tables should contain descriptive column labels [78].

3.3 Summary

Overall, the guidelines emphasize the need for descriptive and succinct language in alt texts, including information on the chart type, axes, and data trends. Some visual attributes (e.g., color) may be omitted unless there is an explicit reason. Many sources echoed that formatted tables are essential in understanding visualizations, even though they are not a part of alt texts. While these guidelines provide a useful starting point, they lack rationales for why each component should or should not be included. Furthermore, the guidelines do not reference empirical evidence of how they support the needs of people with visual impairments.

4 Phase 2: Analyzing Current Practices

In addition to surveying guidelines, we inspected current practices to further understand the status quo. The goal is to examine whether authors follow the guidelines, and if not, to identify which strategies authors use when formulating alt text for visualizations.

We sought to collect alt text and visualization pairs from online media outlets as this is a common way in which the general public consumes visualizations. We sampled 30 visualizations from three major news outlets (NYT, The Washington Post, FiveThirtyEight) who frequently published visualizations alongside articles. However, we found no alt texts associated with the sampled visualizations (no alt attribute if the visualization was presented as an image element, nor desc tag if the visualization was presented as an SVG element). Hence, we shifted our approach to obtain examples from academic publications. Specifically, we collected alt text and visualization pairs from IEEE VIS & TVCG, ACM ASSETS, and ACM CHI over the last two years (i.e., 2019, 2020). We chose these three publication venues to favor authors with expertise in visualizations and graphics (IEEE VIS & TVCG), with knowledge of accessibility (ACM ASSETS), or authors that are often prompted to write alt texts for visualizations (ACM CHI as the submission system encourages authors to submit alt texts for figures).

4.1 Data collection & data preliminary

We downloaded the academic publications in a PDF format from the IEEE Xplore digital library and ACM digital library. We then converted the PDFs to accessible text containing the embedded alt texts using Adobe Acrobat. Next, we used a custom Python script to identify and extract alt texts from the accessible text. We also downloaded the figures from Semantic scholar and mapped them to the extracted alt texts by matching both the publication DOI and the figure number.

We collected total of 2,278 publications (VIS&TVCG:723, ASSETS:95, CHI:1,460) with 7,493 figures (VIS&TVCG:2,518, ASSETS:281, CHI:4,694). Among those figures, 40% contained alternative text descriptions (VIS&TVCG:0%, ASSETS:65%, CHI:51%). To filter only visualizations (e.g., bar, line, area chart, scatterplot, boxplot etc.) from plain images, one researcher reviewed the figures manually. After filtering, we removed the alt texts that only contain the figure number (e.g., “Figure 1”) or random placeholders (e.g., “abc”), resulting in 0 alt text-visualization pairs from the IEEE VIS & TVCG collection, 89 pairs from the ASSETS collection and 752 pairs from the CHI collection.

For each pair in the aggregated collection, we marked whether the alt texts mentioned summary, chart type, axes, data trends, visual attributes (color and shape), and data points. Two researchers annotated all alt text-visualization pairs independently and discussed them together to resolve disagreements (there were around 5% of disagreements initially). We also calculated the length of each alt text using NLTK’s sentence tokenizer [10].

Figure 1: The frequency of each component that appears in the alt texts from the ASSETS and CHI collections.

4.2 Results

Figure 1 shows the frequency that each component appears in alt texts for the ASSETS and CHI collections. Almost all alt texts include a summary that describes the visualization’s overall topic (e.g., “Input speed of three techniques”). The second most common component for the ASSETS collection was axes information (65 out of 89 alt texts, 73%). In the CHI collection, chart type was mentioned 34% of the time (254 out of 752 alt texts). In both collections, visual attributes (e.g., color and shape) were the least common among all components.

We also analyzed which combination of components was most used when formulating alt texts (Fig. 2). A summary of the visualization together with axes, ticks, and data points was the most common combination (18%) in the ASSETS collection. The second most common combination also included data trends in addition to the summary, axes, ticks, and data points (13%). Finally, alt texts that only provide a summary were the third most common (8%). In the CHI collection, a summary alone was the most common alt text type (48%), followed by a summary with the chart type (12%) and a summary with data points (4%).

Figure 2: The frequency of the top-3 most common combinations of components used in alt texts from the ASSETS and CHI collections.

The average length was 3.5 sentences (Median=3, SD=3.3). Alt texts from ASSETS (M=4.4, SD=2.9) were slightly longer (t=-2.8, p¡0.01) than those in the CHI collection (M=3.5, SD=3.5).

4.3 Summary

The analysis shows that the authors do not entirely follow current guidelines. For example, the authors only mention the actual data points, chart type, and data trends about half of the time in the ASSETS collection and less than a third of the time in the CHI collection. While alt texts from the ASSETS collection are better aligned with the guidelines, only half of the alt texts contain the chart type (e.g., “this chart describes” instead of “this bar chart describes”). Without mentioning the chart type, visually impaired individuals may struggle to imagine the depicted visualization. The CHI collection seriously lacks visualization accessibility since 48% of the time, the only available information is the summary. Length-wise, the collected alt-texts were longer on average than the guidelines often prescribe, but the numbers were highly varied.

All 30 visualizations we sampled from major news outlets did not contain alt texts associated with the visualizations. Since visualizations are a critical part of information consumption, this observation is alarming. Similarly, the IEEE VIS & TVCG collection did not include any text alternatives to the figures. To exclude the possibility that the script couldn’t detect alt texts from the collection, we randomly chose 100 papers and manually checked them through Adobe Acrobat. We urge the IEEE VIS community to employ a system that encourages the use of alt texts to make science more accessible to people with visual impairments.

5 Phase 3: Semi-Structured Interview

To evaluate the guidelines and further understand the needs of people with visual impairments concerning visualization alt texts, we conducted semi-structured interviews. Specifically, the goal of the study is to identify empirical evidence to motivate guidelines and further derive implementation in alt text formulation for visualization. We examined participants’ behaviors in the news consumption scenario, which we believe is one of the frequent encounters of visualizations online for the general public.

5.1 Method

5.1.1 Participants

We solicited study participation by circulating an IRB-approved flyer on listservs hosted by organizations serving the blind and low vision population. Our recruitment criteria were designed to recruit participants who are 1) at least 18-year-old, 2) legally blind, and 3) using screen readers daily. We received 150 responses from potential participants, contacted participants for the interview on a first-come-first-served basis, and recruited participants until we saturated the findings [4]. The final pool of participants consisted of 22 participants (15 female, 7 male), whose ages ranged from 20 to 46 (M=28.6, SD=6.2, Table 2

). Among 22 participants, 21 were blind, and 1 had low vision. All interviews were conducted via Zoom. Each interview lasted on average 60 minutes with a standard deviation of approximately 10 minutes. We compensated their participation with a $20 Visa gift card.

Figure 3: Study stimuli. Each participant saw all of the visualizations parked with a different style of alt text. V1 [17], V2 [63], V3 [74], V4 [75]

5.1.2 Study stimuli

To provide the context of a real-world visualization reading scenario, we prepared four articles from online news covering different topics. We included multiple chart types to maximize participants’ exposure to different components (e.g., bar, line) of the visualization, as well as the number of variables encoded to enable participants to examine additional encodings (e.g., color), to expose them to varying complexities. Figure 3 shows the selected visualizations.

In creating the study stimuli, we shortened the original article text to only one or two paragraphs related to the visualization, so as to conduct the study in a reasonable amount of time (e.g., 1 hour).

Three out of the four visualizations (V2, V3, V4) were in an SVG format. For SVG elements, desc tags serve to place alt texts, equivalent to the alt attribute for an image. However, due to potential comparability issues of desc tags with some screen readers [71], we converted the three visualizations to a bitmap image and added an alt text via the alt attribute.

For each of the four visualizations, we formulated alt texts in four different styles informed by Phase 1. The four styles contain different visualization components and different specificity mentioned in the guidelines to prompt participants to think about their preference when some components and the specificity are present and absent (Fig 3):

  • [leftmargin=3mm]

  • Brief description (Fig. 3a): Since several guidelines from Sec. 3 emphasize to have a short description [18, 78], we formulated a brief description. We first described the type of visualization [78] followed by a summary of the data that the visualization represents.

  • Detailed description (Fig. 3b): A few guidelines encourage to include more detailed aspects of the visualization [18, 70]. In this style, we first described the type of chart followed by a summary, a depiction of visualization elements including axes, range, marks, and mapping. To evaluate the guideline discouraging description of visual attributes (e.g., color, shape) [70], we included them in this style of description, when applicable.

  • Data trends (Fig. 3c): Data trends are another component that many guidelines suggested to include in alt texts [70, 78, 18]. We described the overall trends that were visually apparent.

  • Data points (Fig. 3d): All the guidelines we surveyed highlight the importance of having access to raw data [14, 78, 70, 18]. To validate the emphasis on having a separate table element in the guidelines, we created two different ways to access the data. The first version includes data points within the alt texts, which sequentially read out the raw data. The second version includes the data in a formatted HTML table. This allows the participants to navigate the table with their screen readers. In this case, we indicated that a table could be found below within the alt text.

We formulated four different alt texts styles for each of the four chosen visualizations, resulting in 16 different stimuli. All participants saw all of the four articles paired up with one of each of the alt text styles. We counterbalanced the pairing between the articles and styles, and randomized the order of presenting the articles. Specifically for the Data points style, half of the participants saw the data points in the alt text and the other half saw the data points in an HTML format.

Figure 4: Study procedure. We asked participants their demographic information, conditions, and prior experience. After that, participants examined each visualization once at a time by reading the alt text with their screen readers. After each visualization, we gave them a full description of the visualization and asked them to improve the alt text based on their information needs. After examining all of the visualizations, we asked questions regarding their preferences and needs toward alt texts.

5.1.3 Procedure

Figure 4 shows the overall procedure of the interview. We first asked participants demographic questions, including their age, gender, education level, occupation, as well as their vision conditions, including their diagnosis, onset age, and remaining vision. Then, we asked about their experience with assistive technologies. Specifically, we asked what assistive technologies they have been using and how long they have been using them. We also asked participants how often they encountered visualizations and the situations they encountered them.

Next, participants were prompted to open the URLs presenting the study stimuli that were sent just before the study started through email. At that point, we asked them to share their screen to observe which part of the study stimuli they were interacting with. While the participants interacted with the stimuli, we asked them to express what they were doing every step of the way (think aloud).

We then asked participants what they learned about the visualization and how they learned that information. The example questions include: What do you think that the chart is about? How does the alt-text help you learn about the chart? Besides alt-text, have you used any other strategies to understand the chart? After each visualization, we gave a full description of the visualization, consisting of the detailed description, data trends, and the summary of data points (Fig. 3b, c, d). We then asked the participants how they would regenerate or refine the alt texts and which aspect of the given alt texts was helpful to envision the visualization and the underlying data.

After examining all four visualizations, we asked their preference regarding length, contents, and usefulness of alt texts. Participants were also asked about the role of visualizations in understanding the article, how they compare the alt texts requirements between images and visualizations, and the perfect alt text system that they envision.

5.1.4 Analysis

We first transcribed the recorded study sessions. We conducted a thematic analysis, a method to identify themes within the data [11]. Three researchers coded three participants’ transcripts independently and discussed them together to create a consolidated codebook. Then, we categorized the codes in the codebook based on the emerging themes using affinity diagrams and axial coding. The discussion resulted in 12 themes (e.g., Prior experience, Visual components) and 152 codes. One researcher coded the rest of the transcripts using the codebook.

5.2 Findings

We present the findings from our thematic analysis of the semi-structured interview conducted on 22 participants.

5.2.1 Prior Experience with Visualizations

Context. Thirteen participants stated they read visualizations in academic settings. P20 stated, “I encounter visualizations a few times a week at school.” The participants that read visualizations in an academic setting expressed that visualizations raise questions such as, “What is this showing? What do these numbers represent?” and lack “any descriptive text except for possibly a caption,” as stated by P3.

Participants also found visualizations while seeking information on the web. As stated by P17, “There’s a lot of charts and graphs when looking for information about current events. The charts and graphs show what areas have increased cases and other information.” In addition, we observed that another common situation where visualizations appear is in professional settings. As noted by P6, “I encounter charts a lot when I’m observing survey data,” and also by P8, “When I am at work there’s some charts [that are related to my work].”

Frequency. In terms of frequency, two participants mentioned that they encounter visualizations on a daily basis, nine participants mentioned more than three times a week, and eleven participants mentioned once a week.

Assistive technology. Braille displays and tactile materials are a common medium used by participants to read visualizations, especially in academic settings. 15 out of 22 participants explicitly mentioned using them. For example, P10 stated she has been reading visualizations “through tactile braille, embossed braille, and hardcopy materials.” P11 shared, P11 “for images specifically I use a screen reader. But the main way that I do this [visualization] is through tactile.” Many participants also expressed their familiarity and extended use of braille displays. For example, P15 mentioned that “I read braille, and I’ve been using that my entire life,” and P10 noted that “I have been reading braille since I was in kindergarten, so I’ve been reading braille for a long time.”

We observed that familiarity with tactile visualizations allowed participants to better understand particular visualizations, such as bar charts. As stated by P10, “We’ve grown up learning those concepts and know like what a bar chart is, I know what a histogram is.” However, some uncommon types of charts that they have never experienced with a braille display might lead to confusion, as noted by P15: ‘‘I’ve never seen a bubble chart before.”

5.2.2 The Motivation to Understand Visualizations

The motivation to understand visualizations spanned several reasons, including the access to information not included within the article text, to inspect the claims made in the article, draw independent conclusions, and share the gained information with others.

To probe different aspects of data that are not mentioned in the article text. Participants discussed that the information obtained from reading the visualizations allows them to observe aspects of the data that are not mentioned in the news article. For example, P14 expressed their frustration that “Even though I get a general understanding, I want to know more than what the author of the article wants you to know. What about the Southeast or the Northeast?” This sentiment was also observed from other participants. P17 noted that “The article talks about a couple of different examples, whereas the visualization has far more industries. So if you wanted to look at an industry that wasn’t highlighted in the article, it is provided in the visualization.”

To inspect the claims and draw conclusions. The desire to inspect the claims made within an article was also widely echoed among participants. As P6 commented, “The article mentions that there were some decreases in jobs in particular areas and it was helpful that I could look at the visualization and see that number to fully understand what was being discussed.” Moreover, while inspecting visualizations, participants had the opportunity to confirm the article’s claims or build upon the information given within the article. For example, P17 shared that “Even though I previously read that the trend was increasing gradually overall, after looking at the table, it seems to show that while it is increasing overall, there are some points where the trend flattens out.” P17 also noted that “If the claims were to be not honest, I would look at the data.” These sentiments about authors’ bias and misinformation were observed by other participants. P2 shares that “I am skeptical of any statistic that comes out of nowhere.” When no data is provided, the participants expressed their inability to do inspect the claim and draw their own conclusions. P9 went further and asserted that their goal to read visualizations was to “obtain the data” to draw their own version of conclusions.

To share with others. Participants expressed that they would like to share the information gained from visualizations with others. P14 noted that “I would like the important information in a visualization because I may want to share it with someone sighted and explain what the visualization is conveying.” Within the aspect of sharing the gained information from a visualization, P14 mentioned that “I would like to know the trends if I was to share it with someone else.”

5.2.3 Constructing Mental Model of Visualizations

Participants expressed the desire to construct a mental model of a visualization that is similar to what sighted people would view in order to visually interpret the visualizations. Creating a mental image seemed to enhance participants’ understanding of visualizations. As mentioned by P9, “I want to make sure I understand the visualization and where the data points are, by trying to visualize it.” Similarly, P1 noted that “I imagine the visualization because I like to understand each piece of information.” While constructing a mental picture, many participants had follow-up questions regarding the visualization and its details. For example, after having a follow-up discussion regarding the visualization, P10 shared that “The additional information helped knowing how this graph is laid out and I can visualize what is being described.” Other participants also stated that having supporting details guided them to construct a better landscape of the visualizations. As demonstrated by P18, “I was drawing with my hand to try to visualize what the chart might look like based on the given alternative text.” P12 shared that “If I had more information about the visualization, it would have helped me to mentally visualize what was depicted.”

Overall, participants expressed the desire to know enough visual details about the visualization to imagine them. Many participants connected this aspect to the knowledge that a sighted person would have, such as P6, who noted that “I want to get the same information that a sighted person is getting when looking at a visualization.” Additionally, P21 shared, “I want to imitate the way that sighted people would skim the visualization. I would want to see it the way that a sighted person would think about it from their phenomenological experience. I am visualizing what I am missing that would be a normal thought in a sighted person’s head.” We observed that creating mental visualizations allowed participants to “think like a sighted person when I am reading an alternative text,” as explained by P15. Notably, many participants expressed their frustration with the limitations of creating a mental picture of the visualizations. In P14’s words, “It is difficult to visualize the whole graph in my head.” This sentiment indicates that, although participants attempted to create mental images, constructing and keeping the mental images for long periods is difficult, likely due to the sequential (and non-interactive) nature of audio delivery.

5.2.4 Information Needs for Alt-texts

We learned the participants’ expectations and needs regarding what descriptive information to include in visualization alt texts.

Chart type. 11 out of 22 participants echoed that providing the type of chart is helpful, especially at the beginning of the alt text. Since they learned charts at school, stating the chart type can prepare them to fill in the missing information specific to each chart type. As P10 mentioned, “I think the type of chart is helpful. Is this a pie chart or bar chart, line graph, etc. I think that’s a good start to guide a person to think of and imagine [the visualization].”

Another benefit of starting by stating the chart type is to indicate the beginning of an alt text for the visualization. For example, P11 shared “So [I would give] priority to the type of charts. Just to give you an indication where the chart is. Especially [because] some screen readers only mention ’graphics’ at the end when they read the image.”

Axis, range, & ticks. Information about the axis in a visualization was often highlighted as necessary to understand visualizations and participants often noticed whenever this information was missing (e.g., with brief descriptions). For example, when P17 was prompted to improve the brief description style of alt texts, they stated: “What’s on the X-axis, what’s on the Y-axis. That is for sure.” P5 also said, “I didn’t hear the X and Y-axis. So, that’s a little harder for me to follow.” The range of values in each axis also communicates boundary statistics of the data set depicted by the visualization, which helps form a more concrete picture. As P7 stated, “It [range] gives me the boundaries in the framework of what the chart is telling me.”

We did not observe evidence that participants wish to know tick information. When we mentioned the axes’ increments in the detailed description, no participants pointed out this information to be helpful. Also, while examining the visualization with brief descriptions, insights or tables, which do not contain tick information, no participants noticed their absence.

Data trends. Another important component of alt text for visualizations, as highlighted by 14 out of 22 participants, is the description of data trends. Compared to alt texts for images, communicating visualizations have a purpose, often encapsulated by the visible data trends. As P3 shared, “There’s usually a main point that they’re trying to emphasize, like the trends in a certain data set. So I think that, for charts, unlike images, it’s important for the alternative text to describe the trends that are being demonstrated.” Participants expressed that authors should describe data trends in visualization alt texts. For example, P1 wanted a summary of the data trends when that information was missing from the alt text “’Seven-day average is overall increasing [in stimulus V1].’ That’s kind of the summary I’d put on the map [in stimulus V2].” P9 mentioned that “[alt text should include] a visual description of what is being described and some relevant pieces of data, for example, trends or peaks.” P10 suggested including a “snapshot” of the visualization in alt texts: “A snapshot you can just take a look, and say: okay, that’s the trend.”

In describing data trends, referencing data is essential to provide a concrete picture of the visualization instead of simply describing the pattern. For example, while examining stimulus V4, P1 mentioned, “[In addition to acknowledging the performance gap, I’d like to know] how wide the gap is between boys and girls. For example, this chart shows the difference of 20%.” Specifying the data range can further help comprehend the data trend, as noted by P4: “[I would describe it as an] upward trend increasing right now, or in the past couple of weeks.” When depicting the trend, referencing visual attributes such as color to describe trends would not be ideal as people may not readily make sense of them. For example, when we mentioned “There is a cluster of orange circles positioned relatively top of the chart” as a part of describing V4, P17 shared that “[I would prefer it to] just say English and math instead of being the orange and blue circles.”

Beyond the trends, 4 out of 22 participants also wanted to be aware of visible outliers, spikes, or dips in the data. For example, P9 shared,

“To be able to visualize it, I would definitely add more of those dates, in particular the spikes or [dates] that have zero [cases].” Similarly, when prompted for their ideal alt text, P2 suggested, “I would say a line graph showing the trends with spikes on these certain dates.” P1 wondered, “There are some trends from one to three million and it’s going up, but are there also some spikes or is it going up overall?”

Colors. Participants’ preferences toward information regarding colors were varied. Some participants wanted to know, while others did not. Learning about the colors seemed to be a personal preference. Among those who favored knowing the colors, they were especially curious when the color encoded data and when contextualizing the color scale was not challenging.

To justify their preference, participants often stated that color helps them picture the visualization. It also helps them understand how the visualization uses colors to represent the data. For example, P16 said that “I personally like that the different colors for the positive and negative changes are mentioned. I just think it levels the playing field a bit.” P3 shared, “Those details, like colors, aren’t necessarily needed, but I think it helps to beef up my understanding of what the chart looks like.” On several occasions, participants wanted color information only when it encoded data. For example, when P2 examined V2, she mentioned, “I don’t really care about the colors [green bar]; I just care about how many vaccines are being distributed.” However, she stated while reading V1, “With the rest of the information depicted in the table, all you needed [in the alt text] was a color key.”

The communication of color was especially frowned upon when it encoded a continuous value (V2) using a color gradient schema. In this case, participants seemed to prefer direct access to the values instead of contextualizing them through the color information. As stated by P18, “If the information in the alt texts were to describe the different colors, that would have just been extraneous. It would have been more information than was necessary. It wouldn’t have been necessary because the percentages are there.”

Regardless of their preferences, participants stated that referencing the colors to explain other aspects of data later in the alt text is overwhelming. For example, P17 shared, “I do remember that orange was English and blue was math and then later the alt texts refer to the color to explain something. So I had to scroll back up and be like, wait, which one was that?” To address the cognitive load of the color mapping, participants wished for more intuitive colors that would help them remember the underlying encoding. P5 suggested, “The positive ones were a light blue and the negative ones were a dark blue, but I would use completely different colors. For the negative ones, I would use a different color like red. That would be helpful to remember.”

Data points. All participants favored having access to the underlying data of the visualization. As highlighted by the participants, having access to the data can be beneficial in several ways. It enables participants to interact with the data by seeking out patterns by themselves or finding specific data entries that they might be interested in. P7 shared that “I’m in Michigan so I wanted to see the data, or what color Michigan was. I wouldn’t be able to do that [with the given alt texts].” Having access to the data also enhances trust in the authors’ claims as, if in doubt, the participants would be able to confirm the claims independently. For example, when asked to comment on an alt text describing the data trends, P14 mentioned, “I don’t mind [the data trends], but I would want to see it for myself anyway.”

When given a table, participants overwhelmingly preferred the table formatted by HTML table elements (i.e., th, td, tr tags), since they are easy to navigate using the arrow keys with a screen reader. We observed that all participants who were given the HTML formatted table were able to fluently navigate the table without having any trouble. P20 mentioned, “I think this [HTML formatted] table was really effective. I think the table in some ways was more effective than the other things [textual information].” P6 echoed the importance of the formatted table: “Having it separate from the alternative text is important because I could more easily look at it with my screen reader.”

Participants who were given non-formatted tables inside the alt text stated their frustration for not being able to navigate them. P14 shared, “The problem is that I cannot use my table navigation commands because this is not an actual table element.” Another disadvantage of having a non-formatted table is that participants are not able to keep track of where they are at. In the case of a formatted table, when moving from a different column, screen readers will read the column name, then the value. P14 noted, “I need a table, like a real table, because otherwise I have to constantly remember which column I’m at.”

Regarding the table contents, participants stated their preference for sorted tables with the sorting criteria explicitly stated, since such tables would be easier to parse. For example, P3 mentioned, “The graph is organized from the biggest gain down to zero, and then down to the biggest loss. I like that the table is organized in the same way. If it was out of order, that would be confusing.” P2 added that “The table caption should indicate how the table is sorted.” Clearly defined data columns, potentially with the respective units and a brief description of their meaning or how they are computed, were also deemed necessary to fully understanding the data. P3 stated, “I don’t really understand some of the numbers. [The achievement gaps] are all numbers less than one. I don’t know how those numbers are calculated.”

5.2.5 Style Needs for Alt-texts

Length. The length suggested by the majority of participants ranges from 2 to 8 sentences. For example, P1 shared that “I would say from three to six or sometimes eight sentences. I don’t mind waiting for information. If I want to read it, I read it.” However, P5 asserted that more than six sentences is too long: “I would say, maybe between four and six sentences, because once you get beyond six sentences, it gets to be a lot, and it can be overwhelming and confusing sometimes.” Some participants specifically stated that it depends on the visualization’s complexity and how much information is already covered by the article. For example, P8 mentioned, “That depends on the chart and what’s being described. Sometimes it’s such a busy chart that you’re going to have to write pages of analysis on what’s in it, and it depends on what information is provided in the article as well.” A few participants mentioned they don’t mind the length as long as it has all the information they need. As P6 stated, “I always want things to be more specific if it can be. So I don’t mind if there’s long alt text.”

Language. Most participants prefer plain terms, as exemplified by P2’s comment, “From an accessibility standpoint [I prefer] simpler terms”, and P13’s preference for “more of a simplified type of language so that it’s not so technical, and it makes it easier.” We learned that participants did not understand some technical terms, such as gradients. As P14 shared, “I didn’t exactly know what they mean by the gradient. Maybe, explain it in simpler terms, like the shade.”

Also, we learned that participants prefer an objective tone. P16 shared that “It almost seems like somebody different than the person writing the article should do the alt text.” P17 mentioned that decorating the data trend with adjectives sounds less objective: “It only moved by this much or only decreased by that much, even the word ’only’, to me feels like a little bit of an opinion.”

Order & navigation of alt text. As stated above, participants prefer the chart type to be stated first, together with summary of the visualization. Then, several participants declared their preference for additional detailed information and, more importantly, followed by data. For example, P14 specifically mentioned that “Ideally [I’d like a] brief description and summarization that covers all the vital points, followed by more data.”

Participants mentioned that this type of structure could also serve different needs of people, as stated by P17, “I really liked the structure of having a brief description and then a detailed description so that if somebody didn’t want to look at all that, and they just wanted the basics, they could get that.” In a similar vein, P18 wished for a more structured alt text: “If it had, ’this is the description of this axis, and this is the description of that axis and here’s the description of the bars,’ that might make it a little easier.”

Participants mentioned they wished to have some interactivity. Simple interactivity can be a capability to navigate sentence by sentence. Unlike plain text, some screen readers do not support sentence by sentence reading for alt texts. As P15 stated, “The only thing I hate about the way that alternative text runs is you can’t read it sentence by sentence. Depending on what operating system I’m using, I can’t. I wish you could just like read it sentence by sentence.”

5.2.6 Summary

Our findings demonstrate that participants were trying to visualize visualizations in their head while listening to alt texts. We also observed that participants’ purposes of understanding visualizations were similar to the purposes that sighted people might have while interacting with visualizations. Therefore, alt texts should be designed to support people with visual impairments to mentally visualize the graphics as well as to complete simple visualization tasks.

Aligned with the guidelines, participants expressed the importance of chart type, axes, data trend described in alt texts. Our findings highlight the necessity of providing a range of axes to bind their imagination. The guidelines recommend that it is unnecessary to mention the visual attributes of the visualizations, such as the color, which was found to have an opposing response from the participants, as several preferred being told the colors being used. Tables were one of the most important aspects that participants desired, as the guidelines outline. Participants depended on the accessible tables to complete many tasks, such as retrieving the data and inferring the underlying patterns. However, the importance of providing details in the table caption, such as how a measure is calculated and how the table is sorted, was lacking in the guidelines. Contrary to the majority guidelines, most participants preferred more than two sentences of alt text to ensure that all of the necessary information is available in alt texts, even if they may skip. Unlike images often auxiliary to the surrounding contents in online news, visualization carries the complementary information to the presented texts, having participants wish to have access to all the information.

Given the findings, the current practice we observed from Phase 2 is not sufficient to satisfy participants’ information needs. For example, especially in the CHI collection, lacking data points would prevent participants from extracting any information other than the mere fact that there is a visualization. Also, lacking chart type information would prevent people with visual impairments from imagining anything, even if the alt text contains all other information about visualization.

6 Discussion

6.1 Recommendations for generating alt texts

Through the study, we observed that visually impaired participants want to use visualizations to accomplish similar goals that sighted people would have. We also observed that participants actively try to construct a mental model of the visualization in their heads while listening to alt texts. To support their goals and the construction of an accurate mental picture through alt texts, we suggest formulating alt texts of visualizations by considering the following:

  • [leftmargin=*]

  • Indicate the start of the alt text by mentioning the chart type. The chart type serves as a “template” that provides people a starting point for constructing a mental picture of the visualization, helping them fill in missing details later. If the type of chart is beyond a commonly used one (e.g., bubble chart), briefly explain how it looks compared to a common type (e.g., bar chart, line chart, scatterplot) if possible.

  • Communicate the scope of the data by explaining the axes and their range. Describing the increments of ticks may not be necessary.

  • Explain the visible data trends as if you would provide a “snapshot” of the visualization. When describing trends, reference the data instead of visual attributes to avoid the unnecessary cognitive effort to remember mappings (e.g., use “the English cluster is located below zero” instead of “the orange cluster is located below zero”). Specify the range of values in which the trend is observed to convey a concrete scene (e.g., use “it shows an increasing trend between Jan 3rd to Feb 3rd” instead of “it shows an increasing trend”).

  • To enrich the mental picture of visualizations, optionally provide a brief description of the mapping between the data and visual attributes (e.g., color, shape) if applicable.

  • To accommodate the different amounts of information needs, place a brief description of the visualization first (i.e., chart type with a summary of the topic), then describe it in detail (e.g., axes, range, mapping, trend). Possibly, mention “Details of the visualization are as follows” after the brief description so that those who do not wish to continue listening can skip. The detailed description can be placed in the alt text, the longdesc attribute 222Although longdesc is deprecated, most screen readers can read it. If a new standard for long descriptions is introduced, we recommend people to use it. of an image element, or <desc> tag in an SVG element.

  • Place the data table as a hidden HTML element next to the visualization, and mention at the end of the alt text how to access the data table (e.g., “The data for this chart is available in the table below”). Alternatively, place the data table on a separate page, and add a link to it in the longdesc. Often, visualization designers sort data by a meaningful variable to make the pattern in the visualization more salient. The same principle should apply to the table design. Sorted tables help people recognize the pattern while navigating the table. Tables laid out in the same order as in the visualization also help people envision the depicted chart with less effort. The table should contain a caption explaining its contents, the sorting criteria, and how the various data fields were calculated. The designer can use the well-crafted labels used in the visualization to name the columns. When possible, prioritize the shortest name that conveys a column’s meaning since screen readers repeatedly read the column name when moving between columns. If the table contains too many rows, sample the data and indicate so in the caption. Ideally, stratify the samples based on the critical variables.

  • Use plain language instead of visualization-specific language (e.g., shade vs. gradient) to enhance accessibility. Avoid decorating data (e.g., only 10%, very large gap) to maintain objectivity.

Since some screen readers do not fully support the desc tag of an svg element (equivalent to the alt attribute of an img element), we recommend designers to consider creating a dummy image element below the visualization to add an alt text in the alt attribute.

6.2 Design implication beyond simple alt texts

Given the apparent limitation of textual descriptions, designers could combine other modalities to complement visualization alt text. We envision a scenario where the alt text describes the visualization components first and sonified data trends are played afterward. As P22 shared, “[First explain the visualization components] then you could play an audio version of the graph. Like, as the daily doses would spike, the sound would go up and down in pitch.” Another scenario considers linking the screen reader system with a haptic display to provide tactile feedback when a reader encounters visualizations online. As P5 noted, “Having a way to convert these visualizations into tactile options such as braille along with the audio of the alternative text would help enhance our form of understanding and learning.”

Visual retrieval operations, which sighted people can perform with visualizations, are not fully enabled for people with visual impairments. For example, readers may want to obtain information from their own state while looking at a choropleth representing the hunger rate or may wish to compare the doses administrated between two dates. We observed that people with visual impairments achieved this goal by examining the data table. However, locating the values of interest by navigating a table element takes a long time and imposes a cognitive burden. Dynamically personalizing the alt text or the table contents (e.g., stating data for the readers state, city, or zip code based on their IP address) can partially lower their burden of locating a specific value. Accurate Q&A modules (e.g., [44]) with natural language query capabilities (e.g., [56]) would drastically reduce the burden of retrieving information from visualizations by allowing people with visual impairments to pose the question they want to answer.

While the practice to provide alt text for images has been promoted, the majority of images on the web lack alt texts [72]. In addition to encouraging designers to provide alt texts, the technologies proposed in the visualization community and other fields can also enhance accessibility by automating the alt text generation process. We envision an alt text generation system that automatically detects the type of chart and the underlying data from a rendered image using the models proposed by Savva et al. [67] or Poco and Heer [62] depending on the chart type. For D3 visualizations, the visual components and the data can be extracted by the model proposed by Harper and Agrawala [38]. Data trends can also be extracted by automated techniques [20, 48]. Then, the system can use a natural language generation pipeline (e.g.,  [7, 77]) to formulate alt text from the extracted components, following the proposed guidelines.

6.3 Limitation & future work

Due to technical issues of supporting accessible SVG for some screen readers [71], we rendered all of the visualizations in our study into images. As a result, all readable elements, such as labels, ticks, and textual annotations, were lost. As a next step, we wish to investigate how we could leverage readable elements in an SVG, in addition to alt text, to enhance the understanding of visualizations for people with visual impairments.

In this work, we focus on alt texts for several static charts with varying numbers of variables and encodings, specifically in online document reading scenarios. Future work may expand the scope of the investigation to cover more complex types of visualizations (e.g., uncertainty visualization) and interactive visualizations, as well as other contexts like academic settings, etc. Also, we primarily focused on people without remaining vision. Future work should explore strategies for communicating visualizations for people with low vision.

Finally, the insights gained from our qualitative investigations can also inform quantitative studies to observe how much the recommended alt text composition can help people with visual impairments complete visualization tasks.

7 Conclusion

We analyzed the existing guidelines and current practices for constructing text alternatives for visualizations. We also reported findings from an interview study with 22 people with visual impairments. Our investigation provides insights into how people with visual impairments wish to use visualizations and how they construct an image in their head while listening to alt texts. We identified information needs in visualization alt text to enhance the accessibility of visualizations. Visualizations are a powerful tool to communicate data and their use is pervasive in the media. Thus, ensuring visualization accessibility for people with visual impairments is essential for information equality. We hope that our findings will contribute to the overall accessibility of visualizations.

References

  • [1] W. 2.0 Using longdesc. Note: https://www.w3.org/TR/WCAG20-TECHS/H45.html Cited by: §2.
  • [2] N. Access NVDA. Note: https://www.nvaccess.org/ Cited by: §2.
  • [3] Adobe Add alternate text and supplementary information to tags. Note: https://www.adobe.com/accessibility/products/acrobat/pdf-repair-add-alternative-text.html Cited by: §2.
  • [4] R. Alroobaea and P. J. Mayhew (2014) How many participants are really enough for usability studies?. In 2014 Science and Information Conference, pp. 48–56. Cited by: §5.1.1.
  • [5] Apple VoiceOver user guide. Note: https://support.apple.com/guide/voiceover/welcome/mac Cited by: §2.
  • [6] C. L. Bennett, J. E, M. E. Mott, E. Cutrell, and M. R. Morris (2018) How teens with visual impairments take, edit, and share photos on social media. In Proceedings of the 2018 CHI conference on human factors in computing systems, pp. 1–12. Cited by: §2.2.
  • [7] R. Bernardi, R. Cakici, D. Elliott, A. Erdem, E. Erdem, N. Ikizler-Cinbis, F. Keller, A. Muscat, and B. Plank (2016) Automatic description generation from images: a survey of models, datasets, and evaluation measures.

    Journal of Artificial Intelligence Research

    55, pp. 409–442.
    Cited by: §6.2.
  • [8] T. Berners-Lee and D. Connolly (1995) HTML 2.0 materials. Note: https://www.w3.org/MarkUp/html-spec/ Cited by: §2.
  • [9] J. P. Bigham, C. Jayant, H. Ji, G. Little, A. Miller, R. C. Miller, R. Miller, A. Tatarowicz, B. White, S. White, and T. Yeh (2010) VizWiz: Nearly Real-time Answers to Visual Questions. pp. 10 (en). Cited by: §2.2.
  • [10] S. Bird, E. Klein, and E. Loper (2009) Natural language processing with python: analyzing text with the natural language toolkit. ” O’Reilly Media, Inc.”. Cited by: §4.1.
  • [11] V. Braun and V. Clarke (2006) Using thematic analysis in psychology. Qualitative research in psychology 3 (2), pp. 77–101. Cited by: §5.1.4.
  • [12] R. Burns, S. Carberry, and S. E. Schwartz (2013) Modeling a graph viewer’s effort in recognizing messages conveyed by grouped bar charts. In International Conference on User Modeling, Adaptation, and Personalization, pp. 114–126. Cited by: §2.1.
  • [13] S. K. Card, J. D. Mackinlay, and B. Shneiderman (1999) Readings in information visualization: using vision to think. Morgan Kaufmann. Cited by: §1.
  • [14] (2018-06) Charts & accessibility. Note: https://accessibility.psu.edu/images/charts/ Cited by: §3.1, §3.2, Table 1, 4th item.
  • [15] C. Chen, R. Zhang, E. Koh, S. Kim, S. Cohen, T. Yu, R. Rossi, and R. Bunescu (2019) Figure captioning with reasoning and sequence-level training. arXiv preprint arXiv:1906.02850. Cited by: §2.1.
  • [16] S. H. Choi and B. N. Walker (2010) Digitizer auditory graph: making graphs accessible to the visually impaired. In CHI’10 Extended Abstracts, pp. 3445–3450. Cited by: §2.1.
  • [17] CNBC Here’s where the jobs are - in one chart. Note: https://www.cnbc.com/2021/01/08/where-the-jobs-are-december-2020-chart.html Cited by: Figure 3.
  • [18] Complex images. Note: https://www.w3.org/WAI/tutorials/images/complex/ Cited by: §3.1, §3.2, §3.2, §3.2, Table 1, 1st item, 2nd item, 3rd item, 4th item.
  • [19] M. Corio and G. Lapalme Generation of texts for information graphics. pp. 10 (en). Cited by: §2.1.
  • [20] Z. Cui, S. K. Badam, M. A. Yalçin, and N. Elmqvist (2019) Datasite: proactive visual data exploration with computation of insight-based recommendations. Information Visualization 18 (2), pp. 251–267. Cited by: §6.2.
  • [21] Y. De Haan, S. Kruikemeier, S. Lecheler, G. Smit, and R. Van der Nat (2018) When does an infographic say more than a thousand words? audience evaluations of news visualizations. Journalism Studies 19 (9), pp. 1293–1312. Cited by: §1.
  • [22] S. Demir, D. Oliver, E. Schwartz, S. Elzer, S. Carberry, K. F. Mccoy, and D. Chester (2010) Interactive sight: textual access to simple bar charts. New Review of Hypermedia and Multimedia 16 (3), pp. 245–279. Cited by: §2.1.
  • [23] S. Elzer, E. Schwartz, S. Carberry, D. Chester, S. Demir, and P. Wu (2007) A browser extension for providing visually impaired users access to the content of bar charts on the web.. In WEBIST (2), pp. 59–66. Cited by: §2.1.
  • [24] C. Engel, E. F. Müller, and G. Weber (2019) SVGPlott: An Accessible Tool to Generate Highly Adaptable, Accessible Audio-tactile Charts for and from Blind and Visually Impaired People. Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments, pp. 186–195. External Links: Document, ISBN 978-1-4503-6232-0, Link Cited by: §2.1.
  • [25] C. Engel and G. Weber (2017) Analysis of tactile chart design. ACM International Conference Proceeding Series Part F1285, pp. 197–200. External Links: Document, ISBN 9781450352277 Cited by: §2.1.
  • [26] C. Engel and G. Weber (2017) Analysis of tactile chart design. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, pp. 197–200. Cited by: §2.1.
  • [27] C. Engel and G. Weber (2017) Improve the accessibility of tactile charts. In IFIP Conference on Human-Computer Interaction, pp. 187–195. Cited by: §2.1.
  • [28] C. Engel and G. Weber (2018) A user study to evaluate tactile charts with blind and visually impaired people. In International Conference on Computers Helping People with Special Needs, pp. 177–184. Cited by: §2.1.
  • [29] M. Fasciano and G. Lapalme (1996) PostGraphe: A System for the Generation of Statistical Graphics and Text. In Eighth International Natural Language Generation Workshop, External Links: Link Cited by: §2.1.
  • [30] A. F. for the Blind Screen readers. Note: https://www.afb.org/blindness-and-low-vision/using-technology/assistive-technology-products/screen-readers Cited by: §2.
  • [31] G. Fusco and V. S. Morash (2015) The tactile graphics helper: providing audio clarification for tactile graphics using machine vision. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, pp. 97–106. Cited by: §2.1.
  • [32] J. A. Gardner and V. Bulatov (2006) Scientific diagrams made easy with iveo tm. In International Conference on Computers for Handicapped Persons, pp. 1243–1250. Cited by: §2.1.
  • [33] C. Gleason, P. Carrington, C. Cassidy, M. R. Morris, K. M. Kitani, and J. P. Bigham (2019) “It’s almost like they’re trying to hide it”: How User-Provided Image Descriptions Have Failed to Make Twitter Accessible. In The World Wide Web Conference on - WWW ’19, San Francisco, CA, USA, pp. 549–559 (en). External Links: ISBN 978-1-4503-6674-8, Link, Document Cited by: §2.2.
  • [34] C. Goncu and K. Marriott (2011) GraVVITAS: generic multi-touch presentation of accessible graphics. In IFIP Conference on Human-Computer Interaction, pp. 30–48. Cited by: §2.1.
  • [35] H. Granz, M. Tuccar, S. Purushe, and G. Grinstein (2013) Implementing disability accommodations in a widely distributed web based visualization and analysis platform–weave. In International Conference on Universal Access in Human-Computer Interaction, pp. 31–39. Cited by: §2.1.
  • [36] D. Guinness, A. Muehlbradt, D. Szafir, and S. K. Kane (2019) RoboGraphics: dynamic tactile graphics powered by mobile robots. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility, pp. 318–328. Cited by: §1.
  • [37] M. E. Hahn, C. M. Mueller, and J. L. Gorlewicz (2019) The comprehension of stem graphics via a multisensory tablet electronic device by students with visual impairments. Journal of Visual Impairment & Blindness 113 (5), pp. 404–418. Cited by: §2.1.
  • [38] J. Harper and M. Agrawala (2014) Deconstructing and restyling d3 visualizations. In Proceedings of the 27th annual ACM symposium on User interface software and technology, pp. 253–262. Cited by: §6.2.
  • [39] A. A. Help Get started on android with talkback. Note: https://support.google.com/accessibility/android/answer/6283677?hl=en Cited by: §2.
  • [40] W. Heuten, D. Wichmann, and S. Boll (2006-01) Interactive 3D sonification for the exploration of city maps. Vol. 189. Note: Pages: 164 External Links: Document Cited by: §2.1.
  • [41] M. Hu (2015) Exploring new paradigms for accessible 3d printed graphs. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, pp. 365–366. Cited by: §2.1.
  • [42] K. Kafle, B. Price, S. Cohen, and C. Kanan (2018)

    Dvqa: understanding data visualizations via question answering

    .
    In

    Proceedings of the IEEE conference on computer vision and pattern recognition

    ,
    pp. 5648–5656. Cited by: §2.1.
  • [43] S. E. Kahou, V. Michalski, A. Atkinson, Á. Kádár, A. Trischler, and Y. Bengio (2017) Figureqa: an annotated figure dataset for visual reasoning. arXiv preprint arXiv:1710.07300. Cited by: §2.1.
  • [44] D. H. Kim, E. Hoque, and M. Agrawala (2020) Answering questions about charts and generating visual explanations. In CHI’20, pp. 1–13. Cited by: §6.2.
  • [45] N. Kim, S. Joyner, A. Riegelhuth, and Y. Kim (2021) Accessible visualization: design space, opportunities, and challenges. In Computer Graphics Forum, Vol. 40, pp. 173–188. Cited by: §2.1.
  • [46] H. Kong, Z. Liu, and K. Karahalios (2019) Trust and recall of information across varying degrees of title-visualization misalignment. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–13. Cited by: §1.
  • [47] S. Landau and K. Gourgey (2001) Development of a talking tactile tablet. Information Technology and Disabilities 7 (2). Cited by: §2.1.
  • [48] P. Law, A. Endert, and J. Stasko (2020) Characterizing automated data insights. arXiv preprint arXiv:2008.13060. Cited by: §6.2.
  • [49] J. Lazar, S. Chakraborty, D. Carroll, R. Weir, B. Sizemore, and H. Henderson (2013) Development and evaluation of two prototypes for providing weather map data to blind users through sonification. Journal of Usability Studies 8 (4), pp. 93–110. External Links: ISSN 1931-3357 Cited by: §2.1.
  • [50] Microsoft Add alternative text to a shape, picture, chart, smartart graphic, or other object. Note: https://support.microsoft.com/en-us/topic/add-alternative-text-to-a-shape-picture-chart-smartart-graphic-or-other-object-44989b2a-903c-4d9a-b742-6a75b451c669 Cited by: §2.
  • [51] Microsoft Complete guide to narrator. Note: https://support.microsoft.com/en-us/windows/complete-guide-to-narrator-e4397a0d-ef4f-b386-d8ae-c172f109bdb1 Cited by: §2.
  • [52] V. O. Mittal, S. Roth, J. D. Moore, J. Mattis, and G. Carenini Generating Explanatory Captions for Information Graphics. pp. 8 (en). Cited by: §2.1.
  • [53] P. Moraes, G. Sina, K. McCoy, and S. Carberry (2014) Evaluating the accessibility of line graphs through textual summaries for visually impaired users. ASSETS14 - Proceedings of the 16th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 83–90. External Links: Document, ISBN 9781450327206 Cited by: §2.1.
  • [54] M. R. Morris, J. Johnson, C. L. Bennett, and E. Cutrell (2018) Rich representations of visual content for screen reader users. In Proceedings of the 2018 CHI conference on human factors in computing systems, pp. 1–11. Cited by: §2.2.
  • [55] M. R. Morris, A. Zolyomi, C. Yao, S. Bahram, J. P. Bigham, and S. K. Kane (2016) “With most of it being pictures now, i rarely use it” understanding twitter’s evolving accessibility to blind users. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 5506–5516. Cited by: §2.2.
  • [56] A. Narechania, A. Srinivasan, and J. Stasko (2021-02) NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries. IEEE Transactions on Visualization and Computer Graphics 27 (2), pp. 369–379. External Links: ISSN 1941-0506, Document Cited by: §6.2.
  • [57] J. Obeid and E. Hoque (2020-11) Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model. arXiv:2010.09142 [cs]. Note: arXiv: 2010.09142 External Links: Link Cited by: §2.1.
  • [58] W. H. Organization Global data on visual impairment. Note: https://www.who.int/blindness/publications/globaldata/en/ Cited by: §2.
  • [59] S. Paneels and J. C. Roberts (2009) Review of designs for haptic data visualization. IEEE Transactions on Haptics 3 (2), pp. 119–137. Cited by: §2.1.
  • [60] B. Patnaik, A. Batch, and N. Elmqvist (2018) Information olfactation: harnessing scent to convey data. IEEE transactions on visualization and computer graphics 25 (1), pp. 726–736. Cited by: §2.1.
  • [61] H. Petrie, C. Harrison, and S. Dev (2005) Describing images on the web: a survey of current practice and prospects for the future. Proceedings of Human Computer Interaction International (HCII) 71, pp. 2. Cited by: §2.2.
  • [62] J. Poco and J. Heer (2017) Reverse-engineering visualizations: recovering visual encodings from chart images. In Computer Graphics Forum, Vol. 36, pp. 353–363. Cited by: §6.2.
  • [63] T. W. Post A growing number of americans are going hungry. Note: https://www.washingtonpost.com/graphics/2020/business/hunger-coronavirus-economy/ Cited by: Figure 3.
  • [64] P. Sakhardande, A. Joshi, C. Jadhav, and M. Joshi (2019) Comparing user performance on parallel-tone, parallel-speech, serial-tone and serial-speech auditory graphs. In IFIP Conference on Human-Computer Interaction, pp. 247–266. Cited by: §1, §2.1.
  • [65] E. Salisbury, E. Kamar, and M. R. Morris (2017) Toward Scalable Social Alt Text: Conversational Crowdsourcing as a Tool for Refining Vision-to-Language Technology for the Blind. pp. 10 (en). Cited by: §2.2.
  • [66] E. Salisbury, E. Kamar, and M. Ringel Morris (2018-07) Evaluating and Complementing Vision-to-Language Technology for People who are Blind with Conversational Crowdsourcing. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, Stockholm, Sweden, pp. 5349–5353 (en). External Links: ISBN 978-0-9992411-2-7, Link, Document Cited by: §2.2.
  • [67] M. Savva, N. Kong, A. Chhajta, L. Fei-Fei, M. Agrawala, and J. Heer (2011) ReVision: automated classification, analysis and redesign of chart images. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology - UIST ’11, Santa Barbara, California, USA, pp. 393 (en). External Links: Document, ISBN 978-1-4503-0716-1 Cited by: §6.2.
  • [68] F. Scientific JAWS. Note: https://www.freedomscientific.com/products/software/jaws/ Cited by: §2.
  • [69] A. Sharif and B. Forouraghi (2018) EvoGraphs—a jquery plugin to create web accessible graphs. In 2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC), pp. 1–4. Cited by: §2.1.
  • [70] (2020-10) Specific guidelines - graphs. Note: http://diagramcenter.org/specific-guidelines-e.html Cited by: §3.1, §3.2, §3.2, §3.2, Table 1, 2nd item, 3rd item, 4th item.
  • [71] Stackoverflow Why does a link not take the title of a contained svg as it’s accessible name?. Note: https://stackoverflow.com/questions/63208144/why-does-a-link-not-take-the-title-of-a-contained-svg-as-its-accessible-nam Cited by: §2, §5.1.2, §6.3.
  • [72] A. Stangl, M. R. Morris, and D. Gurari (2020) ”Person, shoes, tree. is the person naked?” what people with vision impairments want in image descriptions. In CHI’20, pp. 1–13. Cited by: §2.2, §2.2, §6.2.
  • [73] M. Taibbi, C. Bernareggi, A. Gerino, D. Ahmetovic, and S. Mascetti (2014) Audiofunctions: eyes-free exploration of mathematical functions on tablets. In International Conference on Computers for Handicapped Persons, pp. 537–544. Cited by: §2.1.
  • [74] T. N. Y. Times See how the vaccine rollout is going in your county and state. Note: https://www.nytimes.com/interactive/2020/us/covid-19-vaccine-doses.html Cited by: Figure 3.
  • [75] T. Upshot Where boys outperform girls in math: rich, white and suburban districts. Note: https://www.nytimes.com/interactive/2018/06/13/upshot/boys-girls-math-reading-tests.html Cited by: Figure 3.
  • [76] O. Vinyals, A. Toshev, S. Bengio, and D. Erhan (2015-06) Show and tell: A neural image caption generator. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp. 3156–3164 (en). External Links: ISBN 978-1-4673-6964-0, Link, Document Cited by: §2.2.
  • [77] O. Vinyals, A. Toshev, S. Bengio, and D. Erhan (2015-06) Show and tell: A neural image caption generator. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3156–3164. External Links: ISSN 1063-6919, Document Cited by: §6.2.
  • [78] D. visualization guidelines - CFPB Design System CFPB design system. Note: https://cfpb.github.io/design-system/guidelines/data-visualization-guidelines Cited by: §3.1, §3.2, §3.2, §3.2, Table 1, 1st item, 3rd item, 4th item.
  • [79] V. Voykinska, S. Azenkot, S. Wu, and G. Leshed (2016) How blind people interact with visual content on social networking services. In Proceedings of the 19th acm conference on computer-supported cooperative work & social computing, pp. 1584–1595. Cited by: §2.2.
  • [80] B. N. Walker and L. M. Mauney (2010) Universal design of auditory graphs: A comparison of sonification mappings for visually impaired and sighted listeners. ACM Transactions on Accessible Computing 2 (3), pp. 1–16. External Links: Document, ISSN 19367228 Cited by: §2.1.
  • [81] S. Wall and S. Brewster (2006) Feeling what you hear. pp. 1123. External Links: Document, ISBN 1595931783 Cited by: §2.1.
  • [82] T. Watanabe and H. Mizukami (2018) Effectiveness of tactile scatter plots: comparison of non-visual data representations. In International Conference on Computers Helping People with Special Needs, pp. 628–635. Cited by: §2.1.
  • [83] L. Watson Using aria to enhance svg accessibility. Note: https://www.tpgi.com/using-aria-enhance-svg-accessibility/ Cited by: §2.
  • [84] (2018-06) Web content accessibility guidelines (wcag) 2.1. W3C. External Links: Link Cited by: §3.1.
  • [85] WebAim Alternative text. Note: https://webaim.org/techniques/alttext/ Cited by: §2.
  • [86] P. Wu, S. Carberry, S. Elzer, and D. Chester (2010) Recognizing the intended message of line graphs. In International Conference on Theory and Application of Diagrams, pp. 220–234. Cited by: §2.1.
  • [87] Q. Wu, P. Wang, C. Shen, A. Dick, and A. Van Den Hengel (2016-06) Ask Me Anything: Free-Form Visual Question Answering Based on Knowledge from External Sources. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 4622–4630 (en). External Links: ISBN 978-1-4673-8851-1, Link, Document Cited by: §2.2.
  • [88] S. Wu, J. Wieland, O. Farivar, and J. Schiller (2017-02) Automatic Alt-text: Computer-generated Image Descriptions for Blind Users on a Social Network Service. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, Portland Oregon USA, pp. 1180–1192 (en). External Links: ISBN 978-1-4503-4335-0, Link, Document Cited by: §2.2.
  • [89] Y. Yang, K. Marriott, M. Butler, C. Goncu, and L. Holloway (2020) Tactile presentation of network data: text, matrix or diagram?. In CHI’20, pp. 1–12. Cited by: §1, §2.1.
  • [90] H. Zhao, C. Plaisant, B. Shneiderman, and J. Lazar (2008-05) Data Sonification for Users with Visual Impairment: A Case Study with Georeferenced Data. ACM Transactions on Computer-Human Interaction 15 (1), pp. 4:1–4:28. External Links: ISSN 1073-0516, Link, Document Cited by: §1, §2.1.
  • [91] Y. Zhao, S. Wu, L. Reynolds, and S. Azenkot (2017) The effect of computer-generated descriptions on photo-sharing experiences of people with visual impairments. Proceedings of the ACM on Human-Computer Interaction 1 (CSCW), pp. 1–22. Cited by: §2.2.