What's Different between Visual Question Answering for Machine "Understanding" Versus for Accessibility?

10/26/2022
by   Yang Trista Cao, et al.
8

In visual question answering (VQA), a machine must answer a question given an associated image. Recently, accessibility researchers have explored whether VQA can be deployed in a real-world setting where users with visual impairments learn about their environment by capturing their visual surroundings and asking questions. However, most of the existing benchmarking datasets for VQA focus on machine "understanding" and it remains unclear how progress on those datasets corresponds to improvements in this real-world use case. We aim to answer this question by evaluating discrepancies between machine "understanding" datasets (VQA-v2) and accessibility datasets (VizWiz) by evaluating a variety of VQA models. Based on our findings, we discuss opportunities and challenges in VQA for accessibility and suggest directions for future work.

READ FULL TEXT

page 2

page 7

page 8

research
06/03/2018

On the Flip Side: Identifying Counterexamples in Visual Question Answering

Visual question answering (VQA) models respond to open-ended natural lan...
research
10/11/2021

Beyond Accuracy: A Consolidated Tool for Visual Question Answering Benchmarking

On the way towards general Visual Question Answering (VQA) systems that ...
research
01/10/2020

Visual Question Answering on 360° Images

In this work, we introduce VQA 360, a novel task of visual question answ...
research
04/08/2020

Understanding Knowledge Gaps in Visual Question Answering: Implications for Gap Identification and Testing

Visual Question Answering (VQA) systems are tasked with answering natura...
research
10/08/2019

Modulated Self-attention Convolutional Network for VQA

As new data-sets for real-world visual reasoning and compositional quest...
research
11/27/2020

Point and Ask: Incorporating Pointing into Visual Question Answering

Visual Question Answering (VQA) has become one of the key benchmarks of ...
research
12/16/2016

The VQA-Machine: Learning How to Use Existing Vision Algorithms to Answer New Questions

One of the most intriguing features of the Visual Question Answering (VQ...

Please sign up or login with your details

Forgot password? Click here to reset