ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

03/19/2022
by   Ahmed Masry, et al.
8

Charts are very popular for analyzing data. When exploring charts, people often ask a variety of complex reasoning questions that involve several logical and arithmetic operations. They also commonly refer to visual features of a chart in their questions. However, most existing datasets do not focus on such complex reasoning questions as their questions are template-based and answers come from a fixed-vocabulary. In this work, we present a large-scale benchmark covering 9.6K human-written questions as well as 23.1K questions generated from human-written chart summaries. To address the unique challenges in our benchmark involving visual and logical reasoning over charts, we present two transformer-based models that combine visual features and the data table of the chart in a unified way to answer questions. While our models achieve the state-of-the-art results on the previous datasets as well as on our benchmark, the evaluation also reveals several challenges in answering complex reasoning questions.

READ FULL TEXT

page 13

page 14

page 17

research
09/03/2019

Data Interpretation over Plots

Reasoning over plots by question answering (QA) is a challenging machine...
research
11/09/2015

Explicit Knowledge-based Reasoning for Visual Question Answering

We describe a method for visual question answering which is capable of r...
research
05/01/2019

Declarative Question Answering over Knowledge Bases containing Natural Language Text with Answer Set Programming

While in recent years machine learning (ML) based approaches have been t...
research
10/12/2022

OpenCQA: Open-ended Question Answering with Charts

Charts are very popular to analyze data and convey important insights. P...
research
07/29/2019

V-PROM: A Benchmark for Visual Reasoning Using Visual Progressive Matrices

One of the primary challenges faced by deep learning is the degree to wh...
research
09/23/2019

Non-monotonic Logical Reasoning Guiding Deep Learning for Explainable Visual Question Answering

State of the art algorithms for many pattern recognition problems rely o...
research
01/13/2018

Benchmark Visual Question Answer Models by using Focus Map

Inferring and Executing Programs for Visual Reasoning proposes a model f...

Please sign up or login with your details

Forgot password? Click here to reset