DeepAI
Log In Sign Up

Check It Again: Progressive Visual Question Answering via Visual Entailment

06/08/2021
by   Qingyi Si, et al.
0

While sophisticated Visual Question Answering models have achieved remarkable success, they tend to answer questions only according to superficial correlations between question and answer. Several recent approaches have been developed to address this language priors problem. However, most of them predict the correct answer according to one best output without checking the authenticity of answers. Besides, they only explore the interaction between image and question, ignoring the semantics of candidate answers. In this paper, we propose a select-and-rerank (SAR) progressive framework based on Visual Entailment. Specifically, we first select the candidate answers relevant to the question or the image, then we rerank the candidate answers by a visual entailment task, which verifies whether the image semantically entails the synthetic statement of the question and each candidate answer. Experimental results show the effectiveness of our proposed framework, which establishes a new state-of-the-art accuracy on VQA-CP v2 with a 7.55

READ FULL TEXT

page 1

page 3

page 8

10/20/2016

Proposing Plausible Answers for Open-ended Visual Question Answering

Answering open-ended questions is an essential capability for any intell...
05/24/2019

Self-Critical Reasoning for Robust Visual Question Answering

Visual Question Answering (VQA) deep-learning systems tend to capture su...
11/01/2016

Solving Visual Madlibs with Multiple Cues

This paper presents an approach for answering fill-in-the-blank multiple...
08/12/2019

Why Does a Visual Question Have Different Answers?

Visual question answering is the task of returning the answer to a quest...
09/04/2002

The partition semantics of questions, syntactically

Groenendijk and Stokhof (1984, 1996; Groenendijk 1999) provide a logical...
10/21/2019

Enforcing Reasoning in Visual Commonsense Reasoning

The task of Visual Commonsense Reasoning is extremely challenging in the...
07/04/2019

Answer Extraction for Why Arabic Questions Answering Systems: EWAQ

With the increasing amount of web information, questions answering syste...