A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input

10/01/2014
by   Mateusz Malinowski, et al.
0

We propose a method for automatically answering questions about images by bringing together recent advances from natural language processing and computer vision. We combine discrete reasoning with uncertain predictions by a multi-world approach that represents uncertainty about the perceived world in a bayesian framework. Our approach can handle human questions of high complexity about realistic scenes and replies with range of answer like counts, object classes, instances and lists of them. The system is directly trained from question-answer pairs. We establish a first benchmark for this task that can be seen as a modern attempt at a visual turing test.

READ FULL TEXT

page 3

page 5

page 8

research
05/05/2015

Ask Your Neurons: A Neural-based Approach to Answering Questions about Images

We address a question answering task on real-world images that is set up...
research
06/17/2016

FVQA: Fact-based Visual Question Answering

Visual Question Answering (VQA) has attracted a lot of attention in both...
research
08/27/2020

Visual Question Answering on Image Sets

We introduce the task of Image-Set Visual Question Answering (ISVQA), wh...
research
05/09/2016

Ask Your Neurons: A Deep Learning Approach to Visual Question Answering

We address a question answering task on real-world images that is set up...
research
09/03/2019

Data Interpretation over Plots

Reasoning over plots by question answering (QA) is a challenging machine...
research
02/28/2019

From Visual to Acoustic Question Answering

We introduce the new task of Acoustic Question Answering (AQA) to promot...
research
04/20/2018

Right Answer for the Wrong Reason: Discovery and Mitigation

Exposing the weaknesses of neural models is crucial for improving their ...

Please sign up or login with your details

Forgot password? Click here to reset