Tutorial on Answering Questions about Images with Deep Learning

10/04/2016
by   Mateusz Malinowski, et al.
0

Together with the development of more accurate methods in Computer Vision and Natural Language Understanding, holistic architectures that answer on questions about the content of real-world images have emerged. In this tutorial, we build a neural-based approach to answer questions about images. We base our tutorial on two datasets: (mostly on) DAQUAR, and (a bit on) VQA. With small tweaks the models that we present here can achieve a competitive performance on both datasets, in fact, they are among the best methods that use a combination of LSTM with a global, full frame CNN representation of an image. We hope that after reading this tutorial, the reader will be able to use Deep Learning frameworks, such as Keras and introduced Kraino, to build various architectures that will lead to a further performance improvement on this challenging task.

READ FULL TEXT

page 2

page 15

page 16

page 20

research
02/12/2020

Component Analysis for Visual Question Answering Architectures

Recent research advances in Computer Vision and Natural Language Process...
research
02/19/2020

VQA-LOL: Visual Question Answering under the Lens of Logic

Logical connectives and their implications on the meaning of a natural l...
research
08/27/2020

Visual Question Answering on Image Sets

We introduce the task of Image-Set Visual Question Answering (ISVQA), wh...
research
07/25/2022

Towards Complex Document Understanding By Discrete Reasoning

Document Visual Question Answering (VQA) aims to understand visually-ric...
research
05/13/2019

Quantifying and Alleviating the Language Prior Problem in Visual Question Answering

Benefiting from the advancement of computer vision, natural language pro...
research
04/18/2019

Towards VQA Models that can Read

Studies have shown that a dominant class of questions asked by visually ...
research
03/03/2014

Representing, reasoning and answering questions about biological pathways - various applications

Biological organisms are composed of numerous interconnected biochemical...

Please sign up or login with your details

Forgot password? Click here to reset