Visual Question Answering as a Meta Learning Task

11/22/2017
by   Damien Teney, et al.
0

The predominant approach to Visual Question Answering (VQA) demands that the model represents within its weights all of the information required to answer any question about any image. Learning this information from any real training set seems unlikely, and representing it in a reasonable number of weights doubly so. We propose instead to approach VQA as a meta learning task, thus separating the question answering method from the information required. At test time, the method is provided with a support set of example questions/answers, over which it reasons to resolve the given question. The support set is not fixed and can be extended without retraining, thereby expanding the capabilities of the model. To exploit this dynamically provided information, we adapt a state-of-the-art VQA model with two techniques from the recent meta learning literature, namely prototypical networks and meta networks. Experiments demonstrate the capability of the system to learn to produce completely novel answers (i.e. never seen during training) from examples provided at test time. In comparison to the existing state of the art, the proposed method produces qualitatively distinct results with higher recall of rare answers, and a better sample efficiency that allows training with little initial data. More importantly, it represents an important step towards vision-and-language methods that can learn and reason on-the-fly.

READ FULL TEXT
research
04/05/2019

Actively Seeking and Learning from Live Data

One of the key limitations of traditional machine learning methods is th...
research
09/26/2019

Overcoming Data Limitation in Medical Visual Question Answering

Traditional approaches for Visual Question Answering (VQA) require large...
research
02/21/2021

Learning Compositional Representation for Few-shot Visual Question Answering

Current methods of Visual Question Answering perform well on the answers...
research
12/16/2016

The VQA-Machine: Learning How to Use Existing Vision Algorithms to Answer New Questions

One of the most intriguing features of the Visual Question Answering (VQ...
research
07/17/2017

Visual Question Answering with Memory-Augmented Networks

This paper exploits a memory-augmented neural network to predict accurat...
research
11/11/2019

Meta Answering for Machine Reading

We investigate a framework for machine reading, inspired by real world i...
research
11/20/2018

VQA with no questions-answers training

Methods for teaching machines to answer visual questions have made signi...

Please sign up or login with your details

Forgot password? Click here to reset