Dynamic Inference with Neural Interpreters

10/12/2021
by   Nasim Rahaman, et al.
14

Modern neural network architectures can leverage large amounts of data to generalize well within the training distribution. However, they are less capable of systematic generalization to data drawn from unseen but related distributions, a feat that is hypothesized to require compositional reasoning and reuse of knowledge. In this work, we present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules, which we call functions. Inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. The proposed architecture can flexibly compose computation along width and depth, and lends itself well to capacity extension after training. To demonstrate the versatility of Neural Interpreters, we evaluate it in two distinct settings: image classification and visual abstract reasoning on Raven Progressive Matrices. In the former, we show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner. In the latter, we find that Neural Interpreters are competitive with respect to the state-of-the-art in terms of systematic generalization

READ FULL TEXT

page 2

page 7

page 16

page 17

page 21

research
09/14/2023

Dynamic MOdularized Reasoning for Compositional Structured Explanation Generation

Despite the success of neural models in solving reasoning tasks, their c...
research
06/11/2022

A Benchmark for Compositional Visual Reasoning

A fundamental component of human vision is our ability to parse complex ...
research
08/03/2021

Generalization in Multimodal Language Learning from Simulation

Neural networks can be powerful function approximators, which are able t...
research
11/14/2019

Attention on Abstract Visual Reasoning

Attention mechanisms have been boosting the performance of deep learning...
research
07/17/2023

Does Visual Pretraining Help End-to-End Reasoning?

We aim to investigate whether end-to-end learning of visual reasoning ca...
research
09/15/2023

D3: Data Diversity Design for Systematic Generalization in Visual Question Answering

Systematic generalization is a crucial aspect of intelligence, which ref...
research
05/02/2020

Obtaining Faithful Interpretations from Compositional Neural Networks

Neural module networks (NMNs) are a popular approach for modeling compos...

Please sign up or login with your details

Forgot password? Click here to reset