Thinking Deeply with Recurrence: Generalizing from Easy to Hard Sequential Reasoning Problems

02/22/2021
by   Avi Schwarzschild, et al.
7

Deep neural networks are powerful machines for visual pattern recognition, but reasoning tasks that are easy for humans may still be difficult for neural models. Humans can extrapolate simple reasoning strategies to solve difficult problems using long sequences of abstract manipulations, i.e., harder problems are solved by thinking for longer. In contrast, the sequential computing budget of feed-forward networks is limited by their depth, and networks trained on simple problems have no way of extending their reasoning capabilities without retraining. In this work, we observe that recurrent networks have the uncanny ability to closely emulate the behavior of non-recurrent deep models, often doing so with far fewer parameters, on both image classification and maze solving tasks. We also explore whether recurrent networks can make the generalization leap from simple problems to hard problems simply by increasing the number of recurrent iterations used as test time. To this end, we show that recurrent networks that are trained to solve simple mazes with few recurrent steps can indeed solve much more complex problems simply by performing additional recurrences during inference.

READ FULL TEXT

page 1

page 4

page 5

page 8

page 12

page 13

page 14

page 15

06/08/2021

Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks

Deep neural networks are powerful machines for visual pattern recognitio...
02/11/2022

End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking

Machine learning systems perform well on pattern matching tasks, but the...
01/22/2021

Solving the Same-Different Task with Convolutional Neural Networks

Deep learning demonstrated major abilities in solving many kinds of diff...
06/30/2022

Learning Iterative Reasoning through Energy Minimization

Deep learning has excelled on complex pattern recognition tasks such as ...
10/04/2019

Few-Shot Abstract Visual Reasoning With Spectral Features

We present an image preprocessing technique capable of improving the per...
04/11/2020

Exploring The Spatial Reasoning Ability of Neural Models in Human IQ Tests

Although neural models have performed impressively well on various tasks...
10/09/2017

full-FORCE: A Target-Based Method for Training Recurrent Networks

Trained recurrent networks are powerful tools for modeling dynamic neura...

Code Repositories