Thinking Deeply with Recurrence: Generalizing from Easy to Hard Sequential Reasoning Problems

02/22/2021
by   Avi Schwarzschild, et al.
7

Deep neural networks are powerful machines for visual pattern recognition, but reasoning tasks that are easy for humans may still be difficult for neural models. Humans can extrapolate simple reasoning strategies to solve difficult problems using long sequences of abstract manipulations, i.e., harder problems are solved by thinking for longer. In contrast, the sequential computing budget of feed-forward networks is limited by their depth, and networks trained on simple problems have no way of extending their reasoning capabilities without retraining. In this work, we observe that recurrent networks have the uncanny ability to closely emulate the behavior of non-recurrent deep models, often doing so with far fewer parameters, on both image classification and maze solving tasks. We also explore whether recurrent networks can make the generalization leap from simple problems to hard problems simply by increasing the number of recurrent iterations used as test time. To this end, we show that recurrent networks that are trained to solve simple mazes with few recurrent steps can indeed solve much more complex problems simply by performing additional recurrences during inference.

READ FULL TEXT

page 1

page 4

page 5

page 8

page 12

page 13

page 14

page 15

research
06/08/2021

Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks

Deep neural networks are powerful machines for visual pattern recognitio...
research
02/11/2022

End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking

Machine learning systems perform well on pattern matching tasks, but the...
research
01/22/2021

Solving the Same-Different Task with Convolutional Neural Networks

Deep learning demonstrated major abilities in solving many kinds of diff...
research
06/30/2022

Learning Iterative Reasoning through Energy Minimization

Deep learning has excelled on complex pattern recognition tasks such as ...
research
11/29/2021

Recurrent Vision Transformer for Solving Visual Reasoning Problems

Although convolutional neural networks (CNNs) showed remarkable results ...
research
10/04/2019

Few-Shot Abstract Visual Reasoning With Spectral Features

We present an image preprocessing technique capable of improving the per...
research
10/09/2017

full-FORCE: A Target-Based Method for Training Recurrent Networks

Trained recurrent networks are powerful tools for modeling dynamic neura...

Please sign up or login with your details

Forgot password? Click here to reset