Sequential Coordination of Deep Models for Learning Visual Arithmetic

09/13/2018
by   Eric Crawford, et al.
0

Achieving machine intelligence requires a smooth integration of perception and reasoning, yet models developed to date tend to specialize in one or the other; sophisticated manipulation of symbols acquired from rich perceptual spaces has so far proved elusive. Consider a visual arithmetic task, where the goal is to carry out simple arithmetical algorithms on digits presented under natural conditions (e.g. hand-written, placed randomly). We propose a two-tiered architecture for tackling this problem. The lower tier consists of a heterogeneous collection of information processing modules, which can include pre-trained deep neural networks for locating and extracting characters from the image, as well as modules performing symbolic transformations on the representations extracted by perception. The higher tier consists of a controller, trained using reinforcement learning, which coordinates the modules in order to solve the high-level task. For instance, the controller may learn in what contexts to execute the perceptual networks and what symbolic transformations to apply to their outputs. The resulting model is able to solve a variety of tasks in the visual arithmetic domain, and has several advantages over standard, architecturally homogeneous feedforward networks including improved sample efficiency.

READ FULL TEXT
research
02/15/2023

Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning?

Compositionality is a pivotal property of symbolic reasoning. However, h...
research
03/08/2020

Transferable Task Execution from Pixels through Deep Planning Domain Learning

While robots can learn models to solve many manipulation tasks from raw ...
research
08/24/2022

Deep Symbolic Learning: Discovering Symbols and Rules from Perceptions

Neuro-Symbolic (NeSy) integration combines symbolic reasoning with Neura...
research
04/25/2020

Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning

As a comprehensive indicator of mathematical thinking and intelligence, ...
research
02/09/2018

Two Is Harder To Recognize Than Tom: the Challenge of Visual Numerosity for Deep Learning

In the spirit of Turing test, we design and conduct a set of visual nume...
research
06/06/2016

Integrated perception with recurrent multi-task neural networks

Modern discriminative predictors have been shown to match natural intell...
research
07/19/2021

Improving exploration in policy gradient search: Application to symbolic optimization

Many machine learning strategies designed to automate mathematical tasks...

Please sign up or login with your details

Forgot password? Click here to reset