Neural Sequence-to-grid Module for Learning Symbolic Rules

01/13/2021
by   Segwang Kim, et al.
7

Logical reasoning tasks over symbols, such as learning arithmetic operations and computer program evaluations, have become challenges to deep learning. In particular, even state-of-the-art neural networks fail to achieve out-of-distribution (OOD) generalization of symbolic reasoning tasks, whereas humans can easily extend learned symbolic rules. To resolve this difficulty, we propose a neural sequence-to-grid (seq2grid) module, an input preprocessor that automatically segments and aligns an input sequence into a grid. As our module outputs a grid via a novel differentiable mapping, any neural network structure taking a grid input, such as ResNet or TextCNN, can be jointly trained with our module in an end-to-end fashion. Extensive experiments show that neural networks having our module as an input preprocessor achieve OOD generalization on various arithmetic and algorithmic problems including number sequence prediction problems, algebraic word problems, and computer program evaluation problems while other state-of-the-art sequence transduction models cannot. Moreover, we verify that our module enhances TextCNN to solve the bAbI QA tasks without external memory.

READ FULL TEXT

page 2

page 6

research
06/29/2023

A Hybrid System for Systematic Generalization in Simple Arithmetic Problems

Solving symbolic reasoning problems that require compositionality and sy...
research
05/19/2018

Number Sequence Prediction Problems and Computational Powers of Neural Network Models

Inspired by number series tests to measure human intelligence, we sugges...
research
11/16/2015

Neural Programmer: Inducing Latent Programs with Gradient Descent

Deep neural networks have achieved impressive supervised classification ...
research
11/19/2021

Solving Visual Analogies Using Neural Algorithmic Reasoning

We consider a class of visual analogical reasoning problems that involve...
research
02/19/2022

Do Transformers use variable binding?

Increasing the explainability of deep neural networks (DNNs) requires ev...
research
07/06/2022

Transformers discover an elementary calculation system exploiting local attention and grid-like problem representation

Mathematical reasoning is one of the most impressive achievements of hum...
research
06/03/2019

Deep Reasoning Networks: Thinking Fast and Slow

We introduce Deep Reasoning Networks (DRNets), an end-to-end framework t...

Please sign up or login with your details

Forgot password? Click here to reset