DeepAI AI Chat
Log In Sign Up

Memory Augmented Recursive Neural Networks

by   Forough Arabshahi, et al.

Recursive neural networks have shown an impressive performance for modeling compositional data compared to their recurrent counterparts. Although recursive neural networks are better at capturing long range dependencies, their generalization performance starts to decay as the test data becomes more compositional and potentially deeper than the training data. In this paper, we present memory-augmented recursive neural networks to address this generalization performance loss on deeper data points. We augment Tree-LSTMs with an external memory, namely neural stacks. We define soft push and pop operations for filling and emptying the memory to ensure that the networks remain end-to-end differentiable. In order to assess the effectiveness of the external memory, we evaluate our model on a neural programming task introduced in the literature called equation verification. Our results indicate that augmenting recursive neural networks with external memory consistently improves the generalization performance on deeper data points compared to the state-of-the-art Tree-LSTM by up to 10


page 1

page 2

page 3

page 4


Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Units

Automated mathematical reasoning is a challenging problem that requires ...

Neural Attention Memory

We propose a novel perspective of the attention mechanism by reinventing...

Recursive Recurrent Nets with Attention Modeling for OCR in the Wild

We present recursive recurrent neural networks with attention modeling (...

Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes

Neural networks augmented with external memory have the ability to learn...

Persistence pays off: Paying Attention to What the LSTM Gating Mechanism Persists

Language Models (LMs) are important components in several Natural Langua...

Towards Formula Translation using Recursive Neural Networks

While it has become common to perform automated translations on natural ...

Code Repositories


Combining Symbolic and Function Evaluation Expressions In Neural Programs

view repo


Tree Stack Memory Units

view repo