Recurrent Neural Networks with External Memory for Language Understanding

05/31/2015
by   Baolin Peng, et al.
0

Recurrent Neural Networks (RNNs) have become increasingly popular for the task of language understanding. In this task, a semantic tagger is deployed to associate a semantic label to each word in an input sequence. The success of RNN may be attributed to its ability to memorize long-term dependence that relates the current-time semantic label prediction to the observations many time instances away. However, the memory capacity of simple RNNs is limited because of the gradient vanishing and exploding problem. We propose to use an external memory to improve memorization capability of RNNs. We conducted experiments on the ATIS dataset, and observed that the proposed model was able to achieve the state-of-the-art results. We compare our proposed model with alternative models and report analysis results that may provide insights for future research.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/06/2017

Label-Dependencies Aware Recurrent Neural Networks

In the last few years, Recurrent Neural Networks (RNNs) have proved effe...
research
03/07/2017

Linguistic Knowledge as Memory for Recurrent Neural Networks

Training recurrent neural networks to model long term dependencies is di...
research
02/18/2020

Assessing the Memory Ability of Recurrent Neural Networks

It is known that Recurrent Neural Networks (RNNs) can remember, in their...
research
06/20/2017

Effective Spoken Language Labeling with Deep Recurrent Neural Networks

Understanding spoken language is a highly complex problem, which can be ...
research
01/03/2021

Recoding latent sentence representations – Dynamic gradient-based activation modification in RNNs

In Recurrent Neural Networks (RNNs), encoding information in a suboptima...
research
02/24/2016

Toward Mention Detection Robustness with Recurrent Neural Networks

One of the key challenges in natural language processing (NLP) is to yie...
research
03/09/2017

DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks

3D scene understanding is important for robots to interact with the 3D w...

Please sign up or login with your details

Forgot password? Click here to reset