A memory enhanced LSTM for modeling complex temporal dependencies

10/25/2019
by   Sneha Aenugu, et al.
0

In this paper, we present Gamma-LSTM, an enhanced long short term memory (LSTM) unit, to enable learning of hierarchical representations through multiple stages of temporal abstractions. Gamma memory, a hierarchical memory unit, forms the central memory of Gamma-LSTM with gates to regulate the information flow into various levels of hierarchy, thus providing the unit with a control to pick the appropriate level of hierarchy to process the input at a given instant of time. We demonstrate better performance of Gamma-LSTM model regular and stacked LSTMs in two settings (pixel-by-pixel MNIST digit classification and natural language inference) placing emphasis on the ability to generalize over long sequences.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/28/2015

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Because of their superior ability to preserve sequence information over ...
research
05/17/2019

Predicting Solar Flares Using a Long Short-Term Memory Network

We present a long short-term memory (LSTM) network for predicting whethe...
research
09/11/2019

Distanced LSTM: Time-Distanced Gates in Long Short-Term Memory Models for Lung Cancer Detection

The field of lung nodule detection and cancer prediction has been rapidl...
research
06/08/2018

Towards Binary-Valued Gates for Robust LSTM Training

Long Short-Term Memory (LSTM) is one of the most widely used recurrent s...
research
11/06/2018

Evaluating the Ability of LSTMs to Learn Context-Free Grammars

While long short-term memory (LSTM) neural net architectures are designe...
research
10/29/2018

Learning Distributed Representations of Symbolic Structure Using Binding and Unbinding Operations

Widely used recurrent units, including Long-short Term Memory (LSTM) and...
research
06/18/2018

Semi-tied Units for Efficient Gating in LSTM and Highway Networks

Gating is a key technique used for integrating information from multiple...

Please sign up or login with your details

Forgot password? Click here to reset