DA-LSTM: A Long Short-Term Memory with Depth Adaptive to Non-uniform Information Flow in Sequential Data

01/18/2019
by   Yifeng Zhang, et al.
0

Much sequential data exhibits highly non-uniform information distribution. This cannot be correctly modeled by traditional Long Short-Term Memory (LSTM). To address that, recent works have extended LSTM by adding more activations between adjacent inputs. However, the approaches often use a fixed depth, which is at the step of the most information content. This one-size-fits-all worst-case approach is not satisfactory, because when little information is distributed to some steps, shallow structures can achieve faster convergence and consume less computation resource. In this paper, we develop a Depth-Adaptive Long Short-Term Memory (DA-LSTM) architecture, which can dynamically adjust the structure depending on information distribution without prior knowledge. Experimental results on real-world datasets show that DA-LSTM costs much less computation resource and substantially reduce convergence time by 41.78% and 46.01 %, compared with Stacked LSTM and Deep Transition LSTM, respectively.

READ FULL TEXT
research
09/16/2015

Guiding Long-Short Term Memory for Image Caption Generation

In this work we focus on the problem of image caption generation. We pro...
research
10/15/2022

Extreme-Long-short Term Memory for Time-series Prediction

The emergence of Long Short-Term Memory (LSTM) solves the problems of va...
research
09/23/2019

Field typing for improved recognition on heterogeneous handwritten forms

Offline handwriting recognition has undergone continuous progress over t...
research
06/21/2021

Long short-term relevance learning

To incorporate prior knowledge as well as measurement uncertainties in t...
research
01/27/2022

Micro-level Reserving for General Insurance Claims using a Long Short-Term Memory Network

Detailed information about individual claims are completely ignored when...
research
12/30/2022

Deep Recurrent Learning Through Long Short Term Memory and TOPSIS

Enterprise resource planning (ERP) software brings resources, data toget...
research
10/13/2022

Meta-learning Based Short-Term Passenger Flow Prediction for Newly-Operated Urban Rail Transit Stations

Accurate short-term passenger flow prediction in urban rail transit stat...

Please sign up or login with your details

Forgot password? Click here to reset