AI Chat AI Image Generator AI Video Text to Speech

Benchmarking of LSTM Networks

08/11/2015

∙

by Thomas M. Breuel, et al.

∙

∙

LSTM (Long Short-Term Memory) recurrent neural networks have been highly successful in a number of application areas. This technical report describes the use of the MNIST and UW3 databases for benchmarking LSTM networks and explores the effect of different architectural and hyperparameter choices on performance. Significant findings include: (1) LSTM performance depends smoothly on learning rates, (2) batching and momentum has no significant effect on performance, (3) softmax training outperforms least square training, (4) peephole units are not useful, (5) the standard non-linearities (tanh and sigmoid) perform best, (6) bidirectional training combined with CTC performs better than other methods.

page 11

page 15

research

∙ 05/08/2020

Sentiment Analysis Using Simplified Long Short-term Memory Recurrent Neural Networks

LSTM or Long Short Term Memory Networks is a specific type of Recurrent ...

39 Karthik Gopalakrishnan, et al. ∙

research

∙ 07/14/2017

Simplified Long Short-term Memory Recurrent Neural Networks: part I

We present five variants of the standard Long Short-term Memory (LSTM) r...

0 Atra Akandeh, et al. ∙

research

∙ 01/18/2019

Slim LSTM networks: LSTM_6 and LSTM_C6

We have shown previously that our parameter-reduced variants of Long Sho...

0 Atra Akandeh, et al. ∙

research

∙ 07/19/2019

Benchmarking a Catchment-Aware Long Short-Term Memory Network (LSTM) for Large-Scale Hydrological Modeling

Regional rainfall-runoff modeling is an old but still mostly out-standin...

0 Frederik Kratzert, et al. ∙

research

∙ 10/30/2019

Neural networks trained with WiFi traces to predict airport passenger behavior

The use of neural networks to predict airport passenger activity choices...

0 Federico Orsini, et al. ∙

research

∙ 10/08/2021

Kinematically consistent recurrent neural networks for learning inverse problems in wave propagation

Although machine learning (ML) is increasingly employed recently for mec...

0 Wrik Mallik, et al. ∙

research

∙ 09/23/2021

LSTM Hyper-Parameter Selection for Malware Detection: Interaction Effects and Hierarchical Selection Approach

Long-Short-Term-Memory (LSTM) networks have shown great promise in artif...

0 Mohit Sewak, et al. ∙