Efficient LSTM Training with Eligibility Traces

09/30/2022
by   Michael Hoyer, et al.
0

Training recurrent neural networks is predominantly achieved via backpropagation through time (BPTT). However, this algorithm is not an optimal solution from both a biological and computational perspective. A more efficient and biologically plausible alternative for BPTT is e-prop. We investigate the applicability of e-prop to long short-term memorys (LSTMs), for both supervised and reinforcement learning (RL) tasks. We show that e-prop is a suitable optimization algorithm for LSTMs by comparing it to BPTT on two benchmarks for supervised learning. This proves that e-prop can achieve learning even for problems with long sequences of several hundred timesteps. We introduce extensions that improve the performance of e-prop, which can partially be applied to other network architectures. With the help of these extensions we show that, under certain conditions, e-prop can outperform BPTT for one of the two benchmarks for supervised learning. Finally, we deliver a proof of concept for the integration of e-prop to RL in the domain of deep recurrent Q-learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/28/2022

Short-Term Plasticity Neurons Learning to Learn and Forget

Short-term plasticity (STP) is a mechanism that stores decaying memories...
research
09/10/2015

Recurrent Reinforcement Learning: A Hybrid Approach

Successful applications of reinforcement learning in real-world problems...
research
12/17/2015

An Empirical Comparison of Neural Architectures for Reinforcement Learning in Partially Observable Environments

This paper explores the performance of fitted neural Q iteration for rei...
research
07/24/2020

Online Spatio-Temporal Learning in Deep Neural Networks

Biological neural networks are equipped with an inherent capability to c...
research
11/23/2022

Lempel-Ziv Networks

Sequence processing has long been a central area of machine learning res...
research
08/04/2018

MCRM: Mother Compact Recurrent Memory A Biologically Inspired Recurrent Neural Network Architecture

LSTMs and GRUs are the most common recurrent neural network architecture...
research
11/20/2019

Avoiding Jammers: A Reinforcement Learning Approach

This paper investigates the anti-jamming performance of a cognitive rada...

Please sign up or login with your details

Forgot password? Click here to reset