A fast memoryless predictive algorithm in a chain of recurrent neural networks

10/05/2020
by   Boris Rubinstein, et al.
0

In the recent publication (arxiv:2007.08063v2 [cs.LG]) a fast prediction algorithm for a single recurrent network (RN) was suggested. In this manuscript we generalize this approach to a chain of RNs and show that it can be implemented in natural neural systems. When the network is used recursively to predict sequence of values the proposed algorithm does not require to store the original input sequence. It increases robustness of the new approach compared to the standard moving/expanding window predictive procedure. We consider requirements on trained networks that allow to implement the proposed algorithm and discuss them in the neuroscience context.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/27/2021

On a novel training algorithm for sequence-to-sequence predictive recurrent networks

Neural networks mapping sequences to sequences (seq2seq) lead to signifi...
research
06/09/2022

It's a super deal – train recurrent network on noisy data and get smooth prediction free

Recent research demonstrate that prediction of time series by predictive...
research
02/09/2018

Predictive Neural Networks

Recurrent neural networks are a powerful means to cope with time series....
research
08/18/2017

Accelerating recurrent neural network training using sequence bucketing and multi-GPU data parallelization

An efficient algorithm for recurrent neural network training is presente...
research
09/28/2020

Distillation of Weighted Automata from Recurrent Neural Networks using a Spectral Approach

This paper is an attempt to bridge the gap between deep learning and gra...
research
06/21/2012

A biological gradient descent for prediction through a combination of STDP and homeostatic plasticity

Identifying, formalizing and combining biological mechanisms which imple...
research
05/15/2018

Characterizing the Temporal Dynamics of Information in Visually Guided Predictive Control Using LSTM Recurrent Neural Networks

Theories for visually guided action account for online control in the pr...

Please sign up or login with your details

Forgot password? Click here to reset