Memory-Loss is Fundamental for Stability and Distinguishes the Echo State Property Threshold in Reservoir Computing Beyond

01/03/2020
by   G Manjunath, et al.
0

Reservoir computing, a highly successful neuromorphic computing scheme used to filter, predict, classify temporal inputs, has entered an era of microchips for several other engineering and biological applications. A basis for reservoir computing is memory-loss or the echo state property. It is an open problem on how design parameters of the reservoir can be optimized to maximize reservoir freedom to map an input robustly and yet have its close-by-variants represented in the reservoir differently. We present a framework to analyze stability due to input and parameter perturbations and make a surprising fundamental conclusion, that the echo state property is equivalent to robustness to input in any nonlinear recurrent neural network that may or may not be in the gambit of reservoir computing. Further, backed by theoretical conclusions, we define and find the difficult-to-describe input specific edge-of-criticality or the echo state property threshold, which defines the boundary between parameter related stability and instability.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/07/2022

Asymptotic Stability in Reservoir Computing

Reservoir Computing is a class of Recurrent Neural Networks with interna...
research
08/05/2023

Edge of stability echo state networks

In this paper, we propose a new Reservoir Computing (RC) architecture, c...
research
09/16/2022

Transport in reservoir computing

Reservoir computing systems are constructed using a driven dynamical sys...
research
03/20/2017

Empirical Analysis of the Necessary and Sufficient Conditions of the Echo State Property

The Echo State Network (ESN) is a specific recurrent network, which has ...
research
11/27/2018

Chasing the Echo State Property

Reservoir Computing (RC) provides an efficient way for designing dynamic...
research
10/06/2022

Biological neurons act as generalization filters in reservoir computing

Reservoir computing is a machine learning paradigm that transforms the t...
research
09/01/2014

Towards a Calculus of Echo State Networks

Reservoir computing is a recent trend in neural networks which uses the ...

Please sign up or login with your details

Forgot password? Click here to reset