Evaluating the Stability of Recurrent Neural Models during Training with Eigenvalue Spectra Analysis

05/08/2019
by   Priyadarshini Panda, et al.
20

We analyze the stability of recurrent networks, specifically, reservoir computing models during training by evaluating the eigenvalue spectra of the reservoir dynamics. To circumvent the instability arising in examining a closed loop reservoir system with feedback, we propose to break the closed loop system. Essentially, we unroll the reservoir dynamics over time while incorporating the feedback effects that preserve the overall temporal integrity of the system. We evaluate our methodology for fixed point and time varying targets with least squares regression and FORCE training, respectively. Our analysis establishes eigenvalue spectra (which is, shrinking of spectral circle as training progresses) as a valid and effective metric to gauge the convergence of training as well as the convergence of the chaotic activity of the reservoir toward stable states.

READ FULL TEXT
research
09/16/2020

Improving Delay Based Reservoir Computing via Eigenvalue Analysis

We analyze the reservoir computation capability of the Lang-Kobayashi sy...
research
06/07/2022

Asymptotic Stability in Reservoir Computing

Reservoir Computing is a class of Recurrent Neural Networks with interna...
research
04/09/2020

Populations of Spiking Neurons for Reservoir Computing: Closed Loop Control of a Compliant Quadruped

Compliant robots can be more versatile than traditional robots, but thei...
research
05/14/2021

Hierarchical Architectures in Reservoir Computing Systems

Reservoir computing (RC) offers efficient temporal data processing with ...
research
04/15/2022

Deep learning based closed-loop optimization of geothermal reservoir production

To maximize the economic benefits of geothermal energy production, it is...
research
03/17/2022

Euler State Networks

Inspired by the numerical solution of ordinary differential equations, i...
research
01/13/2021

Reservoir Computers Modal Decomposition and Optimization

The topology of a network associated with a reservoir computer is often ...

Please sign up or login with your details

Forgot password? Click here to reset