Initializing LSTM internal states via manifold learning

04/27/2021
by   Felix P. Kemeth, et al.
0

We present an approach, based on learning an intrinsic data manifold, for the initialization of the internal state values of LSTM recurrent neural networks, ensuring consistency with the initial observed input data. Exploiting the generalized synchronization concept, we argue that the converged, "mature" internal states constitute a function on this learned manifold. The dimension of this manifold then dictates the length of observed input time series data required for consistent initialization. We illustrate our approach through a partially observed chemical model system, where initializing the internal LSTM states in this fashion yields visibly improved performance. Finally, we show that learning this data manifold enables the transformation of partially observed dynamics into fully observed ones, facilitating alternative identification paths for nonlinear dynamical systems.

READ FULL TEXT
research
10/27/2021

Learning Stable Deep Dynamics Models for Partially Observed or Delayed Dynamical Systems

Learning how complex dynamical systems evolve over time is a key challen...
research
04/05/2019

Short note on the behavior of recurrent neural network for noisy dynamical system

The behavior of recurrent neural network for the data-driven simulation ...
research
10/31/2019

A Dynamically Controlled Recurrent Neural Network for Modeling Dynamical Systems

This work proposes a novel neural network architecture, called the Dynam...
research
02/28/2022

On the sample complexity of stabilizing linear dynamical systems from data

Learning controllers from data for stabilizing dynamical systems typical...
research
05/01/2023

Autoencoders for discovering manifold dimension and coordinates in data from complex dynamical systems

While many phenomena in physics and engineering are formally high-dimens...
research
06/24/2021

Physics perception in sloshing scenes with guaranteed thermodynamic consistency

Physics perception very often faces the problem that only limited data o...

Please sign up or login with your details

Forgot password? Click here to reset