Tensorized Transformer for Dynamical Systems Modeling

06/05/2020
by   Anna Shalova, et al.
6

The identification of nonlinear dynamics from observations is essential for the alignment of the theoretical ideas and experimental data. The last, in turn, is often corrupted by the side effects and noise of different natures, so probabilistic approaches could give a more general picture of the process. At the same time, high-dimensional probabilities modeling is a challenging and data-intensive task. In this paper, we establish a parallel between the dynamical systems modeling and language modeling tasks. We propose a transformer-based model that incorporates geometrical properties of the data and provide an iterative training algorithm allowing the fine-grid approximation of the conditional probabilities of high-dimensional dynamical systems.

READ FULL TEXT

page 2

page 6

page 8

research
02/10/2020

Deep Representation Learning for Dynamical Systems Modeling

Proper states' representations are the key to the successful dynamics mo...
research
11/11/2020

Application of Adaptive Multilevel Splitting to High-Dimensional Dynamical Systems

Stochastic nonlinear dynamical systems can undergo rapid transitions rel...
research
09/09/2022

Deriving dynamical systems for language based on the Tolerance Principle

In this research note, I derive explicit dynamical systems for language ...
research
01/31/2023

Continuous Spatiotemporal Transformers

Modeling spatiotemporal dynamical systems is a fundamental challenge in ...
research
07/25/2022

Dimension of Activity in Random Neural Networks

Neural networks are high-dimensional nonlinear dynamical systems that pr...
research
07/16/2021

Estimating covariant Lyapunov vectors from data

Covariant Lyapunov vectors (CLVs) characterize the directions along whic...
research
08/25/2023

In-context learning for model-free system identification

In traditional system identification, we estimate a model of an unknown ...

Please sign up or login with your details

Forgot password? Click here to reset