Learning low-dimensional state embeddings and metastable clusters from time series data

06/01/2019
by   Yifan Sun, et al.
0

This paper studies how to find compact state embeddings from high-dimensional Markov state trajectories, where the transition kernel has a small intrinsic rank. In the spirit of diffusion map, we propose an efficient method for learning a low-dimensional state embedding and capturing the process's dynamics. This idea also leads to a kernel reshaping method for more accurate nonparametric estimation of the transition function. State embedding can be used to cluster states into metastable sets, thereby identifying the slow dynamics. Sharp statistical error bounds and misclassification rate are proved. Experiment on a simulated dynamical system shows that the state clustering method indeed reveals metastable structures. We also experiment with time series generated by layers of a Deep-Q-Network when playing an Atari game. The embedding method identifies game states to be similar if they share similar future events, even though their raw data are far different.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

page 8

04/18/2019

A kernel-based method for coarse graining complex dynamical systems

We present a novel kernel-based machine learning algorithm for identifyi...
05/03/2021

Learning Good State and Action Representations via Tensor Decomposition

The transition kernel of a continuous-state-action Markov decision proce...
09/19/2018

Twisty Takens: A Geometric Characterization of Good Observations on Dense Trajectories

In nonlinear time series analysis and dynamical systems theory, Takens' ...
06/25/2019

Visualizing High Dimensional Dynamical Processes

Manifold learning techniques for dynamical systems and time series have ...
08/24/2019

Heterogeneous Relational Kernel Learning

Recent work has developed Bayesian methods for the automatic statistical...
11/25/2021

Learning Low-Dimensional Quadratic-Embeddings of High-Fidelity Nonlinear Dynamics using Deep Learning

Learning dynamical models from data plays a vital role in engineering de...
10/28/2021

Deeptime: a Python library for machine learning dynamical models from time series data

Generation and analysis of time-series data is relevant to many quantita...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.