Predictive State Representations: A New Theory for Modeling Dynamical Systems

07/11/2012
by   Satinder Singh, et al.
0

Modeling dynamical systems, both for control purposes and to make predictions about their behavior, is ubiquitous in science and engineering. Predictive state representations (PSRs) are a recently introduced class of models for discrete-time dynamical systems. The key idea behind PSRs and the closely related OOMs (Jaeger's observable operator models) is to represent the state of the system as a set of predictions of observable outcomes of experiments one can do in the system. This makes PSRs rather different from history-based models such as nth-order Markov models and hidden-state-based models such as HMMs and POMDPs. We introduce an interesting construct, the systemdynamics matrix, and show how PSRs can be derived simply from it. We also use this construct to show formally that PSRs are more general than both nth-order Markov models and HMMs/POMDPs. Finally, we discuss the main difference between PSRs and OOMs and conclude with directions for future work.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/09/2012

Temporal-Difference Networks for Dynamical Systems with Continuous Observations and Actions

Temporal-difference (TD) networks are a class of predictive state repres...
research
07/04/2012

Predictive Linear-Gaussian Models of Stochastic Dynamical Systems

Models of dynamical systems based on predictive state representations (P...
research
02/26/2019

Deep Variational Koopman Models: Inferring Koopman Observations for Uncertainty-Aware Dynamics Modeling and Control

Koopman theory asserts that a nonlinear dynamical system can be mapped t...
research
07/11/2012

Dynamical Systems Trees

We propose dynamical systems trees (DSTs) as a flexible class of models ...
research
01/31/2020

Zeta Functions and the (Linear) Logic of Markov Processes

In a series of papers, the author introduced models of linear logic know...
research
09/14/2022

Algorithmic (Semi-)Conjugacy via Koopman Operator Theory

Iterative algorithms are of utmost importance in decision and control. W...
research
10/01/2020

A Finite Memory Interacting Pólya Contagion Network and its Approximating Dynamical Systems

We introduce a new model for contagion spread using a network of interac...

Please sign up or login with your details

Forgot password? Click here to reset