Predictive Coding for Locally-Linear Control

03/02/2020
by   Rui Shu, et al.
5

High-dimensional observations and unknown dynamics are major challenges when applying optimal control to many real-world decision making tasks. The Learning Controllable Embedding (LCE) framework addresses these challenges by embedding the observations into a lower dimensional latent space, estimating the latent dynamics, and then performing control directly in the latent space. To ensure the learned latent dynamics are predictive of next-observations, all existing LCE approaches decode back into the observation space and explicitly perform next-observation prediction—a challenging high-dimensional task that furthermore introduces a large number of nuisance parameters (i.e., the decoder) which are discarded during control. In this paper, we propose a novel information-theoretic LCE approach and show theoretically that explicit next-observation prediction can be replaced with predictive coding. We then use predictive coding to develop a decoder-free LCE model whose latent dynamics are amenable to locally-linear control. Extensive experiments on benchmark tasks show that our model reliably learns a controllable latent space that leads to superior performance when compared with state-of-the-art LCE baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/04/2019

Prediction, Consistency, Curvature: Representation Learning for Locally-Linear Control

Many real-world sequential decision-making problems can be formulated as...
06/14/2021

Temporal Predictive Coding For Model-Based Planning In Latent Space

High-dimensional observations are a major challenge in the application o...
11/02/2019

Beta DVBF: Learning State-Space Models for Control from High Dimensional Observations

Learning a model of dynamics from high-dimensional images can be a core ...
06/23/2020

Projective Latent Space Decluttering

High-dimensional latent representations learned by neural network classi...
10/15/2021

Learn Proportional Derivative Controllable Latent Space from Pixels

Recent advances in latent space dynamics model from pixels show promisin...
01/31/2022

Continuous Forecasting via Neural Eigen Decomposition of Stochastic Dynamics

Motivated by a real-world problem of blood coagulation control in Hepari...
03/31/2020

Cross Scene Prediction via Modeling Dynamic Correlation using Latent Space Shared Auto-Encoders

This work addresses on the following problem: given a set of unsynchroni...