Predictive Coding for Locally-Linear Control

by   Rui Shu, et al.

High-dimensional observations and unknown dynamics are major challenges when applying optimal control to many real-world decision making tasks. The Learning Controllable Embedding (LCE) framework addresses these challenges by embedding the observations into a lower dimensional latent space, estimating the latent dynamics, and then performing control directly in the latent space. To ensure the learned latent dynamics are predictive of next-observations, all existing LCE approaches decode back into the observation space and explicitly perform next-observation prediction—a challenging high-dimensional task that furthermore introduces a large number of nuisance parameters (i.e., the decoder) which are discarded during control. In this paper, we propose a novel information-theoretic LCE approach and show theoretically that explicit next-observation prediction can be replaced with predictive coding. We then use predictive coding to develop a decoder-free LCE model whose latent dynamics are amenable to locally-linear control. Extensive experiments on benchmark tasks show that our model reliably learns a controllable latent space that leads to superior performance when compared with state-of-the-art LCE baselines.


page 1

page 2

page 3

page 4


Prediction, Consistency, Curvature: Representation Learning for Locally-Linear Control

Many real-world sequential decision-making problems can be formulated as...

Temporal Predictive Coding For Model-Based Planning In Latent Space

High-dimensional observations are a major challenge in the application o...

Beta DVBF: Learning State-Space Models for Control from High Dimensional Observations

Learning a model of dynamics from high-dimensional images can be a core ...

Projective Latent Space Decluttering

High-dimensional latent representations learned by neural network classi...

Learn Proportional Derivative Controllable Latent Space from Pixels

Recent advances in latent space dynamics model from pixels show promisin...

Continuous Forecasting via Neural Eigen Decomposition of Stochastic Dynamics

Motivated by a real-world problem of blood coagulation control in Hepari...

Cross Scene Prediction via Modeling Dynamic Correlation using Latent Space Shared Auto-Encoders

This work addresses on the following problem: given a set of unsynchroni...