Stochastic Optimal Control as Approximate Input Inference

10/07/2019
by   Joe Watson, et al.
0

Optimal control of stochastic nonlinear dynamical systems is a major challenge in the domain of robot learning. Given the intractability of the global control problem, state-of-the-art algorithms focus on approximate sequential optimization techniques, that heavily rely on heuristics for regularization in order to achieve stable convergence. By building upon the duality between inference and control, we develop the view of Optimal Control as Input Estimation, devising a probabilistic stochastic optimal control formulation that iteratively infers the optimal input distributions by minimizing an upper bound of the control cost. Inference is performed through Expectation Maximization and message passing on a probabilistic graphical model of the dynamical system, and time-varying linear Gaussian feedback controllers are extracted from the joint state-action distribution. This perspective incorporates uncertainty quantification, effective initialization through priors, and the principled regularization inherent to the Bayesian treatment. Moreover, it can be shown that for deterministic linearized systems, our framework derives the maximum entropy linear quadratic optimal control law. We provide a complete and detailed derivation of our probabilistic approach and highlight its advantages in comparison to other deterministic and probabilistic solvers.

READ FULL TEXT

Authors

page 20

05/17/2021

Stochastic Control through Approximate Bayesian Input Inference

Optimal control under uncertainty is a prevailing challenge in control, ...
10/01/2020

Active Inference or Control as Inference? A Unifying View

Active inference (AI) is a persuasive theoretical framework from computa...
09/17/2019

A new scalable algorithm for computational optimal control under uncertainty

We address the design and synthesis of optimal control strategies for hi...
04/11/2022

Maximum entropy optimal density control of discrete-time linear systems and Schrödinger bridges

We consider an entropy-regularized version of optimal density control of...
04/07/2021

Optimal Control for Structurally Sparse Systems using Graphical Inference

Dynamical systems with a distributed yet interconnected structure, like ...
10/06/2021

Entropy Regularised Deterministic Optimal Control: From Path Integral Solution to Sample-Based Trajectory Optimisation

Sample-based trajectory optimisers are a promising tool for the control ...
03/10/2021

Advancing Trajectory Optimization with Approximate Inference: Exploration, Covariance Control and Adaptive Risk

Discrete-time stochastic optimal control remains a challenging problem f...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.