The Power of Learned Locally Linear Models for Nonlinear Policy Optimization

05/16/2023
by   Daniel Pfrommer, et al.
10

A common pipeline in learning-based control is to iteratively estimate a model of system dynamics, and apply a trajectory optimization algorithm - e.g. 𝚒𝙻𝚀𝚁 - on the learned model to minimize a target cost. This paper conducts a rigorous analysis of a simplified variant of this strategy for general nonlinear systems. We analyze an algorithm which iterates between estimating local linear models of nonlinear system dynamics and performing 𝚒𝙻𝚀𝚁-like policy updates. We demonstrate that this algorithm attains sample complexity polynomial in relevant problem parameters, and, by synthesizing locally stabilizing gains, overcomes exponential dependence in problem horizon. Experimental results validate the performance of our algorithm, and compare to natural deep-learning baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/22/2020

Data-based Receding Horizon Control of Linear Network Systems

We propose a distributed data-based predictive control scheme to stabili...
research
06/24/2019

A note on locally optimal designs for generalized linear models with restricted support

Optimal designs for generalized linear models require a prior knowledge ...
research
04/03/2022

Learning Linear Representations of Nonlinear Dynamics Using Deep Learning

The vast majority of systems of practical interest are characterised by ...
research
01/22/2020

Local Policy Optimization for Trajectory-Centric Reinforcement Learning

The goal of this paper is to present a method for simultaneous trajector...
research
04/24/2023

Synthesizing Stable Reduced-Order Visuomotor Policies for Nonlinear Systems via Sums-of-Squares Optimization

We present a method for synthesizing dynamic, reduced-order output-feedb...
research
08/14/2020

Bayesian model selection in additive partial linear models via locally adaptive splines

We consider a model selection problem for additive partial linear models...
research
02/20/2020

Non-asymptotic and Accurate Learning of Nonlinear Dynamical Systems

We consider the problem of learning stabilizable systems governed by non...

Please sign up or login with your details

Forgot password? Click here to reset