Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems

03/08/2022
by   Quentin Le Lidec, et al.
0

Optimal control (OC) algorithms such as Differential Dynamic Programming (DDP) take advantage of the derivatives of the dynamics to efficiently control physical systems. Yet, in the presence of nonsmooth dynamical systems, such class of algorithms are likely to fail due, for instance, to the presence of discontinuities in the dynamics derivatives or because of non-informative gradient. On the contrary, reinforcement learning (RL) algorithms have shown better empirical results in scenarios exhibiting non-smooth effects (contacts, frictions, etc). Our approach leverages recent works on randomized smoothing (RS) to tackle non-smoothness issues commonly encountered in optimal control, and provides key insights on the interplay between RL and OC through the prism of RS methods. This naturally leads us to introduce the randomized Differential Dynamic Programming (R-DDP) algorithm accounting for deterministic but non-smooth dynamics in a very sample-efficient way. The experiments demonstrate that our method is able to solve classic robotic problems with dry friction and frictional contacts, where classical OC algorithms are likely to fail and RL algorithms require in practice a prohibitive number of samples to find an optimal solution.

READ FULL TEXT

page 1

page 7

research
11/14/2020

Sparsity-Inducing Optimal Control via Differential Dynamic Programming

Optimal control is a popular approach to synthesize highly dynamic motio...
research
05/15/2023

A multilinear HJB-POD method for the optimal control of PDEs

Optimal control problems driven by evolutionary partial differential equ...
research
09/11/2019

Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Control

We introduce Crocoddyl (Contact RObot COntrol by Differential DYnamic Li...
research
04/07/2022

Parameterized Differential Dynamic Programming

Differential Dynamic Programming (DDP) is an efficient trajectory optimi...
research
02/20/2020

Differential Dynamic Programming Neural Optimizer

Interpretation of Deep Neural Networks (DNNs) training as an optimal con...
research
05/28/2021

Reinforcement Learning reveals fundamental limits on the mixing of active particles

The control of far-from-equilibrium physical systems, including active m...
research
06/19/2021

Learning to Reach, Swim, Walk and Fly in One Trial: Data-Driven Control with Scarce Data and Side Information

We develop a learning-based control algorithm for unknown dynamical syst...

Please sign up or login with your details

Forgot password? Click here to reset