Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics

10/24/2019
by   Shuo Li, et al.
0

This paper proposes a framework for safe reinforcement learning that can handle stochastic nonlinear dynamical systems. We focus on the setting where the nominal dynamics are known, and are subject to additive stochastic disturbances with known distribution. Our goal is to ensure the safety of a control policy trained using reinforcement learning, e.g., in a simulated environment. We build on the idea of model predictive shielding (MPS), where a backup controller is used to override the learned policy as needed to ensure safety. The key challenge is how to compute a backup policy in the context of stochastic dynamics. We propose to use a tube-based robust NMPC controller as the backup controller. We estimate the tubes using sampled trajectories, leveraging ideas from statistical learning theory to obtain high-probability guarantees. We empirically demonstrate that our approach can ensure safety in stochastic systems, including cart-pole and a non-holonomic particle with random obstacles.

READ FULL TEXT
research
05/25/2019

Safe Reinforcement Learning via Online Shielding

Reinforcement learning is a promising approach to learning control polic...
research
06/06/2020

Automatic Policy Synthesis to Improve the Safety of Nonlinear Dynamical Systems

Learning controllers merely based on a performance metric has been prove...
research
09/02/2020

Adaptive CVaR Optimization for Dynamical Systems with Path Space Stochastic Search

We present a general framework for optimizing the Conditional Value-at-R...
research
03/03/2020

ABC-LMPC: Safe Sample-Based Learning MPC for Stochastic Nonlinear Dynamical Systems with Adjustable Boundary Conditions

Sample-based learning model predictive control (LMPC) strategies have re...
research
04/19/2023

Model Based Reinforcement Learning for Personalized Heparin Dosing

A key challenge in sequential decision making is optimizing systems safe...
research
09/21/2023

Learning to Recover for Safe Reinforcement Learning

Safety controllers is widely used to achieve safe reinforcement learning...
research
07/31/2018

Learning Stabilizable Dynamical Systems via Control Contraction Metrics

We propose a novel framework for learning stabilizable nonlinear dynamic...

Please sign up or login with your details

Forgot password? Click here to reset