Safe Model-based Reinforcement Learning with Stability Guarantees

05/23/2017
by   Felix Berkenkamp, et al.
0

Reinforcement learning is a powerful paradigm for learning optimal policies from experimental data. However, to find optimal policies, most reinforcement learning algorithms explore all possible actions, which may be harmful for real-world systems. As a consequence, learning algorithms are rarely applied on safety-critical systems in the real world. In this paper, we present a learning algorithm that explicitly considers safety, defined in terms of stability guarantees. Specifically, we extend control-theoretic results on Lyapunov stability verification and show how to use statistical models of the dynamics to obtain high-performance control policies with provable stability certificates. Moreover, under additional regularity assumptions in terms of a Gaussian process prior, we prove that one can effectively and safely collect data in order to learn about the dynamics and thus both improve control performance and expand the safe region of the state space. In our experiments, we show how the resulting algorithm can safely optimize a neural network policy on a simulated inverted pendulum, without the pendulum ever falling down.

READ FULL TEXT
research
08/02/2018

The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamic Systems

Learning algorithms have shown considerable prowess in simulation by all...
research
03/22/2018

Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learning

Learning-based methods have been successful in solving complex control t...
research
10/26/2018

Stability-certified reinforcement learning: A control-theoretic perspective

We investigate the important problem of certifying stability of reinforc...
research
09/07/2023

A computationally lightweight safe learning algorithm

Safety is an essential asset when learning control policies for physical...
research
08/11/2020

Learning Event-triggered Control from Data through Joint Optimization

We present a framework for model-free learning of event-triggered contro...
research
12/14/2020

Safe Reinforcement Learning with Stability Safety Guarantees Using Robust MPC

Reinforcement Learning offers tools to optimize policies based on the da...
research
02/15/2022

Safe Reinforcement Learning by Imagining the Near Future

Safe reinforcement learning is a promising path toward applying reinforc...

Please sign up or login with your details

Forgot password? Click here to reset