DeepAI AI Chat
Log In Sign Up

End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks

03/21/2019
by   Richard Cheng, et al.
0

Reinforcement Learning (RL) algorithms have found limited success beyond simulated applications, and one main reason is the absence of safety guarantees during the learning process. Real world systems would realistically fail or break before an optimal controller can be learned. To address this issue, we propose a controller architecture that combines (1) a model-free RL-based controller with (2) model-based controllers utilizing control barrier functions (CBFs) and (3) on-line learning of the unknown system dynamics, in order to ensure safety during learning. Our general framework leverages the success of RL algorithms to learn high-performance controllers, while the CBF-based controllers both guarantee safety and guide the learning process by constraining the set of explorable polices. We utilize Gaussian Processes (GPs) to model the system dynamics and its uncertainties. Our novel controller synthesis algorithm, RL-CBF, guarantees safety with high probability during the learning process, regardless of the RL algorithm used, and demonstrates greater policy exploration efficiency. We test our algorithm on (1) control of an inverted pendulum and (2) autonomous car-following with wireless vehicle-to-vehicle communication, and show that our algorithm attains much greater sample efficiency in learning than other state-of-the-art algorithms and maintains safety during the entire learning process.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/29/2022

Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions

Reinforcement Learning (RL) and continuous nonlinear control have been s...
10/21/2020

Safety Verification of Model Based Reinforcement Learning Controllers

Model-based reinforcement learning (RL) has emerged as a promising tool ...
09/24/2017

Learning Unmanned Aerial Vehicle Control for Autonomous Target Following

While deep reinforcement learning (RL) methods have achieved unprecedent...
02/06/2019

Augmenting Learning Components for Safety in Resource Constrained Autonomous Robots

This paper deals with resource constrained autonomous robots commonly fo...
10/12/2020

Control Barrier Functions for Unknown Nonlinear Systems using Gaussian Processes

This paper focuses on the controller synthesis for unknown, nonlinear sy...
09/23/2022

Synthesize Efficient Safety Certificates for Learning-Based Safe Control using Magnitude Regularization

Energy-function-based safety certificates can provide provable safety gu...