Safe Model-Based Reinforcement Learning Using Robust Control Barrier Functions

10/11/2021
by   Yousef Emam, et al.
9

Reinforcement Learning (RL) is effective in many scenarios. However, it typically requires the exploration of a sufficiently large number of state-action pairs, some of which may be unsafe. Consequently, its application to safety-critical systems remains a challenge. Towards this end, an increasingly common approach to address safety involves the addition of a safety layer that projects the RL actions onto a safe set of actions. In turn, a challenge for such frameworks is how to effectively couple RL with the safety layer to improve the learning performance. In the context of leveraging control barrier functions for safe RL training, prior work focuses on a restricted class of barrier functions and utilizes an auxiliary neural net to account for the effects of the safety layer which inherently results in an approximation. In this paper, we frame safety as a differentiable robust-control-barrier-function layer in a model-based RL framework. As such, this approach both ensures safety and effectively guides exploration during training resulting in increased sample efficiency as demonstrated in the experiments.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/21/2021

Safe Reinforcement Learning Using Robust Action Governor

Reinforcement Learning (RL) is essentially a trial-and-error learning pr...
research
11/30/2022

Safe Model-Free Reinforcement Learning using Disturbance-Observer-Based Control Barrier Functions

Safe reinforcement learning (RL) with assured satisfaction of hard state...
research
04/16/2021

Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions

This paper studies the problem of developing an approximate dynamic prog...
research
06/06/2023

Value Functions are Control Barrier Functions: Verification of Safe Policies using Control Theory

Guaranteeing safe behaviour of reinforcement learning (RL) policies pose...
research
12/02/2021

Safe Reinforcement Learning for Grid Voltage Control

Under voltage load shedding has been considered as a standard approach t...
research
01/03/2023

Safe Reinforcement Learning for an Energy-Efficient Driver Assistance System

Reinforcement learning (RL)-based driver assistance systems seek to impr...
research
08/04/2021

Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations

Training-time safety violations have been a major concern when we deploy...

Please sign up or login with your details

Forgot password? Click here to reset