Safe Reinforcement Learning with Contrastive Risk Prediction

09/10/2022
by   Hanping Zhang, et al.
0

As safety violations can lead to severe consequences in real-world robotic applications, the increasing deployment of Reinforcement Learning (RL) in robotic domains has propelled the study of safe exploration for reinforcement learning (safe RL). In this work, we propose a risk preventive training method for safe RL, which learns a statistical contrastive classifier to predict the probability of a state-action pair leading to unsafe states. Based on the predicted risk probabilities, we can collect risk preventive trajectories and reshape the reward function with risk penalties to induce safe RL policies. We conduct experiments in robotic simulation environments. The results show the proposed approach has comparable performance with the state-of-the-art model-based methods and outperforms conventional model-free safe RL approaches.

READ FULL TEXT
research
02/26/2021

Safe Distributional Reinforcement Learning

Safety in reinforcement learning (RL) is a key property in both training...
research
12/07/2021

MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerance

Safe exploration is critical for using reinforcement learning (RL) in ri...
research
10/14/2022

Safe Model-Based Reinforcement Learning with an Uncertainty-Aware Reachability Certificate

Safe reinforcement learning (RL) that solves constraint-satisfactory pol...
research
08/24/2023

Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theory

Risk-sensitive reinforcement learning (RL) has garnered significant atte...
research
03/16/2022

How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies

Autonomous driving has the potential to revolutionize mobility and is he...
research
07/08/2022

Ablation Study of How Run Time Assurance Impacts the Training and Performance of Reinforcement Learning Agents

Reinforcement Learning (RL) has become an increasingly important researc...
research
02/15/2022

Safe Reinforcement Learning by Imagining the Near Future

Safe reinforcement learning is a promising path toward applying reinforc...

Please sign up or login with your details

Forgot password? Click here to reset