Guiding Safe Exploration with Weakest Preconditions

09/28/2022
by   Greg Anderson, et al.
0

In reinforcement learning for safety-critical settings, it is often desirable for the agent to obey safety constraints at all points in time, including during training. We present a novel neurosymbolic approach called SPICE to solve this safe exploration problem. SPICE uses an online shielding layer based on symbolic weakest preconditions to achieve a more precise safety analysis than existing tools without unduly impacting the training process. We evaluate the approach on a suite of continuous control benchmarks and show that it can achieve comparable performance to existing safe learning techniques while incurring fewer safety violations. Additionally, we present theoretical results showing that SPICE converges to the optimal safe policy under reasonable assumptions.

READ FULL TEXT

page 8

page 19

research
02/24/2021

Towards Safe Continuing Task Reinforcement Learning

Safety is a critical feature of controller design for physical systems. ...
research
07/10/2023

Probabilistic Counterexample Guidance for Safer Reinforcement Learning (Extended Version)

Safe exploration aims at addressing the limitations of Reinforcement Lea...
research
07/07/2020

Provably Safe PAC-MDP Exploration Using Analogies

A key challenge in applying reinforcement learning to safety-critical do...
research
04/23/2023

System III: Learning with Domain Knowledge for Safety Constraints

Reinforcement learning agents naturally learn from extensive exploration...
research
11/04/2021

Infinite Time Horizon Safety of Bayesian Neural Networks

Bayesian neural networks (BNNs) place distributions over the weights of ...
research
02/13/2023

Provably Safe Reinforcement Learning with Step-wise Violation Constraints

In this paper, we investigate a novel safe reinforcement learning proble...
research
09/27/2019

Safe Reinforcement Learning on Autonomous Vehicles

There have been numerous advances in reinforcement learning, but the typ...

Please sign up or login with your details

Forgot password? Click here to reset