Safety-Guided Deep Reinforcement Learning via Online Gaussian Process Estimation

by   Jiameng Fan, et al.
Boston University

An important facet of reinforcement learning (RL) has to do with how the agent goes about exploring the environment. Traditional exploration strategies typically focus on efficiency and ignore safety. However, for practical applications, ensuring safety of the agent during exploration is crucial since performing an unsafe action or reaching an unsafe state could result in irreversible damage to the agent. The main challenge of safe exploration is that characterizing the unsafe states and actions is difficult for large continuous state or action spaces and unknown environments. In this paper, we propose a novel approach to incorporate estimations of safety to guide exploration and policy search in deep reinforcement learning. By using a cost function to capture trajectory-based safety, our key idea is to formulate the state-action value function of this safety cost as a candidate Lyapunov function and extend control-theoretic results to approximate its derivative using online Gaussian Process (GP) estimation. We show how to use these statistical models to guide the agent in unknown environments to obtain high-performance control policies with provable stability certificates.


page 1

page 2

page 3

page 4


Safe Exploration for Constrained Reinforcement Learning with Provable Guarantees

We consider the problem of learning an episodic safe control policy that...

Exploring Unknown States with Action Balance

Exploration is a key problem in reinforcement learning. Recently bonus-b...

Improving Safety in Deep Reinforcement Learning using Unsupervised Action Planning

One of the key challenges to deep reinforcement learning (deep RL) is to...

Knowledge-Guided Exploration in Deep Reinforcement Learning

This paper proposes a new method to drastically speed up deep reinforcem...

Safe Exploration of State and Action Spaces in Reinforcement Learning

In this paper, we consider the important problem of safe exploration in ...

Autoregressive Policies for Continuous Control Deep Reinforcement Learning

Reinforcement learning algorithms rely on exploration to discover new be...

Gaussian Process Self-triggered Policy Search in Weakly Observable Environments

The environments of such large industrial machines as waste cranes in wa...

Please sign up or login with your details

Forgot password? Click here to reset