DeepAI AI Chat
Log In Sign Up

Generalizing from a few environments in safety-critical reinforcement learning

07/02/2019
by   Zachary Kenton, et al.
University of Oxford
7

Before deploying autonomous agents in the real world, we need to be confident they will perform safely in novel situations. Ideally, we would expose agents to a very wide range of situations during training, allowing them to learn about every possible danger, but this is often impractical. This paper investigates safety and generalization from a limited number of training environments in deep reinforcement learning (RL). We find RL algorithms can fail dangerously on unseen test environments even when performing perfectly on training environments. Firstly, in a gridworld setting, we show that catastrophes can be significantly reduced with simple modifications, including ensemble model averaging and the use of a blocking classifier. In the more challenging CoinRun environment we find similar methods do not significantly reduce catastrophes. However, we do find that the uncertainty information from the ensemble is useful for predicting whether a catastrophe will occur within a few steps and hence whether human intervention should be requested.

READ FULL TEXT

page 3

page 6

page 12

05/07/2022

Search-Based Testing of Reinforcement Learning

Evaluation of deep reinforcement learning (RL) is inherently challenging...
05/12/2022

Provably Safe Deep Reinforcement Learning for Robotic Manipulation in Human Environments

Deep reinforcement learning (RL) has shown promising results in the moti...
10/07/2020

Online Safety Assurance for Deep Reinforcement Learning

Recently, deep learning has been successfully applied to a variety of ne...
02/13/2020

Adapting to Unseen Environments through Explicit Representation of Context

In order to deploy autonomous agents to domains such as autonomous drivi...
10/10/2022

Benchmarking Reinforcement Learning Techniques for Autonomous Navigation

Deep reinforcement learning (RL) has brought many successes for autonomo...
12/14/2021

Quantifying Multimodality in World Models

Model-based Deep Reinforcement Learning (RL) assumes the availability of...
01/01/2021

When Is Generalizable Reinforcement Learning Tractable?

Agents trained by reinforcement learning (RL) often fail to generalize b...

Code Repositories

maze

Maze environment for DRL generalization studies


view repo