DeepAI AI Chat
Log In Sign Up

Generalizing from a few environments in safety-critical reinforcement learning

by   Zachary Kenton, et al.
University of Oxford

Before deploying autonomous agents in the real world, we need to be confident they will perform safely in novel situations. Ideally, we would expose agents to a very wide range of situations during training, allowing them to learn about every possible danger, but this is often impractical. This paper investigates safety and generalization from a limited number of training environments in deep reinforcement learning (RL). We find RL algorithms can fail dangerously on unseen test environments even when performing perfectly on training environments. Firstly, in a gridworld setting, we show that catastrophes can be significantly reduced with simple modifications, including ensemble model averaging and the use of a blocking classifier. In the more challenging CoinRun environment we find similar methods do not significantly reduce catastrophes. However, we do find that the uncertainty information from the ensemble is useful for predicting whether a catastrophe will occur within a few steps and hence whether human intervention should be requested.


page 3

page 6

page 12


Search-Based Testing of Reinforcement Learning

Evaluation of deep reinforcement learning (RL) is inherently challenging...

Provably Safe Deep Reinforcement Learning for Robotic Manipulation in Human Environments

Deep reinforcement learning (RL) has shown promising results in the moti...

Online Safety Assurance for Deep Reinforcement Learning

Recently, deep learning has been successfully applied to a variety of ne...

Adapting to Unseen Environments through Explicit Representation of Context

In order to deploy autonomous agents to domains such as autonomous drivi...

Benchmarking Reinforcement Learning Techniques for Autonomous Navigation

Deep reinforcement learning (RL) has brought many successes for autonomo...

Quantifying Multimodality in World Models

Model-based Deep Reinforcement Learning (RL) assumes the availability of...

When Is Generalizable Reinforcement Learning Tractable?

Agents trained by reinforcement learning (RL) often fail to generalize b...

Code Repositories


Maze environment for DRL generalization studies

view repo