Virtuously Safe Reinforcement Learning

by   Henrik Aslund, et al.

We show that when a third party, the adversary, steps into the two-party setting (agent and operator) of safely interruptible reinforcement learning, a trade-off has to be made between the probability of following the optimal policy in the limit, and the probability of escaping a dangerous situation created by the adversary. So far, the work on safely interruptible agents has assumed a perfect perception of the agent about its environment (no adversary), and therefore implicitly set the second probability to zero, by explicitly seeking a value of one for the first probability. We show that (1) agents can be made both interruptible and adversary-resilient, and (2) the interruptibility can be made safe in the sense that the agent itself will not seek to avoid it. We also solve the problem that arises when the agent does not go completely greedy, i.e. issues with safe exploration in the limit. Resilience to perturbed perception, safe exploration in the limit, and safe interruptibility are the three pillars of what we call virtuously safe reinforcement learning.



There are no comments yet.


page 1

page 2

page 3

page 4


Curiosity Killed the Cat and the Asymptotically Optimal Agent

Reinforcement learners are agents that learn to pick actions that lead t...

Adversary agent reinforcement learning for pursuit-evasion

A reinforcement learning environment with adversary agents is proposed i...

Risk Averse Robust Adversarial Reinforcement Learning

Deep reinforcement learning has recently made significant progress in so...

Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning

In reinforcement learning, agents learn by performing actions and observ...

Provably Safe PAC-MDP Exploration Using Analogies

A key challenge in applying reinforcement learning to safety-critical do...

Reinforcement Learning for the Soccer Dribbling Task

We propose a reinforcement learning solution to the soccer dribbling tas...

Learning to be safe, in finite time

This paper aims to put forward the concept that learning to take safe ac...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.