Learning to Switch Between Machines and Humans

02/11/2020
by   Vahid Balazadeh-Meresht, et al.
0

Reinforcement learning algorithms have been mostly developed and evaluated under the assumption that they will operate in a fully autonomous manner—they will take all actions. However, in safety critical applications, full autonomy faces a variety of technical, societal and legal challenges, which have precluded the use of reinforcement learning policies in real-world systems. In this work, our goal is to develop algorithms that, by learning to switch control between machines and humans, allow existing reinforcement learning policies to operate under different automation levels. More specifically, we first formally define the learning to switch problem using finite horizon Markov decision processes. Then, we show that, if the human policy is known, we can find the optimal switching policy directly by solving a set of recursive equations using backwards induction. However, in practice, the human policy is often unknown. To overcome this, we develop an algorithm that uses upper confidence bounds on the human policy to find a sequence of switching policies whose total regret with respect to the optimal switching policy is sublinear. Simulation experiments on two important tasks in autonomous driving—lane keeping and obstacle avoidance—demonstrate the effectiveness of the proposed algorithms and illustrate our theoretical findings.

READ FULL TEXT

page 7

page 8

page 9

research
02/13/2022

Sample-Efficient Reinforcement Learning with loglog(T) Switching Cost

We study the problem of reinforcement learning (RL) with low (policy) sw...
research
03/30/2017

Enter the Matrix: A Virtual World Approach to Safely Interruptable Autonomous Systems

Robots and autonomous systems that operate around humans will likely alw...
research
04/11/2022

External control of a genetic toggle switch via Reinforcement Learning

We investigate the problem of using a learning-based strategy to stabili...
research
02/24/2023

Logarithmic Switching Cost in Reinforcement Learning beyond Linear MDPs

In many real-life reinforcement learning (RL) problems, deploying new po...
research
12/12/2022

A Survey on Reinforcement Learning Security with Application to Autonomous Driving

Reinforcement learning allows machines to learn from their own experienc...
research
09/23/2021

Reinforcement Learning Under Algorithmic Triage

Methods to learn under algorithmic triage have predominantly focused on ...
research
08/06/2021

Anomaly Search with Multiple Plays under Delay and Switching Costs

The problem of searching for L anomalous processes among M processes is ...

Please sign up or login with your details

Forgot password? Click here to reset