Reinforcement Learning with Videos: Combining Offline Observations with Interaction

11/12/2020
by   Karl Schmeckpeper, et al.
15

Reinforcement learning is a powerful framework for robots to acquire skills from experience, but often requires a substantial amount of online data collection. As a result, it is difficult to collect sufficiently diverse experiences that are needed for robots to generalize broadly. Videos of humans, on the other hand, are a readily available source of broad and interesting experiences. In this paper, we consider the question: can we perform reinforcement learning directly on experience collected by humans? This problem is particularly difficult, as such videos are not annotated with actions and exhibit substantial visual domain shift relative to the robot's embodiment. To address these challenges, we propose a framework for reinforcement learning with videos (RLV). RLV learns a policy and value function using experience collected by humans in combination with data collected by robots. In our experiments, we find that RLV is able to leverage such videos to learn challenging vision-based skills with less than half as many samples as RL methods that learn from scratch.

READ FULL TEXT

page 1

page 6

page 7

page 8

page 14

page 15

page 16

research
06/16/2020

Accelerating Online Reinforcement Learning with Offline Datasets

Reinforcement learning provides an appealing formalism for learning cont...
research
04/16/2021

MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale

General-purpose robotic systems must master a large repertoire of divers...
research
09/19/2021

Lifelong Robotic Reinforcement Learning by Retaining Experiences

Multi-task learning ideally allows robots to acquire a diverse repertoir...
research
12/30/2019

Learning Predictive Models From Observation and Interaction

Learning predictive models from interaction with the world allows an age...
research
10/03/2016

Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Search

In principle, reinforcement learning and policy search methods can enabl...
research
03/16/2020

DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction

Deep reinforcement learning can learn effective policies for a wide rang...
research
09/19/2021

Dual Behavior Regularized Reinforcement Learning

Reinforcement learning has been shown to perform a range of complex task...

Please sign up or login with your details

Forgot password? Click here to reset