Human-in-the-Loop Imitation Learning using Remote Teleoperation

12/12/2020
by   Ajay Mandlekar, et al.
15

Imitation Learning is a promising paradigm for learning complex robot manipulation skills by reproducing behavior from human demonstrations. However, manipulation tasks often contain bottleneck regions that require a sequence of precise actions to make meaningful progress, such as a robot inserting a pod into a coffee machine to make coffee. Trained policies can fail in these regions because small deviations in actions can lead the policy into states not covered by the demonstrations. Intervention-based policy learning is an alternative that can address this issue – it allows human operators to monitor trained policies and take over control when they encounter failures. In this paper, we build a data collection system tailored to 6-DoF manipulation settings, that enables remote human operators to monitor and intervene on trained policies. We develop a simple and effective algorithm to train the policy iteratively on new data collected by the system that encourages the policy to learn how to traverse bottlenecks through the interventions. We demonstrate that agents trained on data collected by our intervention-based system and algorithm outperform agents trained on an equivalent number of samples collected by non-interventional demonstrators, and further show that our method outperforms multiple state-of-the-art baselines for learning from the human interventions on a challenging robot threading task and a coffee making task. Additional results and videos at https://sites.google.com/stanford.edu/iwr .

READ FULL TEXT

page 1

page 4

02/04/2022

BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning

In this paper, we study the problem of enabling a vision-based robotic m...
12/12/2020

Learning Multi-Arm Manipulation Through Collaborative Teleoperation

Imitation Learning (IL) is a powerful paradigm to teach robots to perfor...
04/21/2022

Learning to Fold Real Garments with One Arm: A Case Study in Cloud-Based Robotics Research

Autonomous fabric manipulation is a longstanding challenge in robotics, ...
03/31/2021

LazyDAgger: Reducing Context Switching in Interactive Imitation Learning

Corrective interventions while a robot is learning to automate a task pr...
11/11/2019

Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset through Human Reasoning and Dexterity

Large, richly annotated datasets have accelerated progress in fields suc...
12/09/2021

Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation

In mobile manipulation (MM), robots can both navigate within and interac...
09/17/2021

ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning

Effective robot learning often requires online human feedback and interv...