Human-in-the-Loop Imitation Learning using Remote Teleoperation

by   Ajay Mandlekar, et al.

Imitation Learning is a promising paradigm for learning complex robot manipulation skills by reproducing behavior from human demonstrations. However, manipulation tasks often contain bottleneck regions that require a sequence of precise actions to make meaningful progress, such as a robot inserting a pod into a coffee machine to make coffee. Trained policies can fail in these regions because small deviations in actions can lead the policy into states not covered by the demonstrations. Intervention-based policy learning is an alternative that can address this issue – it allows human operators to monitor trained policies and take over control when they encounter failures. In this paper, we build a data collection system tailored to 6-DoF manipulation settings, that enables remote human operators to monitor and intervene on trained policies. We develop a simple and effective algorithm to train the policy iteratively on new data collected by the system that encourages the policy to learn how to traverse bottlenecks through the interventions. We demonstrate that agents trained on data collected by our intervention-based system and algorithm outperform agents trained on an equivalent number of samples collected by non-interventional demonstrators, and further show that our method outperforms multiple state-of-the-art baselines for learning from the human interventions on a challenging robot threading task and a coffee making task. Additional results and videos at .


page 1

page 4


Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperation

We tackle the problem of developing humanoid loco-manipulation skills wi...

Learning Multi-Arm Manipulation Through Collaborative Teleoperation

Imitation Learning (IL) is a powerful paradigm to teach robots to perfor...

Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment

With the rapid growth of computing powers and recent advances in deep le...

Learning to Fold Real Garments with One Arm: A Case Study in Cloud-Based Robotics Research

Autonomous fabric manipulation is a longstanding challenge in robotics, ...

LazyDAgger: Reducing Context Switching in Interactive Imitation Learning

Corrective interventions while a robot is learning to automate a task pr...

ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning

Effective robot learning often requires online human feedback and interv...

Please sign up or login with your details

Forgot password? Click here to reset