HG-DAgger: Interactive Imitation Learning with Human Experts

by   Michael Kelly, et al.

Imitation learning has proven to be useful for many real-world problems, but approaches such as behavioral cloning suffer from data mismatch and compounding error issues. One attempt to address these limitations is the DAgger algorithm, which uses the state distribution induced by the novice to sample corrective actions from the expert. Such sampling schemes, however, require the expert to provide action labels without being fully in control of the system. This can decrease safety and, when using humans as experts, is likely to degrade the quality of the collected labels due to perceived actuator lag. In this work, we propose HG-DAgger, a variant of DAgger that is more suitable for interactive imitation learning from human experts in real-world systems. In addition to training a novice policy, HG-DAgger also learns a safety threshold for a model-uncertainty-based risk metric that can be used to predict the performance of the fully trained novice in different regions of the state space. We evaluate our method on both a simulated and real-world autonomous driving task, and demonstrate improved performance over both DAgger and behavioral cloning.


DropoutDAgger: A Bayesian Approach to Safe Imitation Learning

While imitation learning is becoming common practice in robotics, this a...

Behavioral Cloning from Observation

Humans often learn how to perform tasks via imitation: they observe othe...

Interactive Imitation Learning in State-Space

Imitation Learning techniques enable programming the behavior of agents ...

Uncertainty-Aware Data Aggregation for Deep Imitation Learning

Estimating statistical uncertainties allows autonomous agents to communi...

Imitation Learning Approach for AI Driving Olympics Trained on Real-world and Simulation Data Simultaneously

In this paper, we describe our winning approach to solving the Lane Foll...

Causal Imitation Learning under Temporally Correlated Noise

We develop algorithms for imitation learning from policy data that was c...

EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning

While imitation learning is often used in robotics, this approach often ...