What Matters for Adversarial Imitation Learning?

by   Manu Orsini, et al.

Adversarial imitation learning has become a popular framework for imitation in continuous control. Over the years, several variations of its components were proposed to enhance the performance of the learned policies as well as the sample complexity of the algorithm. In practice, these choices are rarely tested all together in rigorous empirical studies. It is therefore difficult to discuss and understand what choices, among the high-level algorithmic options as well as low-level implementation details, matter. To tackle this issue, we implement more than 50 of these choices in a generic adversarial imitation learning framework and investigate their impacts in a large-scale study (>500k trained agents) with both synthetic and human-generated demonstrations. While many of our findings confirm common practices, some of them are surprising or even contradict prior work. In particular, our results suggest that artificial demonstrations are not a good proxy for human data and that the very common practice of evaluating imitation algorithms only with synthetic demonstrations may lead to algorithms which perform poorly in the more realistic scenarios with human demonstrations.


page 3

page 28

page 29

page 38

page 39

page 40

page 41

page 42


Adversarial Imitation Learning from Video using a State Observer

The imitation learning research community has recently made significant ...

DDCO: Discovery of Deep Continuous Options for Robot Learning from Demonstrations

An option is a short-term skill consisting of a control policy for a spe...

Video Imitation GAN: Learning control policies by imitating raw videos using generative adversarial reward estimation

Natural imitation in humans usually consists of mimicking visual demonst...

Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatch

Learning from demonstrations in the wild (e.g. YouTube videos) is a tant...

Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information

The use of imitation learning to learn a single policy for a complex tas...

Provably Efficient Adversarial Imitation Learning with Unknown Transitions

Imitation learning (IL) has proven to be an effective method for learnin...

Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations

Multimodal demonstrations provide robots with an abundance of informatio...

Please sign up or login with your details

Forgot password? Click here to reset