Task-Relevant Adversarial Imitation Learning

by   Konrad Zolna, et al.

We show that a critical problem in adversarial imitation from high-dimensional sensory data is the tendency of discriminator networks to distinguish agent and expert behaviour using task-irrelevant features beyond the control of the agent. We analyze this problem in detail and propose a solution as well as several baselines that outperform standard Generative Adversarial Imitation Learning (GAIL). Our proposed solution, Task-Relevant Adversarial Imitation Learning (TRAIL), uses a constrained optimization objective to overcome task-irrelevant features. Comprehensive experiments show that TRAIL can solve challenging manipulation tasks from pixels by imitating human operators, where other agents such as behaviour cloning (BC), standard GAIL, improved GAIL variants including our newly proposed baselines, and Deterministic Policy Gradients from Demonstrations (DPGfD) fail to find solutions, even when the other agents have access to task reward.


page 1

page 2

page 6

page 12


Latent Policies for Adversarial Imitation Learning

This paper considers learning robot locomotion and manipulation tasks fr...

Imitation by Predicting Observations

Imitation learning enables agents to reuse and adapt the hard-won expert...

Combating False Negatives in Adversarial Imitation Learning

In adversarial imitation learning, a discriminator is trained to differe...

Domain-Robust Visual Imitation Learning with Mutual Information Constraints

Human beings are able to understand objectives and learn by simply obser...

Causal Transfer for Imitation Learning and Decision Making under Sensor-shift

Learning from demonstrations (LfD) is an efficient paradigm to train AI ...

Active Learning within Constrained Environments through Imitation of an Expert Questioner

Active learning agents typically employ a query selection algorithm whic...

ADAIL: Adaptive Adversarial Imitation Learning

We present the ADaptive Adversarial Imitation Learning (ADAIL) algorithm...