An Algorithmic Perspective on Imitation Learning

by   Takayuki Osa, et al.

As robots and other intelligent agents move from simple environments and problems to more complex, unstructured settings, manually programming their behavior has become increasingly challenging and expensive. Often, it is easier for a teacher to demonstrate a desired behavior rather than attempt to manually engineer it. This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning. This work provides an introduction to imitation learning. It covers the underlying assumptions, approaches, and how they relate; the rich set of algorithms developed to tackle the problem; and advice on effective tools and implementation. We intend this paper to serve two audiences. First, we want to familiarize machine learning experts with the challenges of imitation learning, particularly those arising in robotics, and the interesting theoretical and practical distinctions between it and more familiar frameworks like statistical supervised learning theory and reinforcement learning. Second, we want to give roboticists and experts in applied artificial intelligence a broader appreciation for the frameworks and tools available for imitation learning.


page 15

page 29


Imitation Learning by Reinforcement Learning

Imitation Learning algorithms learn a policy from demonstrations of expe...

The Past and Present of Imitation Learning: A Citation Chain Study

Imitation Learning is a promising area of active research. Over the last...

Robust Imitation Learning from Noisy Demonstrations

Learning from noisy demonstrations is a practical but highly challenging...

Imitation Learning with Recurrent Neural Networks

We present a novel view that unifies two frameworks that aim to solve se...

What Matters for Adversarial Imitation Learning?

Adversarial imitation learning has become a popular framework for imitat...

Learning Goal-Oriented Visual Dialog Agents: Imitating and Surpassing Analytic Experts

This paper tackles the problem of learning a questioner in the goal-orie...

Neural Networks for Beginners. A fast implementation in Matlab, Torch, TensorFlow

This report provides an introduction to some Machine Learning tools with...