Watch and Match: Supercharging Imitation with Regularized Optimal Transport

06/30/2022
by   Siddhant Haldar, et al.
7

Imitation learning holds tremendous promise in learning policies efficiently for complex decision making problems. Current state-of-the-art algorithms often use inverse reinforcement learning (IRL), where given a set of expert demonstrations, an agent alternatively infers a reward function and the associated optimal policy. However, such IRL approaches often require substantial online interactions for complex control problems. In this work, we present Regularized Optimal Transport (ROT), a new imitation learning algorithm that builds on recent advances in optimal transport based trajectory-matching. Our key technical insight is that adaptively combining trajectory-matching rewards with behavior cloning can significantly accelerate imitation even with only a few demonstrations. Our experiments on 20 visual control tasks across the DeepMind Control Suite, the OpenAI Robotics Suite, and the Meta-World Benchmark demonstrate an average of 7.8X faster imitation to reach 90 expert performance compared to prior state-of-the-art methods. On real-world robotic manipulation, with just one demonstration and an hour of online training, ROT achieves an average success rate of 90.1

READ FULL TEXT

page 1

page 6

page 7

page 18

page 19

page 21

page 22

page 23

research
06/19/2019

Wasserstein Adversarial Imitation Learning

Imitation Learning describes the problem of recovering an expert policy ...
research
07/20/2023

On Combining Expert Demonstrations in Imitation Learning via Optimal Transport

Imitation learning (IL) seeks to teach agents specific tasks through exp...
research
03/24/2023

Optimal Transport for Offline Imitation Learning

With the advent of large datasets, offline reinforcement learning (RL) i...
research
11/23/2021

Sample Efficient Imitation Learning via Reward Function Trained in Advance

Imitation learning (IL) is a framework that learns to imitate expert beh...
research
06/10/2022

Imitation Learning via Differentiable Physics

Existing imitation learning (IL) methods such as inverse reinforcement l...
research
11/06/2019

A Divergence Minimization Perspective on Imitation Learning Methods

In many settings, it is desirable to learn decision-making and control p...
research
05/07/2021

CoDE: Collocation for Demonstration Encoding

Roboticists frequently turn to Imitation learning (IL) for data efficien...

Please sign up or login with your details

Forgot password? Click here to reset