Log In Sign Up

Self-Imitation Learning by Planning

by   Sha Luo, et al.

Imitation learning (IL) enables robots to acquire skills quickly by transferring expert knowledge, which is widely adopted in reinforcement learning (RL) to initialize exploration. However, in long-horizon motion planning tasks, a challenging problem in deploying IL and RL methods is how to generate and collect massive, broadly distributed data such that these methods can generalize effectively. In this work, we solve this problem using our proposed approach called self-imitation learning by planning (SILP), where demonstration data are collected automatically by planning on the visited states from the current policy. SILP is inspired by the observation that successfully visited states in the early reinforcement learning stage are collision-free nodes in the graph-search based motion planner, so we can plan and relabel robot's own trials as demonstrations for policy learning. Due to these self-generated demonstrations, we relieve the human operator from the laborious data preparation process required by IL and RL methods in solving complex motion planning tasks. The evaluation results show that our SILP method achieves higher success rates and enhances sample efficiency compared to selected baselines, and the policy learned in simulation performs well in a real-world placement task with changing goals and obstacles.


page 1

page 6


Automatic Curricula via Expert Demonstrations

We propose Automatic Curricula via Expert Demonstrations (ACED), a reinf...

Divide Conquer Imitation Learning

When cast into the Deep Reinforcement Learning framework, many robotics ...

What data do we need for training an AV motion planner?

We investigate what grade of sensor data is required for training an imi...

Guided Imitation of Task and Motion Planning

While modern policy optimization methods can do complex manipulation fro...

The Atari Grand Challenge Dataset

Recent progress in Reinforcement Learning (RL), fueled by its combinatio...

Learning Deep Policies for Physics-Based Manipulation in Clutter

Uncertainty in modeling real world physics makes transferring traditiona...

Hierarchical Imitation and Reinforcement Learning

We study the problem of learning policies over long time horizons. We pr...