DeepAI AI Chat
Log In Sign Up

Demonstration-Guided Reinforcement Learning with Learned Skills

by   Karl Pertsch, et al.
University of Southern California

Demonstration-guided reinforcement learning (RL) is a promising approach for learning complex behaviors by leveraging both reward feedback and a set of target task demonstrations. Prior approaches for demonstration-guided RL treat every new task as an independent learning problem and attempt to follow the provided demonstrations step-by-step, akin to a human trying to imitate a completely unseen behavior by following the demonstrator's exact muscle movements. Naturally, such learning will be slow, but often new behaviors are not completely unseen: they share subtasks with behaviors we have previously learned. In this work, we aim to exploit this shared subtask structure to increase the efficiency of demonstration-guided RL. We first learn a set of reusable skills from large offline datasets of prior experience collected across many tasks. We then propose Skill-based Learning with Demonstrations (SkiLD), an algorithm for demonstration-guided RL that efficiently leverages the provided demonstrations by following the demonstrated skills instead of the primitive actions, resulting in substantial performance improvements over prior demonstration-guided RL approaches. We validate the effectiveness of our approach on long-horizon maze navigation and complex robot manipulation tasks.


page 6

page 7

page 14


Skill-based Meta-Reinforcement Learning

While deep reinforcement learning methods have shown impressive results ...

Prim-LAfD: A Framework to Learn and Adapt Primitive-Based Skills from Demonstrations for Insertion Tasks

Learning generalizable insertion skills in a data-efficient manner has l...

Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot

Task automation of surgical robot has the potentials to improve surgical...

Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback

A promising approach to solving challenging long-horizon tasks has been ...

Object Manipulation Learning by Imitation

We aim to enable robot to learn object manipulation by imitation. Given ...