Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback

08/11/2021
by   Xiaofei Wang, et al.
0

A promising approach to solving challenging long-horizon tasks has been to extract behavior priors (skills) by fitting generative models to large offline datasets of demonstrations. However, such generative models inherit the biases of the underlying data and result in poor and unusable skills when trained on imperfect demonstration data. To better align skill extraction with human intent we present Skill Preferences (SkiP), an algorithm that learns a model over human preferences and uses it to extract human-aligned skills from offline data. After extracting human-preferred skills, SkiP also utilizes human feedback to solve down-stream tasks with RL. We show that SkiP enables a simulated kitchen robot to solve complex multi-step manipulation tasks and substantially outperforms prior leading RL algorithms with human preferences as well as leading skill extraction algorithms without human preferences.

READ FULL TEXT
research
07/21/2021

Demonstration-Guided Reinforcement Learning with Learned Skills

Demonstration-guided reinforcement learning (RL) is a promising approach...
research
07/19/2021

Hierarchical Few-Shot Imitation with Skill Transition Models

A desirable property of autonomous agents is the ability to both solve l...
research
03/29/2023

Plan4MC: Skill Reinforcement Learning and Planning for Open-World Minecraft Tasks

We study building a multi-task agent in Minecraft. Without human demonst...
research
03/01/2022

Interactive Human-in-the-loop Coordination of Manipulation Skills Learned from Demonstration

Learning from demonstration (LfD) provides a fast, intuitive and efficie...
research
10/29/2020

"What, not how": Solving an under-actuated insertion task from scratch

Robot manipulation requires a complex set of skills that need to be care...
research
06/22/2023

SoftGPT: Learn Goal-oriented Soft Object Manipulation Skills by Generative Pre-trained Heterogeneous Graph Transformer

Soft object manipulation tasks in domestic scenes pose a significant cha...
research
07/15/2019

Mutual Reinforcement Learning

Recently, collaborative robots have begun to train humans to achieve com...

Please sign up or login with your details

Forgot password? Click here to reset