DeepAI AI Chat
Log In Sign Up

Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback

08/11/2021
by   Xiaofei Wang, et al.
berkeley college
0

A promising approach to solving challenging long-horizon tasks has been to extract behavior priors (skills) by fitting generative models to large offline datasets of demonstrations. However, such generative models inherit the biases of the underlying data and result in poor and unusable skills when trained on imperfect demonstration data. To better align skill extraction with human intent we present Skill Preferences (SkiP), an algorithm that learns a model over human preferences and uses it to extract human-aligned skills from offline data. After extracting human-preferred skills, SkiP also utilizes human feedback to solve down-stream tasks with RL. We show that SkiP enables a simulated kitchen robot to solve complex multi-step manipulation tasks and substantially outperforms prior leading RL algorithms with human preferences as well as leading skill extraction algorithms without human preferences.

READ FULL TEXT
07/21/2021

Demonstration-Guided Reinforcement Learning with Learned Skills

Demonstration-guided reinforcement learning (RL) is a promising approach...
07/19/2021

Hierarchical Few-Shot Imitation with Skill Transition Models

A desirable property of autonomous agents is the ability to both solve l...
03/29/2023

Plan4MC: Skill Reinforcement Learning and Planning for Open-World Minecraft Tasks

We study building a multi-task agent in Minecraft. Without human demonst...
04/25/2022

Skill-based Meta-Reinforcement Learning

While deep reinforcement learning methods have shown impressive results ...
10/29/2020

"What, not how": Solving an under-actuated insertion task from scratch

Robot manipulation requires a complex set of skills that need to be care...
10/13/2022

Knowledge acquisition via interactive Distributed Cognitive skill Modules

The human's cognitive capacity for problem solving is always limited to ...
07/28/2022

Learning Dynamic Manipulation Skills from Haptic-Play

In this paper, we propose a data-driven skill learning approach to solve...