DeepAI AI Chat
Log In Sign Up

Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback

by   Xiaofei Wang, et al.
berkeley college

A promising approach to solving challenging long-horizon tasks has been to extract behavior priors (skills) by fitting generative models to large offline datasets of demonstrations. However, such generative models inherit the biases of the underlying data and result in poor and unusable skills when trained on imperfect demonstration data. To better align skill extraction with human intent we present Skill Preferences (SkiP), an algorithm that learns a model over human preferences and uses it to extract human-aligned skills from offline data. After extracting human-preferred skills, SkiP also utilizes human feedback to solve down-stream tasks with RL. We show that SkiP enables a simulated kitchen robot to solve complex multi-step manipulation tasks and substantially outperforms prior leading RL algorithms with human preferences as well as leading skill extraction algorithms without human preferences.


Demonstration-Guided Reinforcement Learning with Learned Skills

Demonstration-guided reinforcement learning (RL) is a promising approach...

Hierarchical Few-Shot Imitation with Skill Transition Models

A desirable property of autonomous agents is the ability to both solve l...

Plan4MC: Skill Reinforcement Learning and Planning for Open-World Minecraft Tasks

We study building a multi-task agent in Minecraft. Without human demonst...

Skill-based Meta-Reinforcement Learning

While deep reinforcement learning methods have shown impressive results ...

"What, not how": Solving an under-actuated insertion task from scratch

Robot manipulation requires a complex set of skills that need to be care...

Knowledge acquisition via interactive Distributed Cognitive skill Modules

The human's cognitive capacity for problem solving is always limited to ...

Learning Dynamic Manipulation Skills from Haptic-Play

In this paper, we propose a data-driven skill learning approach to solve...