DeepAI AI Chat
Log In Sign Up

"What, not how": Solving an under-actuated insertion task from scratch

by   Giulia Vezzani, et al.

Robot manipulation requires a complex set of skills that need to be carefully combined and coordinated to solve a task. Yet, most ReinforcementLearning (RL) approaches in robotics study tasks which actually consist only of a single manipulation skill, such as grasping an object or inserting a pre-grasped object. As a result the skill ('how' to solve the task) but not the actual goal of a complete manipulation ('what' to solve) is specified. In contrast, we study a complex manipulation goal that requires an agent to learn and combine diverse manipulation skills. We propose a challenging, highly under-actuated peg-in-hole task with a free, rotational asymmetrical peg, requiring a broad range of manipulation skills. While correct peg (re-)orientation is a requirement for successful insertion, there is no reward associated with it. Hence an agent needs to understand this pre-condition and learn the skill to fulfil it. The final insertion reward is sparse, allowing freedom in the solution and leading to complex emerging behaviour not envisioned during the task design. We tackle the problem in a multi-task RL framework using Scheduled Auxiliary Control (SAC-X) combined with Regularized Hierarchical Policy Optimization (RHPO) which successfully solves the task in simulation and from scratch on a single robot where data is severely limited.


page 2

page 8

page 12

page 15


Multi-skill Mobile Manipulation for Object Rearrangement

We study a modular approach to tackle long-horizon mobile manipulation t...

Efficient Skill Acquisition for Complex Manipulation Tasks in Obstructed Environments

Data efficiency in robotic skill acquisition is crucial for operating ro...

Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discovery

Current reinforcement learning (RL) in robotics often experiences diffic...

Learning by Playing - Solving Sparse Reward Tasks from Scratch

We propose Scheduled Auxiliary Control (SAC-X), a new learning paradigm ...

Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback

A promising approach to solving challenging long-horizon tasks has been ...

Object Manipulation Learning by Imitation

We aim to enable robot to learn object manipulation by imitation. Given ...

Autonomous Extension of a Symbolic Mobile Manipulation Skill Set

Today's methods of programming mobile manipulation systems' behavior for...