Robot Learning of Mobile Manipulation with Reachability Behavior Priors

03/08/2022
by   Snehal Jauhri, et al.
5

Mobile Manipulation (MM) systems are ideal candidates for taking up the role of a personal assistant in unstructured real-world environments. Among other challenges, MM requires effective coordination of the robot's embodiments for executing tasks that require both mobility and manipulation. Reinforcement Learning (RL) holds the promise of endowing robots with adaptive behaviors, but most methods require prohibitively large amounts of data for learning a useful control policy. In this work, we study the integration of robotic reachability priors in actor-critic RL methods for accelerating the learning of MM for reaching and fetching tasks. Namely, we consider the problem of optimal base placement and the subsequent decision of whether to activate the arm for reaching a 6D target. For this, we devise a novel Hybrid RL method that handles discrete and continuous actions jointly, resorting to the Gumbel-Softmax reparameterization. Next, we train a reachability prior using data from the operational robot workspace, inspired by classical methods. Subsequently, we derive Boosted Hybrid RL (BHyRL), a novel algorithm for learning Q-functions by modeling them as a sum of residual approximators. Every time a new task needs to be learned, we can transfer our learned residuals and learn the component of the Q-function that is task-specific, hence, maintaining the task structure from prior behaviors. Moreover, we find that regularizing the target policy with a prior policy yields more expressive behaviors. We evaluate our method in simulation in reaching and fetching tasks of increasing difficulty, and we show the superior performance of BHyRL against baseline methods. Finally, we zero-transfer our learned 6D fetching policy with BHyRL to our MM robot TIAGo++. For more details and code release, please refer to our project site: irosalab.com/rlmmbp

READ FULL TEXT

page 1

page 6

page 7

research
08/13/2019

Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real

Manipulation and locomotion are closely related problems that are often ...
research
05/16/2023

Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions

Reinforcement learning (RL) exhibits impressive performance when managin...
research
03/24/2021

CLAMGen: Closed-Loop Arm Motion Generation via Multi-view Vision-Based RL

We propose a vision-based reinforcement learning (RL) approach for close...
research
11/27/2020

Adaptable Automation with Modular Deep Reinforcement Learning and Policy Transfer

Recent advances in deep Reinforcement Learning (RL) have created unprece...
research
03/15/2022

Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learning

Most successes in robotic manipulation have been restricted to single-ar...
research
10/28/2021

Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives

Despite the potential of reinforcement learning (RL) for building genera...
research
03/09/2023

Learning Arm-Assisted Fall Damage Reduction and Recovery for Legged Mobile Manipulators

Adaptive falling and recovery skills greatly extend the applicability of...

Please sign up or login with your details

Forgot password? Click here to reset