Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates

10/03/2016
by   Shixiang Gu, et al.
0

Reinforcement learning holds the promise of enabling autonomous robots to learn large repertoires of behavioral skills with minimal human intervention. However, robotic applications of reinforcement learning often compromise the autonomy of the learning process in favor of achieving training times that are practical for real physical systems. This typically involves introducing hand-engineered policy representations and human-supplied demonstrations. Deep reinforcement learning alleviates this limitation by training general-purpose neural network policies, but applications of direct deep reinforcement learning algorithms have so far been restricted to simulated settings and relatively simple tasks, due to their apparent high sample complexity. In this paper, we demonstrate that a recent deep reinforcement learning algorithm based on off-policy training of deep Q-functions can scale to complex 3D manipulation tasks and can learn deep neural network policies efficiently enough to train on real physical robots. We demonstrate that the training times can be further reduced by parallelizing the algorithm across multiple robots which pool their policy updates asynchronously. Our experimental evaluation shows that our method can learn a variety of 3D manipulation skills in simulation and a complex door opening skill on real robots without any prior demonstrations or manually designed representations.

READ FULL TEXT

page 1

page 4

page 7

02/08/2021

Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review

Deep learning has provided new ways of manipulating, processing and anal...
09/28/2017

Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations

Dexterous multi-fingered hands are extremely versatile and provide a gen...
10/03/2016

Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Search

In principle, reinforcement learning and policy search methods can enabl...
05/27/2019

Hypothesis-Driven Skill Discovery for Hierarchical Deep Reinforcement Learning

Deep reinforcement learning encompasses many versatile tools for designi...
08/23/2018

LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations

Reinforcement learning approaches have long appealed to the data managem...
07/28/2021

Fully Autonomous Real-World Reinforcement Learning for Mobile Manipulation

We study how robots can autonomously learn skills that require a combina...
11/18/2017

Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning

Deep reinforcement learning algorithms can learn complex behavioral skil...