Asynchronous Methods for Deep Reinforcement Learning

02/04/2016
by   Volodymyr Mnih, et al.
0

We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers. We present asynchronous variants of four standard reinforcement learning algorithms and show that parallel actor-learners have a stabilizing effect on training allowing all four methods to successfully train neural network controllers. The best performing method, an asynchronous variant of actor-critic, surpasses the current state-of-the-art on the Atari domain while training for half the time on a single multi-core CPU instead of a GPU. Furthermore, we show that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input.

READ FULL TEXT

page 8

page 17

05/19/2017

Atari games and Intel processors

The asynchronous nature of the state-of-the-art reinforcement learning a...
12/21/2020

Combining Deep Reinforcement Learning And Local Control For The Acrobot Swing-up And Balance Task

In this work we present a novel extension of soft actor critic, a state ...
10/28/2019

Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control

Current Deep Reinforcement Learning algorithms still heavily rely on han...
08/19/2017

A Brief Survey of Deep Reinforcement Learning

Deep reinforcement learning is poised to revolutionise the field of AI a...
08/13/2018

Visual Sensor Network Reconfiguration with Deep Reinforcement Learning

We present an approach for reconfiguration of dynamic visual sensor netw...
06/09/2019

Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning

Multi-simulator training has contributed to the recent success of Deep R...
12/28/2019

SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning

We introduce SLM Lab, a software framework for reproducible reinforcemen...

Code Repositories

async_deep_reinforce

Asynchronous Methods for Deep Reinforcement Learning


view repo

async-rl

Replicating "Asynchronous Methods for Deep Reinforcement Learning" (http://arxiv.org/abs/1602.01783)


view repo

async-deep-rl

A Tensorflow based implementation of "Asynchronous Methods for Deep Reinforcement Learning": https://arxiv.org/abs/1602.01783


view repo

deep_rl_acrobot

Using deep reinforcement learning (DDPG & A3C) to solve Acrobot


view repo

a3c-distributed_tensorflow

Distributed Tensorflow Implementation of Asynchronous Methods for Deep Reinforcement Learning


view repo