Asynchronous Methods for Deep Reinforcement Learning

by   Volodymyr Mnih, et al.

We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers. We present asynchronous variants of four standard reinforcement learning algorithms and show that parallel actor-learners have a stabilizing effect on training allowing all four methods to successfully train neural network controllers. The best performing method, an asynchronous variant of actor-critic, surpasses the current state-of-the-art on the Atari domain while training for half the time on a single multi-core CPU instead of a GPU. Furthermore, we show that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input.


page 8

page 17


Atari games and Intel processors

The asynchronous nature of the state-of-the-art reinforcement learning a...

Combining Deep Reinforcement Learning And Local Control For The Acrobot Swing-up And Balance Task

In this work we present a novel extension of soft actor critic, a state ...

Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control

Current Deep Reinforcement Learning algorithms still heavily rely on han...

A Brief Survey of Deep Reinforcement Learning

Deep reinforcement learning is poised to revolutionise the field of AI a...

Visual Sensor Network Reconfiguration with Deep Reinforcement Learning

We present an approach for reconfiguration of dynamic visual sensor netw...

Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning

Multi-simulator training has contributed to the recent success of Deep R...

SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning

We introduce SLM Lab, a software framework for reproducible reinforcemen...

Code Repositories


Asynchronous Methods for Deep Reinforcement Learning

view repo


Replicating "Asynchronous Methods for Deep Reinforcement Learning" (

view repo


A Tensorflow based implementation of "Asynchronous Methods for Deep Reinforcement Learning":

view repo


Using deep reinforcement learning (DDPG & A3C) to solve Acrobot

view repo


Distributed Tensorflow Implementation of Asynchronous Methods for Deep Reinforcement Learning

view repo