DeepAI AI Chat
Log In Sign Up

Massively Parallel Methods for Deep Reinforcement Learning

by   Arun Nair, et al.

We present the first massively distributed architecture for deep reinforcement learning. This architecture uses four main components: parallel actors that generate new behaviour; parallel learners that are trained from stored experience; a distributed neural network to represent the value function or behaviour policy; and a distributed store of experience. We used our architecture to implement the Deep Q-Network algorithm (DQN). Our distributed algorithm was applied to 49 games from Atari 2600 games from the Arcade Learning Environment, using identical hyperparameters. Our performance surpassed non-distributed DQN in 41 of the 49 games and also reduced the wall-time required to achieve these results by an order of magnitude on most games.


page 3

page 4

page 7

page 8


Control with Distributed Deep Reinforcement Learning: Learn a Better Policy

Distributed approach is a very effective method to improve training effi...

Metaoptimization on a Distributed System for Deep Reinforcement Learning

Training intelligent agents through reinforcement learning is a notoriou...

WALL-E: An Efficient Reinforcement Learning Research Framework

There are two halves to RL systems: experience collection time and polic...

A Convergent and Efficient Deep Q Network Algorithm

Despite the empirical success of the deep Q network (DQN) reinforcement ...

Reinforcement Learning Experience Reuse with Policy Residual Representation

Experience reuse is key to sample-efficient reinforcement learning. One ...

Playing Atari with Six Neurons

Deep reinforcement learning on Atari games maps pixel directly to action...

An adaptive synchronization approach for weights of deep reinforcement learning

Deep Q-Networks (DQN) is one of the most well-known methods of deep rein...