Massively Parallel Methods for Deep Reinforcement Learning

07/15/2015
by   Arun Nair, et al.
0

We present the first massively distributed architecture for deep reinforcement learning. This architecture uses four main components: parallel actors that generate new behaviour; parallel learners that are trained from stored experience; a distributed neural network to represent the value function or behaviour policy; and a distributed store of experience. We used our architecture to implement the Deep Q-Network algorithm (DQN). Our distributed algorithm was applied to 49 games from Atari 2600 games from the Arcade Learning Environment, using identical hyperparameters. Our performance surpassed non-distributed DQN in 41 of the 49 games and also reduced the wall-time required to achieve these results by an order of magnitude on most games.

READ FULL TEXT

page 3

page 4

page 7

page 8

research
11/26/2018

Control with Distributed Deep Reinforcement Learning: Learn a Better Policy

Distributed approach is a very effective method to improve training effi...
research
02/07/2019

Metaoptimization on a Distributed System for Deep Reinforcement Learning

Training intelligent agents through reinforcement learning is a notoriou...
research
01/18/2019

WALL-E: An Efficient Reinforcement Learning Research Framework

There are two halves to RL systems: experience collection time and polic...
research
06/29/2021

A Convergent and Efficient Deep Q Network Algorithm

Despite the empirical success of the deep Q network (DQN) reinforcement ...
research
05/31/2019

Reinforcement Learning Experience Reuse with Policy Residual Representation

Experience reuse is key to sample-efficient reinforcement learning. One ...
research
06/04/2018

Playing Atari with Six Neurons

Deep reinforcement learning on Atari games maps pixel directly to action...
research
08/16/2020

An adaptive synchronization approach for weights of deep reinforcement learning

Deep Q-Networks (DQN) is one of the most well-known methods of deep rein...

Please sign up or login with your details

Forgot password? Click here to reset