DeepAI AI Chat
Log In Sign Up

Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning

06/09/2019
by   Mahmoud Assran, et al.
Facebook
McGill University
0

Multi-simulator training has contributed to the recent success of Deep Reinforcement Learning by stabilizing learning and allowing for higher training throughputs. We propose Gossip-based Actor-Learner Architectures (GALA) where several actor-learners (such as A2C agents) are organized in a peer-to-peer communication topology, and exchange information through asynchronous gossip in order to take advantage of a large number of distributed simulators. We prove that GALA agents remain within an epsilon-ball of one-another during training when using loosely coupled asynchronous communication. By reducing the amount of synchronization between agents, GALA is more computationally efficient and scalable compared to A2C, its fully-synchronous counterpart. GALA also outperforms A2C, being more robust and sample efficient. We show that we can run several loosely coupled GALA agents in parallel on a single GPU and achieve significantly higher hardware utilization and frame-rates than vanilla A2C at comparable power draws.

READ FULL TEXT
02/04/2016

Asynchronous Methods for Deep Reinforcement Learning

We propose a conceptually simple and lightweight framework for deep rein...
03/01/2020

Asynchronous Policy Evaluation in Distributed Reinforcement Learning over Networks

This paper proposes a fully asynchronous scheme for policy evaluation of...
04/04/2021

Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation

Many real-world applications such as robotics provide hard constraints o...
04/13/2018

Robust Dual View Deep Agent

Motivated by recent advance of machine learning using Deep Reinforcement...
06/21/2020

Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning

Increasing the scale of reinforcement learning experiments has allowed r...
04/13/2018

Robust Dual View Depp Agent

Motivated by recent advance of machine learning using Deep Reinforcement...
11/28/2022

AcceRL: Policy Acceleration Framework for Deep Reinforcement Learning

Deep reinforcement learning has achieved great success in various fields...