DeepAI AI Chat
Log In Sign Up

Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning

by   Mahmoud Assran, et al.
McGill University

Multi-simulator training has contributed to the recent success of Deep Reinforcement Learning by stabilizing learning and allowing for higher training throughputs. We propose Gossip-based Actor-Learner Architectures (GALA) where several actor-learners (such as A2C agents) are organized in a peer-to-peer communication topology, and exchange information through asynchronous gossip in order to take advantage of a large number of distributed simulators. We prove that GALA agents remain within an epsilon-ball of one-another during training when using loosely coupled asynchronous communication. By reducing the amount of synchronization between agents, GALA is more computationally efficient and scalable compared to A2C, its fully-synchronous counterpart. GALA also outperforms A2C, being more robust and sample efficient. We show that we can run several loosely coupled GALA agents in parallel on a single GPU and achieve significantly higher hardware utilization and frame-rates than vanilla A2C at comparable power draws.


Asynchronous Methods for Deep Reinforcement Learning

We propose a conceptually simple and lightweight framework for deep rein...

Asynchronous Policy Evaluation in Distributed Reinforcement Learning over Networks

This paper proposes a fully asynchronous scheme for policy evaluation of...

Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation

Many real-world applications such as robotics provide hard constraints o...

Robust Dual View Deep Agent

Motivated by recent advance of machine learning using Deep Reinforcement...

Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning

Increasing the scale of reinforcement learning experiments has allowed r...

Robust Dual View Depp Agent

Motivated by recent advance of machine learning using Deep Reinforcement...

AcceRL: Policy Acceleration Framework for Deep Reinforcement Learning

Deep reinforcement learning has achieved great success in various fields...