TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning

by   Peng Sun, et al.

Competitive Self-Play (CSP) based Multi-Agent Reinforcement Learning (MARL) has shown phenomenal breakthroughs recently. Strong AIs are achieved for several benchmarks, including Dota 2, Glory of Kings, Quake III, StarCraft II, to name a few. Despite the success, the MARL training is extremely data thirsty, requiring typically billions of (if not trillions of) frames be seen from the environment during training in order for learning a high performance agent. This poses non-trivial difficulties for researchers or engineers and prevents the application of MARL to a broader range of real-world problems. To address this issue, in this manuscript we describe a framework, referred to as TLeague, that aims at large-scale training and implements several main-stream CSP-MARL algorithms. The training can be deployed in either a single machine or a cluster of hybrid machines (CPUs and GPUs), where the standard Kubernetes is supported in a cloud native manner. TLeague achieves a high throughput and a reasonable scale-up when performing distributed training. Thanks to the modular design, it is also easy to extend for solving other multi-agent problems or implementing and verifying MARL algorithms. We present experiments over StarCraft II, ViZDoom and Pommerman to show the efficiency and effectiveness of TLeague. The code is open-sourced and available at


page 6

page 12


Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning

Increasing the scale of reinforcement learning experiments has allowed r...

Efficient Distributed Framework for Collaborative Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning for incomplete information environmen...

Arena: a toolkit for Multi-Agent Reinforcement Learning

We introduce Arena, a toolkit for multi-agent reinforcement learning (MA...

Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space

Learning competitive behaviors in multi-agent settings such as racing re...

Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learning

Solving complex real-world tasks, e.g., autonomous fleet control, often ...

Coding for Distributed Multi-Agent Reinforcement Learning

This paper aims to mitigate straggler effects in synchronous distributed...

VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning

While many multi-robot coordination problems can be solved optimally by ...

Code Repositories