TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game

by   Lei Han, et al.

StarCraft, one of the most difficult esport games with long-standing history of professional tournaments, has attracted generations of players and fans, and also, intense attentions in artificial intelligence research. Recently, Google's DeepMind announced AlphaStar, a grandmaster level AI in StarCraft II. In this paper, we introduce a new AI agent, named TStarBot-X, that is trained under limited computation resources and can play competitively with expert human players. TStarBot-X takes advantage of important techniques introduced in AlphaStar, and also benefits from substantial innovations including new league training methods, novel multi-agent roles, rule-guided policy search, lightweight neural network architecture, and importance sampling in imitation learning, etc. We show that with limited computation resources, a faithful reimplementation of AlphaStar can not succeed and the proposed techniques are necessary to ensure TStarBot-X's competitive performance. We reveal all technical details that are complementary to those mentioned in AlphaStar, showing the most sensitive parts in league training, reinforcement learning and imitation learning that affect the performance of the agents. Most importantly, this is an open-sourced study that all codes and resources (including the trained model parameters) are publicly accessible via https://github.com/tencent-ailab/tleague_projpage We expect this study could be beneficial for both academic and industrial future research in solving complex problems like StarCraft, and also, might provide a sparring partner for all StarCraft II players and other AI agents.


page 9

page 10

page 17

page 21


SCC: an efficient deep reinforcement learning agent mastering the game of StarCraft II

AlphaStar, the AI that reaches GrandMaster level in StarCraft II, is a r...

TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game

Starcraft II (SCII) is widely considered as the most challenging Real Ti...

No Press Diplomacy: Modeling Multi-Agent Gameplay

Diplomacy is a seven-player non-stochastic, non-cooperative game, where ...

On Efficient Reinforcement Learning for Full-length Game of StarCraft II

StarCraft II (SC2) poses a grand challenge for reinforcement learning (R...

Accelerating Training in Pommerman with Imitation and Reinforcement Learning

The Pommerman simulation was recently developed to mimic the classic Jap...

The Arcade Learning Environment: An Evaluation Platform for General Agents

In this article we introduce the Arcade Learning Environment (ALE): both...

Sample Efficient Training in Multi-Agent Adversarial Games with Limited Teammate Communication

We describe our solution approach for Pommerman TeamRadio, a competition...

Code Repositories