DeepAI AI Chat
Log In Sign Up

On Reinforcement Learning for Full-length Game of StarCraft

09/23/2018
by   Zhen-Jia Pang, et al.
Nanjing University
0

StarCraft II poses a grand challenge for reinforcement learning. The main difficulties of it include huge state and action space and a long-time horizon. In this paper, we investigate a hierarchical reinforcement learning approach for StarCraft II. The hierarchy involves two levels of abstraction. One is the macro-action automatically extracted from expert's trajectories, which reduces the action space in an order of magnitude yet remains effective. The other is a two-layer hierarchical architecture which is modular and easy to scale, enabling a curriculum transferring from simpler tasks to more complex tasks. The reinforcement training algorithm for this architecture is also investigated. On a 64x64 map and using restrictive units, we achieve a winning rate of more than 99% against the difficulty level-1 built-in AI. Through the curriculum transfer learning algorithm and a mixture of combat model, we can achieve over 93% winning rate of Protoss against the most difficult non-cheating built-in AI (level-7) of Terran, training within two days using a single machine with only 48 CPU cores and 8 K40 GPUs. It also shows strong generalization performance, when tested against never seen opponents including cheating levels built-in AI and all levels of Zerg and Protoss built-in AI. We hope this study could shed some light on the future research of large-scale reinforcement learning.

READ FULL TEXT

page 11

page 13

09/23/2022

On Efficient Reinforcement Learning for Full-length Game of StarCraft II

StarCraft II (SC2) poses a grand challenge for reinforcement learning (R...
04/03/2018

StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learning

Real-time strategy games have been an important field of game artificial...
09/19/2018

TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game

Starcraft II (SCII) is widely considered as the most challenging Real Ti...
06/09/2011

Accelerating Reinforcement Learning by Composing Solutions of Automatically Identified Subtasks

This paper discusses a system that accelerates reinforcement learning by...
11/08/2018

Modular Architecture for StarCraft II with Deep Reinforcement Learning

We present a novel modular architecture for StarCraft II AI. The archite...
03/07/2018

A Brandom-ian view of Reinforcement Learning towards strong-AI

The analytic philosophy of Robert Brandom, based on the ideas of pragmat...
04/14/2021

An Introduction of mini-AlphaStar

StarCraft II (SC2) is a real-time strategy game, in which players produc...