Deep RTS: A Game Environment for Deep Reinforcement Learning in Real-Time Strategy Games

by   Per-Arne Andersen, et al.

Reinforcement learning (RL) is an area of research that has blossomed tremendously in recent years and has shown remarkable potential for artificial intelligence based opponents in computer games. This success is primarily due to the vast capabilities of convolutional neural networks, that can extract useful features from noisy and complex data. Games are excellent tools to test and push the boundaries of novel RL algorithms because they give valuable insight into how well an algorithm can perform in isolated environments without the real-life consequences. Real-time strategy games (RTS) is a genre that has tremendous complexity and challenges the player in short and long-term planning. There is much research that focuses on applied RL in RTS games, and novel advances are therefore anticipated in the not too distant future. However, there are to date few environments for testing RTS AIs. Environments in the literature are often either overly simplistic, such as microRTS, or complex and without the possibility for accelerated learning on consumer hardware like StarCraft II. This paper introduces the Deep RTS game environment for testing cutting-edge artificial intelligence algorithms for RTS games. Deep RTS is a high-performance RTS game made specifically for artificial intelligence research. It supports accelerated learning, meaning that it can learn at a magnitude of 50 000 times faster compared to existing RTS games. Deep RTS has a flexible configuration, enabling research in several different RTS scenarios, including partially observable state-spaces and map complexity. We show that Deep RTS lives up to our promises by comparing its performance with microRTS, ELF, and StarCraft II on high-end consumer hardware. Using Deep RTS, we show that a Deep Q-Network agent beats random-play agents over 70 the time. Deep RTS is publicly available at


page 4

page 5


Deep Reinforcement Learning using Capsules in Advanced Game Environments

Reinforcement Learning (RL) is a research area that has blossomed tremen...

FlashRL: A Reinforcement Learning Platform for Flash Games

Reinforcement Learning (RL) is a research area that has blossomed tremen...

Testing match-3 video games with Deep Reinforcement Learning

Testing a video game is a critical step for the production process and r...

Pgx: Hardware-accelerated parallel game simulation for reinforcement learning

We propose Pgx, a collection of board game simulators written in JAX. Th...

An electronic-game framework for evaluating coevolutionary algorithms

One of the common artificial intelligence applications in electronic gam...

ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

In this paper, we propose ELF, an Extensive, Lightweight and Flexible pl...

Deep Reinforcement Learning for Navigation in AAA Video Games

In video games, non-player characters (NPCs) are used to enhance the pla...

Please sign up or login with your details

Forgot password? Click here to reset