Human-level performance in first-person multiplayer games with population-based deep reinforcement learning

by   Max Jaderberg, et al.

Recent progress in artificial intelligence through reinforcement learning (RL) has shown great success on increasingly complex single-agent environments and two-player turn-based games. However, the real-world contains multiple agents, each learning and acting independently to cooperate and compete with other agents, and environments reflecting this degree of complexity remain an open challenge. In this work, we demonstrate for the first time that an agent can achieve human-level in a popular 3D multiplayer first-person video game, Quake III Arena Capture the Flag, using only pixels and game points as input. These results were achieved by a novel two-tier optimisation process in which a population of independent RL agents are trained concurrently from thousands of parallel matches with agents playing in teams together and against each other on randomly generated environments. Each agent in the population learns its own internal reward signal to complement the sparse delayed reward from winning, and selects actions using a novel temporally hierarchical representation that enables the agent to reason at multiple timescales. During game-play, these agents display human-like behaviours such as navigating, following, and defending based on a rich learned representation that is shown to encode high-level game knowledge. In an extensive tournament-style evaluation the trained agents exceeded the win-rate of strong human players both as teammates and opponents, and proved far stronger than existing state-of-the-art agents. These results demonstrate a significant jump in the capabilities of artificial agents, bringing us closer to the goal of human-level intelligence.


page 3

page 9

page 32

page 33

page 35

page 36

page 37

page 40


Beating the World's Best at Super Smash Bros. with Deep Reinforcement Learning

There has been a recent explosion in the capabilities of game-playing ar...

Obstacle Tower: A Generalization Challenge in Vision, Control, and Planning

The rapid pace of research in Deep Reinforcement Learning has been drive...

Learning model-based strategies in simple environments with hierarchical q-networks

Recent advances in deep learning have allowed artificial agents to rival...

Artificial Agents Learn Flexible Visual Representations by Playing a Hiding Game

The ubiquity of embodied gameplay, observed in a wide variety of animal ...

Learning to Shoot in First Person Shooter Games by Stabilizing Actions and Clustering Rewards for Reinforcement Learning

While reinforcement learning (RL) has been applied to turn-based board g...

Human-Like Navigation Behavior: A Statistical Evaluation Framework

Recent advancements in deep reinforcement learning have brought forth an...

The Tools Challenge: Rapid Trial-and-Error Learning in Physical Problem Solving

Many animals, and an increasing number of artificial agents, display sop...

Please sign up or login with your details

Forgot password? Click here to reset