Human-level performance in first-person multiplayer games with population-based deep reinforcement learning

07/03/2018
by   Max Jaderberg, et al.
2

Recent progress in artificial intelligence through reinforcement learning (RL) has shown great success on increasingly complex single-agent environments and two-player turn-based games. However, the real-world contains multiple agents, each learning and acting independently to cooperate and compete with other agents, and environments reflecting this degree of complexity remain an open challenge. In this work, we demonstrate for the first time that an agent can achieve human-level in a popular 3D multiplayer first-person video game, Quake III Arena Capture the Flag, using only pixels and game points as input. These results were achieved by a novel two-tier optimisation process in which a population of independent RL agents are trained concurrently from thousands of parallel matches with agents playing in teams together and against each other on randomly generated environments. Each agent in the population learns its own internal reward signal to complement the sparse delayed reward from winning, and selects actions using a novel temporally hierarchical representation that enables the agent to reason at multiple timescales. During game-play, these agents display human-like behaviours such as navigating, following, and defending based on a rich learned representation that is shown to encode high-level game knowledge. In an extensive tournament-style evaluation the trained agents exceeded the win-rate of strong human players both as teammates and opponents, and proved far stronger than existing state-of-the-art agents. These results demonstrate a significant jump in the capabilities of artificial agents, bringing us closer to the goal of human-level intelligence.

READ FULL TEXT

page 3

page 9

page 32

page 33

page 35

page 36

page 37

page 40

research
02/21/2017

Beating the World's Best at Super Smash Bros. with Deep Reinforcement Learning

There has been a recent explosion in the capabilities of game-playing ar...
research
02/04/2019

Obstacle Tower: A Generalization Challenge in Vision, Control, and Planning

The rapid pace of research in Deep Reinforcement Learning has been drive...
research
01/20/2018

Learning model-based strategies in simple environments with hierarchical q-networks

Recent advances in deep learning have allowed artificial agents to rival...
research
12/17/2019

Artificial Agents Learn Flexible Visual Representations by Playing a Hiding Game

The ubiquity of embodied gameplay, observed in a wide variety of animal ...
research
06/13/2018

Learning to Shoot in First Person Shooter Games by Stabilizing Actions and Clustering Rewards for Reinforcement Learning

While reinforcement learning (RL) has been applied to turn-based board g...
research
03/10/2022

Human-Like Navigation Behavior: A Statistical Evaluation Framework

Recent advancements in deep reinforcement learning have brought forth an...
research
07/22/2019

The Tools Challenge: Rapid Trial-and-Error Learning in Physical Problem Solving

Many animals, and an increasing number of artificial agents, display sop...

Please sign up or login with your details

Forgot password? Click here to reset