Mastering Atari with Discrete World Models

10/05/2020
by   Danijar Hafner, et al.
17

Intelligent agents need to generalize from past experience to achieve goals in complex environments. World models facilitate such generalization and allow learning behaviors from imagined outcomes to increase sample-efficiency. While learning world models from image inputs has recently become feasible for some tasks, modeling Atari games accurately enough to derive successful behaviors has remained an open challenge for many years. We introduce DreamerV2, a reinforcement learning agent that learns behaviors purely from predictions in the compact latent space of a powerful world model. The world model uses discrete representations and is trained separately from the policy. DreamerV2 constitutes the first agent that achieves human-level performance on the Atari benchmark of 55 tasks by learning behaviors inside a separately trained world model. With the same computational budget and wall-clock time, DreamerV2 reaches 200M frames and exceeds the final performance of the top single-GPU agents IQN and Rainbow.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/03/2019

Dream to Control: Learning Behaviors by Latent Imagination

Learned world models summarize an agent's experience to facilitate learn...
research
06/08/2022

Deep Hierarchical Planning from Pixels

Intelligent agents need to select long sequences of actions to solve com...
research
10/26/2022

Leveraging Demonstrations with Latent Space Priors

Demonstrations provide insight into relevant state or action space regio...
research
10/12/2020

Discrete Latent Space World Models for Reinforcement Learning

Sample efficiency remains a fundamental issue of reinforcement learning....
research
02/12/2019

Deep Reinforcement Learning from Policy-Dependent Human Feedback

To widen their accessibility and increase their utility, intelligent age...
research
02/21/2018

Learning to Play with Intrinsically-Motivated Self-Aware Agents

Infants are experts at playing, with an amazing ability to generate nove...
research
09/04/2018

Recurrent World Models Facilitate Policy Evolution

A generative recurrent neural network is quickly trained in an unsupervi...

Please sign up or login with your details

Forgot password? Click here to reset