Model-Based Reinforcement Learning for Atari

by   Łukasz Kaiser, et al.

Model-free reinforcement learning (RL) can be used to learn effective policies for complex tasks, such as Atari games, even from image observations. However, this typically requires very large amounts of interaction -- substantially more, in fact, than a human would need to learn the same games. How can people learn so quickly? Part of the answer may be that people can learn how the game works and predict which actions will lead to desirable outcomes. In this paper, we explore how video prediction models can similarly enable agents to solve Atari games with orders of magnitude fewer interactions than model-free methods. We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting. Our experiments evaluate SimPLe on a range of Atari games and achieve competitive results with only 100K interactions between the agent and the environment (400K frames), which corresponds to about two hours of real-time play.


page 2

page 4

page 7

page 8

page 9

page 10


Deep vs. Deep Bayesian: Reinforcement Learning on a Multi-Robot Competitive Experiment

Deep Reinforcement Learning (RL) experiments are commonly performed in s...

RLFlow: Optimising Neural Network Subgraph Transformation with World Models

We explored the use of reinforcement learning (RL) agents that can learn...

On Multi-Agent Learning in Team Sports Games

In recent years, reinforcement learning has been successful in solving v...

A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games

Reinforcement learning is concerned with identifying reward-maximizing b...

Learning Policies from Human Data for Skat

Decision-making in large imperfect information games is difficult. Thank...

Trial without Error: Towards Safe Reinforcement Learning via Human Intervention

AI systems are increasingly applied to complex tasks that involve intera...

Loss of Plasticity in Continual Deep Reinforcement Learning

The ability to learn continually is essential in a complex and changing ...

Code Repositories

Please sign up or login with your details

Forgot password? Click here to reset