Observe and Look Further: Achieving Consistent Performance on Atari

05/29/2018
by   Tobias Pohlen, et al.
0

Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks such as Atari 2600 games. We identify three key challenges that any algorithm needs to master in order to perform well on all games: processing diverse reward distributions, reasoning over long time horizons, and exploring efficiently. In this paper, we propose an algorithm that addresses each of these challenges and is able to learn human-level policies on nearly all Atari games. A new transformed Bellman operator allows our algorithm to process rewards of varying densities and scales; an auxiliary temporal consistency loss allows us to train stably using a discount factor of γ = 0.999 (instead of γ = 0.99) extending the effective planning horizon by an order of magnitude; and we ease the exploration problem by using human demonstrations that guide the agent towards rewarding states. When tested on a set of 42 Atari games, our algorithm exceeds the performance of an average human on 40 games using a common set of hyper parameters. Furthermore, it is the first deep RL algorithm to solve the first level of Montezuma's Revenge.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/30/2020

Agent57: Outperforming the Atari Human Benchmark

Atari games have been a long-standing benchmark in the reinforcement lea...
research
04/05/2022

Jump-Start Reinforcement Learning

Reinforcement learning (RL) provides a theoretical framework for continu...
research
09/12/2018

Multi-task Deep Reinforcement Learning with PopArt

The reinforcement learning community has made great strides in designing...
research
06/03/2022

Challenges to Solving Combinatorially Hard Long-Horizon Deep RL Tasks

Deep reinforcement learning has shown promise in discrete domains requir...
research
05/29/2018

Playing hard exploration games by watching YouTube

Deep reinforcement learning methods traditionally struggle with tasks wh...
research
04/27/2020

First return then explore

The promise of reinforcement learning is to solve complex sequential dec...
research
06/29/2018

Counting to Explore and Generalize in Text-based Games

We propose a recurrent RL agent with an episodic exploration mechanism t...

Please sign up or login with your details

Forgot password? Click here to reset