MinAtar: An Atari-inspired Testbed for More Efficient Reinforcement Learning Experiments

03/07/2019
by   Kenny Young, et al.
0

The Arcade Learning Environment (ALE) is a popular platform for evaluating reinforcement learning agents. Much of the appeal comes from the fact that Atari games are varied, showcase aspects of competency we expect from an intelligent agent, and are not biased towards any particular solution approach. The challenge of the ALE includes 1) the representation learning problem of extracting pertinent information from the raw pixels, and 2) the behavioural learning problem of leveraging complex, delayed associations between actions and rewards. Often, in reinforcement learning research, we care more about the latter, but the representation learning problem adds significant computational expense. In response, we introduce MinAtar, short for miniature Atari, a new evaluation platform that captures the general mechanics of specific Atari games, while simplifying certain aspects. In particular, we reduce the representational complexity to focus more on behavioural challenges. MinAtar consists of analogues to five Atari games which play out on a 10x10 grid. MinAtar provides a 10x10xn state representation. The n channels correspond to game-specific objects, such as ball, paddle and brick in the game Breakout. While significantly simplified, these domains are still rich enough to allow for interesting behaviours. To demonstrate the challenges posed by these domains, we evaluated a smaller version of the DQN architecture. We also tried variants of DQN without experience replay, and without a target network, to assess the impact of those two prominent components in the MinAtar environments. In addition, we evaluated a simpler agent that used actor-critic with eligibility traces, online updating, and no experience replay. We hope that by introducing a set of simplified, Atari-like games we can allow researchers to more efficiently investigate the unique behavioural challenges provided by the ALE.

READ FULL TEXT
research
09/25/2019

Off-Policy Actor-Critic with Shared Experience Replay

We investigate the combination of actor-critic reinforcement learning al...
research
07/18/2016

Playing Atari Games with Deep Reinforcement Learning and Human Checkpoint Replay

This paper introduces a novel method for learning how to play the most d...
research
07/24/2019

Terminal Prediction as an Auxiliary Task for Deep Reinforcement Learning

Deep reinforcement learning has achieved great successes in recent years...
research
06/29/2018

TextWorld: A Learning Environment for Text-based Games

We introduce TextWorld, a sandbox learning environment for the training ...
research
07/19/2012

The Arcade Learning Environment: An Evaluation Platform for General Agents

In this article we introduce the Arcade Learning Environment (ALE): both...
research
02/28/2012

Relational Reinforcement Learning in Infinite Mario

Relational representations in reinforcement learning allow for the use o...
research
04/23/2018

Crawling in Rogue's dungeons with (partitioned) A3C

Rogue is a famous dungeon-crawling video-game of the 80ies, the ancestor...

Please sign up or login with your details

Forgot password? Click here to reset