Let's Play Again: Variability of Deep Reinforcement Learning Agents in Atari Environments

by   Kaleigh Clary, et al.
University of Massachusetts Amherst
Smith College

Reproducibility in reinforcement learning is challenging: uncontrolled stochasticity from many sources, such as the learning algorithm, the learned policy, and the environment itself have led researchers to report the performance of learned agents using aggregate metrics of performance over multiple random seeds for a single environment. Unfortunately, there are still pernicious sources of variability in reinforcement learning agents that make reporting common summary statistics an unsound metric for performance. Our experiments demonstrate the variability of common agents used in the popular OpenAI Baselines repository. We make the case for reporting post-training agent performance as a distribution, rather than a point estimate.


page 1

page 2

page 3

page 4


Imagination-Augmented Agents for Deep Reinforcement Learning

We introduce Imagination-Augmented Agents (I2As), a novel architecture f...

Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers

In this paper, we propose a new data poisoning attack and apply it to de...

Deep Reinforcement Learning that Matters

In recent years, significant progress has been made in solving challengi...

Continual Reinforcement Learning with TELLA

Training reinforcement learning agents that continually learn across mul...

Deterministic Implementations for Reproducibility in Deep Reinforcement Learning

While deep reinforcement learning (DRL) has led to numerous successes in...

Co-design of Embodied Neural Intelligence via Constrained Evolution

We introduce a novel co-design method for autonomous moving agents' shap...

FedFormer: Contextual Federation with Attention in Reinforcement Learning

A core issue in federated reinforcement learning is defining how to aggr...

Please sign up or login with your details

Forgot password? Click here to reset