Acme: A Research Framework for Distributed Reinforcement Learning

by   Matt Hoffman, et al.

Deep reinforcement learning has led to many recent-and groundbreaking-advancements. However, these advances have often come at the cost of both the scale and complexity of the underlying RL algorithms. Increases in complexity have in turn made it more difficult for researchers to reproduce published RL algorithms or rapidly prototype ideas. To address this, we introduce Acme, a tool to simplify the development of novel RL algorithms that is specifically designed to enable simple agent implementations that can be run at various scales of execution. Our aim is also to make the results of various RL algorithms developed in academia and industrial labs easier to reproduce and extend. To this end we are releasing baseline implementations of various algorithms, created using our framework. In this work we introduce the major design decisions behind Acme and show how these are used to construct these baselines. We also experiment with these agents at different scales of both complexity and computation-including distributed versions. Ultimately, we show that the design decisions behind Acme lead to agents that can be scaled both up and down and that, for the most part, greater levels of parallelization result in agents with equivalent performance, just faster.


Dopamine: A Research Framework for Deep Reinforcement Learning

Deep reinforcement learning (deep RL) research has grown significantly i...

Generating Explanations from Deep Reinforcement Learning Using Episodic Memory

Deep Reinforcement Learning (RL) involves the use of Deep Neural Network...

Distributed Reinforcement Learning is a Dataflow Problem

Researchers and practitioners in the field of reinforcement learning (RL...

What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study

In recent years, on-policy reinforcement learning (RL) has been successf...

Catalyst.RL: A Distributed Framework for Reproducible RL Research

Despite the recent progress in deep reinforcement learning field (RL), a...

Mava: a research framework for distributed multi-agent reinforcement learning

Breakthrough advances in reinforcement learning (RL) research have led t...

Vizarel: A System to Help Better Understand RL Agents

Visualization tools for supervised learning have allowed users to interp...

Code Repositories


A library of reinforcement learning components and agents

view repo


Abstractions and demos for a multicore and distributed RL framework

view repo


RL Hedge Box using ACME Framework

view repo

Please sign up or login with your details

Forgot password? Click here to reset