Acme: A Research Framework for Distributed Reinforcement Learning

06/01/2020
by   Matt Hoffman, et al.
22

Deep reinforcement learning has led to many recent-and groundbreaking-advancements. However, these advances have often come at the cost of both the scale and complexity of the underlying RL algorithms. Increases in complexity have in turn made it more difficult for researchers to reproduce published RL algorithms or rapidly prototype ideas. To address this, we introduce Acme, a tool to simplify the development of novel RL algorithms that is specifically designed to enable simple agent implementations that can be run at various scales of execution. Our aim is also to make the results of various RL algorithms developed in academia and industrial labs easier to reproduce and extend. To this end we are releasing baseline implementations of various algorithms, created using our framework. In this work we introduce the major design decisions behind Acme and show how these are used to construct these baselines. We also experiment with these agents at different scales of both complexity and computation-including distributed versions. Ultimately, we show that the design decisions behind Acme lead to agents that can be scaled both up and down and that, for the most part, greater levels of parallelization result in agents with equivalent performance, just faster.

READ FULL TEXT
12/14/2018

Dopamine: A Research Framework for Deep Reinforcement Learning

Deep reinforcement learning (deep RL) research has grown significantly i...
05/18/2022

Generating Explanations from Deep Reinforcement Learning Using Episodic Memory

Deep Reinforcement Learning (RL) involves the use of Deep Neural Network...
11/25/2020

Distributed Reinforcement Learning is a Dataflow Problem

Researchers and practitioners in the field of reinforcement learning (RL...
06/10/2020

What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study

In recent years, on-policy reinforcement learning (RL) has been successf...
02/28/2019

Catalyst.RL: A Distributed Framework for Reproducible RL Research

Despite the recent progress in deep reinforcement learning field (RL), a...
07/10/2020

Vizarel: A System to Help Better Understand RL Agents

Visualization tools for supervised learning have allowed users to interp...
10/21/2018

RLgraph: Flexible Computation Graphs for Deep Reinforcement Learning

Reinforcement learning (RL) tasks are challenging to implement, execute ...

Code Repositories

acme

A library of reinforcement learning components and agents


view repo

rlflow

Abstractions and demos for a multicore and distributed RL framework


view repo

hedgingbox

RL Hedge Box using ACME Framework


view repo