Reverb: A Framework For Experience Replay

02/09/2021
by   Albin Cassirer, et al.
0

A central component of training in Reinforcement Learning (RL) is Experience: the data used for training. The mechanisms used to generate and consume this data have an important effect on the performance of RL algorithms. In this paper, we introduce Reverb: an efficient, extensible, and easy to use system designed specifically for experience replay in RL. Reverb is designed to work efficiently in distributed configurations with up to thousands of concurrent clients. The flexible API provides users with the tools to easily and accurately configure the replay buffer. It includes strategies for selecting and removing elements from the buffer, as well as options for controlling the ratio between sampled and inserted elements. This paper presents the core design of Reverb, gives examples of how it can be applied, and provides empirical results of Reverb's performance characteristics.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/04/2017

A Deeper Look at Experience Replay

Experience replay plays an important role in the success of deep reinfor...
research
09/29/2020

Lucid Dreaming for Experience Replay: Refreshing Past States with the Current Policy

Experience replay (ER) improves the data efficiency of off-policy reinfo...
research
11/01/2022

Event Tables for Efficient Experience Replay

Experience replay (ER) is a crucial component of many deep reinforcement...
research
06/08/2020

Balancing a CartPole System with Reinforcement Learning – A Tutorial

In this paper, we provide the details of implementing various reinforcem...
research
08/23/2021

Collect Infer – a fresh look at data-efficient Reinforcement Learning

This position paper proposes a fresh look at Reinforcement Learning (RL)...
research
12/08/2021

Replay For Safety

Experience replay <cit.> is a widely used technique to achieve efficient...
research
10/03/2021

Parallel Actors and Learners: A Framework for Generating Scalable RL Implementations

Reinforcement Learning (RL) has achieved significant success in applicat...

Please sign up or login with your details

Forgot password? Click here to reset