RL Unplugged: Benchmarks for Offline Reinforcement Learning

by   Caglar Gulcehre, et al.

Offline methods for reinforcement learning have the potential to help bridge the gap between reinforcement learning research and real-world applications. They make it possible to learn policies from offline datasets, thus overcoming concerns associated with online data collection in the real-world, including cost, safety, or ethical concerns. In this paper, we propose a benchmark called RL Unplugged to evaluate and compare offline RL methods. RL Unplugged includes data from a diverse range of domains including games (e.g., Atari benchmark) and simulated motor control problems (e.g. DM Control Suite). The datasets include domains that are partially or fully observable, use continuous or discrete actions, and have stochastic vs. deterministic dynamics. We propose detailed evaluation protocols for each domain in RL Unplugged and provide an extensive analysis of supervised learning and offline RL methods using these protocols. We will release data for all our tasks and open-source all algorithms presented in this paper. We hope that our suite of benchmarks will increase the reproducibility of experiments and make it possible to study challenging tasks with a limited computational budget, thus making RL research both more systematic and more accessible across the community. Moving forward, we view RL Unplugged as a living benchmark suite that will evolve and grow with datasets contributed by the research community and ourselves.


page 1

page 2

page 19


Benchmarking Offline Reinforcement Learning on Real-Robot Hardware

Learning policies from previously recorded data is a promising direction...

RiverBench: an Open RDF Streaming Benchmark Suite

RDF data streaming has been explored by the Semantic Web community from ...

Datasets and Benchmarks for Offline Safe Reinforcement Learning

This paper presents a comprehensive benchmarking suite tailored to offli...

Behaviour Suite for Reinforcement Learning

This paper introduces the Behaviour Suite for Reinforcement Learning, or...

CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning

While Reinforcement Learning has made great strides towards solving ever...

Off-the-Grid MARL: a Framework for Dataset Generation with Baselines for Cooperative Offline Multi-Agent Reinforcement Learning

Being able to harness the power of large, static datasets for developing...

Interpretable performance analysis towards offline reinforcement learning: A dataset perspective

Offline reinforcement learning (RL) has increasingly become the focus of...

Please sign up or login with your details

Forgot password? Click here to reset