Pretraining Reinforcement Learning: Sharpening the Axe Before Cutting the Tree

10/06/2021
by   Saurav Kadavath, et al.
0

Pretraining is a common technique in deep learning for increasing performance and reducing training time, with promising experimental results in deep reinforcement learning (RL). However, pretraining requires a relevant dataset for training. In this work, we evaluate the effectiveness of pretraining for RL tasks, with and without distracting backgrounds, using both large, publicly available datasets with minimal relevance, as well as case-by-case generated datasets labeled via self-supervision. Results suggest filters learned during training on less relevant datasets render pretraining ineffective, while filters learned during training on the in-distribution datasets reliably reduce RL training time and improve performance after 80k RL training steps. We further investigate, given a limited number of environment steps, how to optimally divide the available steps into pretraining and RL training to maximize RL performance. Our code is available on GitHub

READ FULL TEXT

page 3

page 5

research
11/08/2022

Pretraining in Deep Reinforcement Learning: A Survey

The past few years have seen rapid progress in combining reinforcement l...
research
10/24/2019

Pre-training in Deep Reinforcement Learning for Automatic Speech Recognition

Deep reinforcement learning (deep RL) is a combination of deep learning ...
research
06/09/2021

Pretraining Representations for Data-Efficient Reinforcement Learning

Data efficiency is a key challenge for deep reinforcement learning. We a...
research
09/09/2018

How clever is the FiLM model, and how clever can it be?

The FiLM model achieves close-to-perfect performance on the diagnostic C...
research
07/24/2022

A Deep Dive into Deep Cluster

Deep Learning has demonstrated a significant improvement against traditi...
research
01/18/2023

PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNav

We study ObjectGoal Navigation - where a virtual robot situated in a new...
research
07/21/2023

Towards practical reinforcement learning for tokamak magnetic control

Reinforcement learning (RL) has shown promising results for real-time co...

Please sign up or login with your details

Forgot password? Click here to reset