Efficient Scheduling of Data Augmentation for Deep Reinforcement Learning

06/01/2022
by   Byungchan Ko, et al.
0

In deep reinforcement learning (RL), data augmentation is widely considered as a tool to induce a set of useful priors about semantic consistency and improve sample efficiency and generalization performance. However, even when the prior is useful for generalization, distilling it to RL agent often interferes with RL training and degenerates sample efficiency. Meanwhile, the agent is forgetful of the prior due to the non-stationary nature of RL. These observations suggest two extreme schedules of distillation: (i) over the entire training; or (ii) only at the end. Hence, we devise a stand-alone network distillation method to inject the consistency prior at any time (even after RL), and a simple yet efficient framework to automatically schedule the distillation. Specifically, the proposed framework first focuses on mastering train environments regardless of generalization by adaptively deciding which or no augmentation to be used for the training. After this, we add the distillation to extract the remaining benefits for generalization from all the augmentations, which requires no additional new samples. In our experiments, we demonstrate the utility of the proposed framework, in particular, that considers postponing the augmentation to the end of RL training.

READ FULL TEXT

page 12

page 16

research
06/23/2020

Automatic Data Augmentation for Generalization in Deep Reinforcement Learning

Deep reinforcement learning (RL) agents often fail to generalize to unse...
research
02/17/2021

Time Matters in Using Data Augmentation for Vision-based Deep Reinforcement Learning

Data augmentation technique from computer vision has been widely conside...
research
06/29/2021

Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation

The generalization gap in reinforcement learning (RL) has been a signifi...
research
07/15/2022

Bootstrap State Representation using Style Transfer for Better Generalization in Deep Reinforcement Learning

Deep Reinforcement Learning (RL) agents often overfit the training envir...
research
10/11/2019

A Simple Randomization Technique for Generalization in Deep Reinforcement Learning

Deep reinforcement learning (RL) agents often fail to generalize to unse...
research
05/13/2020

Proxy Experience Replay: Federated Distillation for Distributed Reinforcement Learning

Traditional distributed deep reinforcement learning (RL) commonly relies...
research
05/13/2020

Proxy Experience Replay: Federated Distillation for Distributed Reinforcement Leargning

Traditional distributed deep reinforcement learning (RL) commonly relies...

Please sign up or login with your details

Forgot password? Click here to reset