Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning

05/25/2023
by   Guozheng Ma, et al.
0

Data augmentation (DA) is a crucial technique for enhancing the sample efficiency of visual reinforcement learning (RL) algorithms. Notably, employing simple observation transformations alone can yield outstanding performance without extra auxiliary representation tasks or pre-trained encoders. However, it remains unclear which attributes of DA account for its effectiveness in achieving sample-efficient visual RL. To investigate this issue and further explore the potential of DA, this work conducts comprehensive experiments to assess the impact of DA's attributes on its efficacy and provides the following insights and improvements: (1) For individual DA operations, we reveal that both ample spatial diversity and slight hardness are indispensable. Building on this finding, we introduce Random PadResize (Rand PR), a new DA operation that offers abundant spatial diversity with minimal hardness. (2) For multi-type DA fusion schemes, the increased DA hardness and unstable data distribution result in the current fusion schemes being unable to achieve higher sample efficiency than their corresponding individual operations. Taking the non-stationary nature of RL into account, we propose a RL-tailored multi-type DA fusion scheme called Cycling Augmentation (CycAug), which performs periodic cycles of different DA operations to increase type diversity while maintaining data distribution consistency. Extensive evaluations on the DeepMind Control suite and CARLA driving simulator demonstrate that our methods achieve superior sample efficiency compared with the prior state-of-the-art methods.

READ FULL TEXT

page 5

page 6

page 7

page 15

page 27

research
10/10/2022

A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning

Visual reinforcement learning (RL), which makes decisions directly from ...
research
03/08/2020

DADA: Differentiable Automatic Data Augmentation

Data augmentation (DA) techniques aim to increase data variability, and ...
research
02/21/2022

Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learning

One of the key challenges in visual Reinforcement Learning (RL) is to le...
research
04/24/2022

EPiDA: An Easy Plug-in Data Augmentation Framework for High Performance Text Classification

Recent works have empirically shown the effectiveness of data augmentati...
research
06/01/2020

Reinforcement learning and Bayesian data assimilation for model-informed precision dosing in oncology

Model-informed precision dosing (MIPD) using therapeutic drug/biomarker ...
research
09/08/2021

Rethinking Data Augmentation for Low-Resource Neural Machine Translation: A Multi-Task Learning Approach

In the context of neural machine translation, data augmentation (DA) tec...
research
09/16/2021

Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires

Quality-Diversity (QD) algorithms are powerful exploration algorithms th...

Please sign up or login with your details

Forgot password? Click here to reset