Local Feature Swapping for Generalization in Reinforcement Learning

04/13/2022
by   David Bertoin, et al.
1

Over the past few years, the acceleration of computing resources and research in deep learning has led to significant practical successes in a range of tasks, including in particular in computer vision. Building on these advances, reinforcement learning has also seen a leap forward with the emergence of agents capable of making decisions directly from visual observations. Despite these successes, the over-parametrization of neural architectures leads to memorization of the data used during training and thus to a lack of generalization. Reinforcement learning agents based on visual inputs also suffer from this phenomenon by erroneously correlating rewards with unrelated visual features such as background elements. To alleviate this problem, we introduce a new regularization technique consisting of channel-consistent local permutations (CLOP) of the feature maps. The proposed permutations induce robustness to spatial correlations and help prevent overfitting behaviors in RL. We demonstrate, on the OpenAI Procgen Benchmark, that RL agents trained with the CLOP method exhibit robustness to visual changes and better generalization properties than agents trained using other state-of-the-art regularization techniques. We also demonstrate the effectiveness of CLOP as a general regularization technique in supervised learning.

READ FULL TEXT

page 5

page 6

page 8

page 9

page 17

page 20

page 21

research
04/18/2018

A Study on Overfitting in Deep Reinforcement Learning

Recent years have witnessed significant progresses in deep Reinforcement...
research
12/06/2018

Quantifying Generalization in Reinforcement Learning

In this paper, we investigate the problem of overfitting in deep reinfor...
research
10/28/2019

Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck

The ability for policies to generalize to new environments is key to the...
research
04/20/2022

Understanding and Preventing Capacity Loss in Reinforcement Learning

The reinforcement learning (RL) problem is rife with sources of non-stat...
research
10/10/2011

Closed-Loop Learning of Visual Control Policies

In this paper we present a general, flexible framework for learning mapp...
research
09/16/2022

Look where you look! Saliency-guided Q-networks for visual RL tasks

Deep reinforcement learning policies, despite their outstanding efficien...
research
06/01/2023

Normalization Enhances Generalization in Visual Reinforcement Learning

Recent advances in visual reinforcement learning (RL) have led to impres...

Please sign up or login with your details

Forgot password? Click here to reset