Unsupervised Visual Attention and Invariance for Reinforcement Learning

04/07/2021
by   Xudong Wang, et al.
0

Vision-based reinforcement learning (RL) is successful, but how to generalize it to unknown test environments remains challenging. Existing methods focus on training an RL policy that is universal to changing visual domains, whereas we focus on extracting visual foreground that is universal, feeding clean invariant vision to the RL policy learner. Our method is completely unsupervised, without manual annotations or access to environment internals. Given videos of actions in a training environment, we learn how to extract foregrounds with unsupervised keypoint detection, followed by unsupervised visual attention to automatically generate a foreground mask per video frame. We can then introduce artificial distractors and train a model to reconstruct the clean foreground mask from noisy observations. Only this learned model is needed during test to provide distraction-free visual input to the RL policy learner. Our Visual Attention and Invariance (VAI) method significantly outperforms the state-of-the-art on visual domain generalization, gaining 15 to 49 229 Manipulation) benchmarks. Our results demonstrate that it is not only possible to learn domain-invariant vision without any supervision, but freeing RL from visual distractions also makes the policy more focused and thus far better.

READ FULL TEXT

page 4

page 8

page 12

page 13

research
06/01/2020

Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learning

A fundamental challenge in reinforcement learning is to learn policies t...
research
12/09/2019

Unsupervised Curricula for Visual Meta-Reinforcement Learning

In principle, meta-reinforcement learning algorithms leverage experience...
research
02/14/2021

Domain Adversarial Reinforcement Learning

We consider the problem of generalization in reinforcement learning wher...
research
01/07/2021

The Distracting Control Suite – A Challenging Benchmark for Reinforcement Learning from Pixels

Robots have to face challenging perceptual settings, including changes i...
research
02/10/2021

Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation

Despite the recent success of deep reinforcement learning (RL), domain a...
research
09/05/2023

Efficient RL via Disentangled Environment and Agent Representations

Agents that are aware of the separation between themselves and their env...
research
06/10/2018

Deep Curiosity Loops in Social Environments

Inspired by infants' intrinsic motivation to learn, which values informa...

Please sign up or login with your details

Forgot password? Click here to reset