Glimpse-Attend-and-Explore: Self-Attention for Active Visual Exploration

08/26/2021
by   Soroush Seifi, et al.
8

Active visual exploration aims to assist an agent with a limited field of view to understand its environment based on partial observations made by choosing the best viewing directions in the scene. Recent methods have tried to address this problem either by using reinforcement learning, which is difficult to train, or by uncertainty maps, which are task-specific and can only be implemented for dense prediction tasks. In this paper, we propose the Glimpse-Attend-and-Explore model which: (a) employs self-attention to guide the visual exploration instead of task-specific uncertainty maps; (b) can be used for both dense and sparse prediction tasks; and (c) uses a contrastive stream to further improve the representations learned. Unlike previous works, we show the application of our model on multiple tasks like reconstruction, segmentation and classification. Our model provides encouraging results while being less dependent on dataset bias in driving the exploration. We further perform an ablation study to investigate the features and attention learned by our model. Finally, we show that our self-attention module learns to attend different regions of the scene by minimizing the loss on the downstream task. Code: https://github.com/soroushseifi/glimpse-attend-explore.

READ FULL TEXT

page 2

page 6

page 8

research
05/12/2020

Planning to Explore via Self-Supervised World Models

Reinforcement learning allows solving complex tasks, however, the learni...
research
04/04/2022

FoV-Net: Field-of-View Extrapolation Using Self-Attention and Uncertainty

The ability to make educated predictions about their surroundings, and a...
research
09/23/2019

Where to Look Next: Unsupervised Active Visual Exploration on 360° Input

We address the problem of active visual exploration of large 360 inputs....
research
07/10/2022

Self-attention on Multi-Shifted Windows for Scene Segmentation

Scene segmentation in images is a fundamental yet challenging problem in...
research
03/05/2019

Learning Exploration Policies for Navigation

Numerous past works have tackled the problem of task-driven navigation. ...
research
04/23/2022

Visual Attention Emerges from Recurrent Sparse Reconstruction

Visual attention helps achieve robust perception under noise, corruption...
research
08/19/2021

Causal Attention for Unbiased Visual Recognition

Attention module does not always help deep models learn causal features ...

Please sign up or login with your details

Forgot password? Click here to reset