FoV-Net: Field-of-View Extrapolation Using Self-Attention and Uncertainty

04/04/2022
by   Liqian Ma, et al.
11

The ability to make educated predictions about their surroundings, and associate them with certain confidence, is important for intelligent systems, like autonomous vehicles and robots. It allows them to plan early and decide accordingly. Motivated by this observation, in this paper we utilize information from a video sequence with a narrow field-of-view to infer the scene at a wider field-of-view. To this end, we propose a temporally consistent field-of-view extrapolation framework, namely FoV-Net, that: (1) leverages 3D information to propagate the observed scene parts from past frames; (2) aggregates the propagated multi-frame information using an attention-based feature aggregation module and a gated self-attention module, simultaneously hallucinating any unobserved scene parts; and (3) assigns an interpretable uncertainty value at each pixel. Extensive experiments show that FoV-Net does not only extrapolate the temporally consistent wide field-of-view scene better than existing alternatives, but also provides the associated uncertainty which may benefit critical decision-making downstream applications. Project page is at http://charliememory.github.io/RAL21_FoV.

READ FULL TEXT

page 1

page 2

page 3

page 6

page 7

research
08/26/2021

Glimpse-Attend-and-Explore: Self-Attention for Active Visual Exploration

Active visual exploration aims to assist an agent with a limited field o...
research
03/26/2022

Exploring Self-Attention for Visual Intersection Classification

In robot vision, self-attention has recently emerged as a technique for ...
research
10/07/2021

SVG-Net: An SVG-based Trajectory Prediction Model

Anticipating motions of vehicles in a scene is an essential problem for ...
research
02/09/2023

Drawing Attention to Detail: Pose Alignment through Self-Attention for Fine-Grained Object Classification

Intra-class variations in the open world lead to various challenges in c...
research
01/11/2022

TSA-Net: Tube Self-Attention Network for Action Quality Assessment

In recent years, assessing action quality from videos has attracted grow...
research
12/05/2020

Understanding Bird's-Eye View Semantic HD-Maps Using an Onboard Monocular Camera

Autonomous navigation requires scene understanding of the action-space t...

Please sign up or login with your details

Forgot password? Click here to reset