FANet: Quality-Aware Feature Aggregation Network for RGB-T Tracking

11/24/2018
by   Yabin Zhu, et al.
0

This paper investigates how to perform robust visual tracking in adverse and challenging conditions using complementary visual and thermal infrared data (RGB-T tracking). We propose a novel deep network architecture "quality-aware Feature Aggregation Network (FANet)" to achieve quality-aware aggregations of both hierarchical features and multimodal information for robust online RGB-T tracking. Unlike existing works that directly concatenate hierarchical deep features, our FANet learns the layer weights to adaptively aggregate them to handle the challenge of significant appearance changes caused by deformation, abrupt motion, background clutter and occlusion within each modality. Moreover, we employ the operations of max pooling, interpolation upsampling and convolution to transform these hierarchical and multi-resolution features into a uniform space at the same resolution for more effective feature aggregation. In different modalities, we elaborately design a multimodal aggregation sub-network to integrate all modalities collaboratively based on the predicted reliability degrees. Extensive experiments on large-scale benchmark datasets demonstrate that our FANet significantly outperforms other state-of-the-art RGB-T tracking methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 9

research
07/17/2019

Multi-Adapter RGBT Tracking

The task of RGBT tracking aims to take the complementary advantages from...
research
07/24/2019

Dense Feature Aggregation and Pruning for RGBT Tracking

How to perform effective information fusion of different modalities is a...
research
11/14/2020

RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss

RGBT tracking has attracted increasing attention since RGB and thermal i...
research
11/14/2020

Duality-Gated Mutual Condition Network for RGBT Tracking

Low-quality modalities contain not only a lot of noisy information but a...
research
07/26/2020

Challenge-Aware RGBT Tracking

RGB and thermal source data suffer from both shared and specific challen...
research
03/12/2021

Siamese Infrared and Visible Light Fusion Network for RGB-T Tracking

Due to the different photosensitive properties of infrared and visible l...
research
03/17/2020

M^5L: Multi-Modal Multi-Margin Metric Learning for RGBT Tracking

Classifying the confusing samples in the course of RGBT tracking is a qu...

Please sign up or login with your details

Forgot password? Click here to reset