Challenge-Aware RGBT Tracking

by   Chenglong Li, et al.

RGB and thermal source data suffer from both shared and specific challenges, and how to explore and exploit them plays a critical role to represent the target appearance in RGBT tracking. In this paper, we propose a novel challenge-aware neural network to handle the modality-shared challenges (e.g., fast motion, scale variation and occlusion) and the modality-specific ones (e.g., illumination variation and thermal crossover) for RGBT tracking. In particular, we design several parameter-shared branches in each layer to model the target appearance under the modality-shared challenges, and several parameterindependent branches under the modality-specific ones. Based on the observation that the modality-specific cues of different modalities usually contains the complementary advantages, we propose a guidance module to transfer discriminative features from one modality to another one, which could enhance the discriminative ability of some weak modality. Moreover, all branches are aggregated together in an adaptive manner and parallel embedded in the backbone network to efficiently form more discriminative target representations. These challenge-aware branches are able to model the target appearance under certain challenges so that the target representations can be learnt by a few parameters even in the situation of insufficient training data. From the experimental results we will show that our method operates at a real-time speed while performing well against the state-of-the-art methods on three benchmark datasets.


page 2

page 5

page 9


RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss

RGBT tracking has attracted increasing attention since RGB and thermal i...

Duality-Gated Mutual Condition Network for RGBT Tracking

Low-quality modalities contain not only a lot of noisy information but a...

Multi-Adapter RGBT Tracking

The task of RGBT tracking aims to take the complementary advantages from...

FANet: Quality-Aware Feature Aggregation Network for RGB-T Tracking

This paper investigates how to perform robust visual tracking in adverse...

Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems

Multispectral pedestrian detection is capable of adapting to insufficien...

Towards Cross-modality Medical Image Segmentation with Online Mutual Knowledge Distillation

The success of deep convolutional neural networks is partially attribute...

BAANet: Learning Bi-directional Adaptive Attention Gates for Multispectral Pedestrian Detection

Thermal infrared (TIR) image has proven effectiveness in providing tempe...