Duality-Gated Mutual Condition Network for RGBT Tracking

by   Andong Lu, et al.

Low-quality modalities contain not only a lot of noisy information but also some discriminative features in RGBT tracking. However, the potentials of low-quality modalities are not well explored in existing RGBT tracking algorithms. In this work, we propose a novel duality-gated mutual condition network to fully exploit the discriminative information of all modalities while suppressing the effects of data noise. In specific, we design a mutual condition module, which takes the discriminative information of a modality as the condition to guide feature learning of target appearance in another modality. Such module can effectively enhance target representations and suppress useless features of all modalities even in the presence of low-quality modalities. To improve the quality of conditions and further reduce data noise, we propose a duality-gated mechanism in the mutual condition module. To deal with the tracking failure caused by sudden camera motion, which often occurs in RGBT tracking, we design a resampling strategy based on optical flow algorithms. It does not increase much computational cost since we perform optical flow calculation only when the model prediction is unreliable and then execute resampling when the sudden camera motion is detected. Extensive experiments on three RGBT tracking benchmark datasets show that our method performs favorably against the state-of-the-art tracking algorithms.


page 5

page 6

page 8

page 9

page 10

page 11

page 12

page 13


Challenge-Aware RGBT Tracking

RGB and thermal source data suffer from both shared and specific challen...

Multi-Adapter RGBT Tracking

The task of RGBT tracking aims to take the complementary advantages from...

FANet: Quality-Aware Feature Aggregation Network for RGB-T Tracking

This paper investigates how to perform robust visual tracking in adverse...

RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss

RGBT tracking has attracted increasing attention since RGB and thermal i...

Mutual Modality Learning for Video Action Classification

The construction of models for video action classification progresses ra...

Dense Feature Aggregation and Pruning for RGBT Tracking

How to perform effective information fusion of different modalities is a...

Please sign up or login with your details

Forgot password? Click here to reset