Multi-Adapter RGBT Tracking

by   Chenglong Li, et al.

The task of RGBT tracking aims to take the complementary advantages from visible spectrum and thermal infrared data to achieve robust visual tracking, and receives more and more attention in recent years. Existing works focus on modality-specific information integration by introducing modality weights to achieve adaptive fusion or learning robust feature representations of different modalities. Although these methods could effectively deploy the modality-specific properties, they ignore the potential values of modality-shared cues as well as instance-aware information, which are crucial for effective fusion of different modalities in RGBT tracking. In this paper, we propose a novel Multi-Adapter convolutional Network (MANet) to jointly perform modality-shared, modality-specific and instance-aware feature learning in an end-to-end trained deep framework for RGBT tracking. We design three kinds of adapters within our network. In a specific, the generality adapter is to extract shared object representations, the modality adapter aims at encoding modality-specific information to deploy their complementary advantages, and the instance adapter is to model the appearance properties and temporal variations of a certain object. Moreover, to reduce computational complexity for real-time demand of visual tracking, we design a parallel structure of generic adapter and modality adapter. Extensive experiments on two RGBT tracking benchmark datasets demonstrate the outstanding performance of the proposed tracker against other state-of-the-art RGB and RGBT tracking algorithms.


page 4

page 8


RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss

RGBT tracking has attracted increasing attention since RGB and thermal i...

RGB-T Tracking Based on Mixed Attention

RGB-T tracking involves the use of images from both visible and thermal ...

FANet: Quality-Aware Feature Aggregation Network for RGB-T Tracking

This paper investigates how to perform robust visual tracking in adverse...

Dense Feature Aggregation and Pruning for RGBT Tracking

How to perform effective information fusion of different modalities is a...

M^5L: Multi-Modal Multi-Margin Metric Learning for RGBT Tracking

Classifying the confusing samples in the course of RGBT tracking is a qu...

Duality-Gated Mutual Condition Network for RGBT Tracking

Low-quality modalities contain not only a lot of noisy information but a...

Challenge-Aware RGBT Tracking

RGB and thermal source data suffer from both shared and specific challen...

Please sign up or login with your details

Forgot password? Click here to reset