M^5L: Multi-Modal Multi-Margin Metric Learning for RGBT Tracking

by   Zhengzheng Tu, et al.

Classifying the confusing samples in the course of RGBT tracking is a quite challenging problem, which hasn't got satisfied solution. Existing methods only focus on enlarging the boundary between positive and negative samples, however, the structured information of samples might be harmed, e.g., confusing positive samples are closer to the anchor than normal positive samples.To handle this problem, we propose a novel Multi-Modal Multi-Margin Metric Learning framework, named M^5L for RGBT tracking in this paper. In particular, we design a multi-margin structured loss to distinguish the confusing samples which play a most critical role in tracking performance boosting. To alleviate this problem, we additionally enlarge the boundaries between confusing positive samples and normal ones, between confusing negative samples and normal ones with predefined margins, by exploiting the structured information of all samples in each modality.Moreover, a cross-modality constraint is employed to reduce the difference between modalities and push positive samples closer to the anchor than negative ones from two modalities.In addition, to achieve quality-aware RGB and thermal feature fusion, we introduce the modality attentions and learn them using a feature fusion module in our network. Extensive experiments on large-scale datasets testify that our framework clearly improves the tracking performance and outperforms the state-of-the-art RGBT trackers.


page 2

page 3

page 4

page 5

page 6

page 7

page 8

page 10


Multi-Adapter RGBT Tracking

The task of RGBT tracking aims to take the complementary advantages from...

Generative-based Fusion Mechanism for Multi-Modal Tracking

Generative models (GMs) have received increasing research interest for t...

Modality-Aware Negative Sampling for Multi-modal Knowledge Graph Embedding

Negative sampling (NS) is widely used in knowledge graph embedding (KGE)...

RGB-T Tracking via Multi-Modal Mutual Prompt Learning

Object tracking based on the fusion of visible and thermal im-ages, know...

Ranked List Loss for Deep Metric Learning

The objective of deep metric learning (DML) is to learn embeddings that ...

Multi-modal Visual Tracking: Review and Experimental Comparison

Visual object tracking, as a fundamental task in computer vision, has dr...

FANet: Quality-Aware Feature Aggregation Network for RGB-T Tracking

This paper investigates how to perform robust visual tracking in adverse...

Please sign up or login with your details

Forgot password? Click here to reset