-
Synthetic data generation for end-to-end thermal infrared tracking
The usage of both off-the-shelf and end-to-end trained deep networks hav...
read it
-
Multi-Adapter RGBT Tracking
The task of RGBT tracking aims to take the complementary advantages from...
read it
-
Dense Feature Aggregation and Pruning for RGBT Tracking
How to perform effective information fusion of different modalities is a...
read it
-
Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking
In this study, we propose a novel RGB-T tracking framework by jointly mo...
read it
-
Multi-Modality Cascaded Fusion Technology for Autonomous Driving
Multi-modality fusion is the guarantee of the stability of autonomous dr...
read it
-
Which Ads to Show? Advertisement Image Assessment with Auxiliary Information via Multi-step Modality Fusion
Assessing aesthetic preference is a fundamental task related to human co...
read it
-
M^5L: Multi-Modal Multi-Margin Metric Learning for RGBT Tracking
Classifying the confusing samples in the course of RGBT tracking is a qu...
read it
Multi-Modal Fusion for End-to-End RGB-T Tracking
We propose an end-to-end tracking framework for fusing the RGB and TIR modalities in RGB-T tracking. Our baseline tracker is DiMP (Discriminative Model Prediction), which employs a carefully designed target prediction network trained end-to-end using a discriminative loss. We analyze the effectiveness of modality fusion in each of the main components in DiMP, i.e. feature extractor, target estimation network, and classifier. We consider several fusion mechanisms acting at different levels of the framework, including pixel-level, feature-level and response-level. Our tracker is trained in an end-to-end manner, enabling the components to learn how to fuse the information from both modalities. As data to train our model, we generate a large-scale RGB-T dataset by considering an annotated RGB tracking dataset (GOT-10k) and synthesizing paired TIR images using an image-to-image translation approach. We perform extensive experiments on VOT-RGBT2019 dataset and RGBT210 dataset, evaluating each type of modality fusing on each model component. The results show that the proposed fusion mechanisms improve the performance of the single modality counterparts. We obtain our best results when fusing at the feature-level on both the IoU-Net and the model predictor, obtaining an EAO score of 0.391 on VOT-RGBT2019 dataset. With this fusion mechanism we achieve the state-of-the-art performance on RGBT210 dataset.
READ FULL TEXT
Comments
There are no comments yet.