Exploring Fusion Strategies for Accurate RGBT Visual Object Tracking

01/21/2022
by   Zhangyong Tang, et al.
0

We address the problem of multi-modal object tracking in video and explore various options of fusing the complementary information conveyed by the visible (RGB) and thermal infrared (TIR) modalities including pixel-level, feature-level and decision-level fusion. Specifically, different from the existing methods, paradigm of image fusion task is heeded for fusion at pixel level. Feature-level fusion is fulfilled by attention mechanism with channels excited optionally. Besides, at decision level, a novel fusion strategy is put forward since an effortless averaging configuration has shown the superiority. The effectiveness of the proposed decision-level fusion strategy owes to a number of innovative contributions, including a dynamic weighting of the RGB and TIR contributions and a linear template update operation. A variant of which produced the winning tracker at the Visual Object Tracking Challenge 2020 (VOT-RGBT2020). The concurrent exploration of innovative pixel- and feature-level fusion strategies highlights the advantages of the proposed decision-level fusion method. Extensive experimental results on three challenging datasets, i.e., GTOT, VOT-RGBT2019, and VOT-RGBT2020, demonstrate the effectiveness and robustness of the proposed method, compared to the state-of-the-art approaches. Code will be shared at https://github.com/Zhangyong-Tang/DFAT.

READ FULL TEXT

page 1

page 10

research
01/22/2022

Temporal Aggregation for Adaptive RGBT Tracking

Visual object tracking with RGB and thermal infrared (TIR) spectra avail...
research
08/12/2019

Learning Target-oriented Dual Attention for Robust RGB-T Tracking

RGB-Thermal object tracking attempt to locate target object using comple...
research
08/30/2019

Multi-Modal Fusion for End-to-End RGB-T Tracking

We propose an end-to-end tracking framework for fusing the RGB and TIR m...
research
08/31/2023

RGB-T Tracking via Multi-Modal Mutual Prompt Learning

Object tracking based on the fusion of visible and thermal im-ages, know...
research
09/16/2021

Dynamic Fusion Network for RGBT Tracking

For both visible and infrared images have their own advantages and disad...
research
07/07/2021

E-PixelHop: An Enhanced PixelHop Method for Object Classification

Based on PixelHop and PixelHop++, which are recently developed using the...
research
09/09/2023

Generation and Recombination for Multifocus Image Fusion with Free Number of Inputs

Multifocus image fusion is an effective way to overcome the limitation o...

Please sign up or login with your details

Forgot password? Click here to reset