Modality-Adaptive Mixup and Invariant Decomposition for RGB-Infrared Person Re-Identification

03/03/2022
by   Zhipeng Huang, et al.
0

RGB-infrared person re-identification is an emerging cross-modality re-identification task, which is very challenging due to significant modality discrepancy between RGB and infrared images. In this work, we propose a novel modality-adaptive mixup and invariant decomposition (MID) approach for RGB-infrared person re-identification towards learning modality-invariant and discriminative representations. MID designs a modality-adaptive mixup scheme to generate suitable mixed modality images between RGB and infrared images for mitigating the inherent modality discrepancy at the pixel-level. It formulates modality mixup procedure as Markov decision process, where an actor-critic agent learns dynamical and local linear interpolation policy between different regions of cross-modality images under a deep reinforcement learning framework. Such policy guarantees modality-invariance in a more continuous latent space and avoids manifold intrusion by the corrupted mixed modality samples. Moreover, to further counter modality discrepancy and enforce invariant visual semantics at the feature-level, MID employs modality-adaptive convolution decomposition to disassemble a regular convolution layer into modality-specific basis layers and a modality-shared coefficient layer. Extensive experimental results on two challenging benchmarks demonstrate superior performance of MID over state-of-the-art methods.

READ FULL TEXT

page 3

page 7

research
04/23/2021

Exploring Modality-shared Appearance Features and Modality-invariant Relation Features for Cross-modality Person Re-Identification

Most existing cross-modality person re-identification works rely on disc...
research
02/10/2020

Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification

RGB-Infrared (IR) person re-identification is very challenging due to th...
research
11/01/2021

Benchmarks for Corruption Invariant Person Re-identification

When deploying person re-identification (ReID) model in safety-critical ...
research
07/03/2020

A Similarity Inference Metric for RGB-Infrared Cross-Modality Person Re-identification

RGB-Infrared (IR) cross-modality person re-identification (re-ID), which...
research
02/29/2020

Cross-Spectrum Dual-Subspace Pairing for RGB-infrared Cross-Modality Person Re-Identification

Due to its potential wide applications in video surveillance and other c...
research
02/02/2023

Exploring Invariant Representation for Visible-Infrared Person Re-Identification

Cross-spectral person re-identification, which aims to associate identit...
research
10/25/2019

Attend to the Difference: Cross-Modality Person Re-identification via Contrastive Correlation

The problem of cross-modality person re-identification has been receivin...

Please sign up or login with your details

Forgot password? Click here to reset