Unsupervised RGB-to-Thermal Domain Adaptation via Multi-Domain Attention Network

10/09/2022
by   Lu Gan, et al.
0

This work presents a new method for unsupervised thermal image classification and semantic segmentation by transferring knowledge from the RGB domain using a multi-domain attention network. Our method does not require any thermal annotations or co-registered RGB-thermal pairs, enabling robots to perform visual tasks at night and in adverse weather conditions without incurring additional costs of data labeling and registration. Current unsupervised domain adaptation methods look to align global images or features across domains. However, when the domain shift is significantly larger for cross-modal data, not all features can be transferred. We solve this problem by using a shared backbone network that promotes generalization, and domain-specific attention that reduces negative transfer by attending to domain-invariant and easily-transferable features. Our approach outperforms the state-of-the-art RGB-to-thermal adaptation method in classification benchmarks, and is successfully applied to thermal river scene segmentation using only synthetic RGB images. Our code is made publicly available at https://github.com/ganlumomo/thermal-uda-attention.

READ FULL TEXT

page 1

page 4

page 5

page 6

research
04/12/2022

Glass Segmentation with RGB-Thermal Image Pairs

This paper proposes a new glass segmentation method utilizing paired RGB...
research
07/18/2023

Online Self-Supervised Thermal Water Segmentation for Aerial Vehicles

We present a new method to adapt an RGB-trained water segmentation netwo...
research
11/24/2020

Online Domain Adaptation for Continuous Cross-Subject Liver Viability Evaluation Based on Irregular Thermal Data

Accurate evaluation of liver viability during its procurement is a chall...
research
03/10/2020

HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images

The majority of learning-based semantic segmentation methods are optimiz...
research
07/10/2023

Test-Time Adaptation for Nighttime Color-Thermal Semantic Segmentation

The ability to scene understanding in adverse visual conditions, e.g., n...
research
07/26/2019

Semantic Deep Intermodal Feature Transfer: Transferring Feature Descriptors Between Imaging Modalities

Under difficult environmental conditions, the view of RGB cameras may be...
research
09/21/2022

Self-adversarial Multi-scale Contrastive Learning for Semantic Segmentation of Thermal Facial Images

Reliable segmentation of thermal facial images in unconstrained settings...

Please sign up or login with your details

Forgot password? Click here to reset