Unsupervised Foggy Scene Understanding via Self Spatial-Temporal Label Diffusion

06/10/2022
by   Liang Liao, et al.
0

Understanding foggy image sequence in the driving scenes is critical for autonomous driving, but it remains a challenging task due to the difficulty in collecting and annotating real-world images of adverse weather. Recently, the self-training strategy has been considered a powerful solution for unsupervised domain adaptation, which iteratively adapts the model from the source domain to the target domain by generating target pseudo labels and re-training the model. However, the selection of confident pseudo labels inevitably suffers from the conflict between sparsity and accuracy, both of which will lead to suboptimal models. To tackle this problem, we exploit the characteristics of the foggy image sequence of driving scenes to densify the confident pseudo labels. Specifically, based on the two discoveries of local spatial similarity and adjacent temporal correspondence of the sequential image data, we propose a novel Target-Domain driven pseudo label Diffusion (TDo-Dif) scheme. It employs superpixels and optical flows to identify the spatial similarity and temporal correspondence, respectively and then diffuses the confident but sparse pseudo labels within a superpixel or a temporal corresponding pair linked by the flow. Moreover, to ensure the feature similarity of the diffused pixels, we introduce local spatial similarity loss and temporal contrastive loss in the model re-training stage. Experimental results show that our TDo-Dif scheme helps the adaptive model achieve 51.92 two publicly available natural foggy datasets (Foggy Zurich and Foggy Driving), which exceeds the state-of-the-art unsupervised domain adaptive semantic segmentation methods. Models and data can be found at https://github.com/velor2012/TDo-Dif.

READ FULL TEXT

page 1

page 5

page 6

page 9

page 10

page 12

page 13

page 14

research
11/26/2022

1st Place Solution to NeurIPS 2022 Challenge on Visual Domain Adaptation

The Visual Domain Adaptation(VisDA) 2022 Challenge calls for an unsuperv...
research
07/25/2022

Improving Pseudo Labels With Intra-Class Similarity for Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) transfers knowledge from a label-ri...
research
03/17/2023

Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding

Predicting attention regions of interest is an important yet challenging...
research
01/26/2021

Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation

Self-training is a competitive approach in domain adaptive segmentation,...
research
12/09/2020

Two-phase Pseudo Label Densification for Self-training based Domain Adaptation

Recently, deep self-training approaches emerged as a powerful solution t...
research
04/07/2019

Unsupervised Domain Adaptation for Multispectral Pedestrian Detection

Multimodal information (e.g., visible and thermal) can generate robust p...
research
12/23/2022

Bridging the Domain Gap in Satellite Pose Estimation: a Self-Training Approach based on Geometrical Constraints

Recently, unsupervised domain adaptation in satellite pose estimation ha...

Please sign up or login with your details

Forgot password? Click here to reset