DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation

11/29/2021
by   Lukas Hoyer, et al.
20

As acquiring pixel-wise annotations of real-world images for semantic segmentation is a costly process, a model can instead be trained with more accessible synthetic data and adapted to real images without requiring their annotations. This process is studied in unsupervised domain adaptation (UDA). Even though a large number of methods propose new adaptation strategies, they are mostly based on outdated network architectures. As the influence of recent network architectures has not been systematically studied, we first benchmark different network architectures for UDA and then propose a novel UDA method, DAFormer, based on the benchmark results. The DAFormer network consists of a Transformer encoder and a multi-level context-aware feature fusion decoder. It is enabled by three simple but crucial training strategies to stabilize the training and to avoid overfitting DAFormer to the source domain: While the Rare Class Sampling on the source domain improves the quality of pseudo-labels by mitigating the confirmation bias of self-training towards common classes, the Thing-Class ImageNet Feature Distance and a learning rate warmup promote feature transfer from ImageNet pretraining. DAFormer significantly improves the state-of-the-art performance by 10.8 mIoU for GTA->Cityscapes and 5.4 mIoU for Synthia->Cityscapes and enables learning even difficult classes such as train, bus, and truck well. The implementation is available at https://github.com/lhoyer/DAFormer.

READ FULL TEXT

page 14

page 15

page 16

page 17

research
04/26/2023

Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentation

Unsupervised domain adaptation (UDA) and domain generalization (DG) enab...
research
07/16/2023

Dual-level Interaction for Domain Adaptive Semantic Segmentation

To circumvent the costly pixel-wise annotations of real-world images in ...
research
04/27/2022

HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation

Unsupervised domain adaptation (UDA) aims to adapt a model trained on th...
research
08/23/2022

Consistency Regularization for Domain Adaptation

Collection of real world annotations for training semantic segmentation ...
research
09/28/2022

Exploiting Instance-based Mixed Sampling via Auxiliary Source Domain Supervision for Domain-adaptive Action Detection

We propose a novel domain adaptive action detection approach and a new a...
research
05/27/2023

Condition-Invariant Semantic Segmentation

Adaptation of semantic segmentation networks to different visual conditi...
research
03/02/2022

Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation

Panoramic images with their 360-degree directional view encompass exhaus...

Please sign up or login with your details

Forgot password? Click here to reset