Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation

by   Donald Shenaj, et al.

Deep neural networks are typically trained in a single shot for a specific task and data distribution, but in real world settings both the task and the domain of application can change. The problem becomes even more challenging in dense predictive tasks, such as semantic segmentation, and furthermore most approaches tackle the two problems separately. In this paper we introduce the novel task of coarse-to-fine learning of semantic segmentation architectures in presence of domain shift. We consider subsequent learning stages progressively refining the task at the semantic level; i.e., the finer set of semantic labels at each learning step is hierarchically derived from the coarser set of the previous step. We propose a new approach (CCDA) to tackle this scenario. First, we employ the maximum squares loss to align source and target domains and, at the same time, to balance the gradients between well-classified and harder samples. Second, we introduce a novel coarse-to-fine knowledge distillation constraint to transfer network capabilities acquired on a coarser set of labels to a set of finer labels. Finally, we design a coarse-to-fine weight initialization rule to spread the importance from each coarse class to the respective finer classes. To evaluate our approach, we design two benchmarks where source knowledge is extracted from the GTA5 dataset and it is transferred to either the Cityscapes or the IDD datasets, and we show how it outperforms the main competitors.


page 7

page 8

page 9

page 17

page 18


Class-Conditional Domain Adaptation on Semantic Segmentation

Semantic segmentation is an important sub-task for many applications, bu...

Multi-Target Adversarial Frameworks for Domain Adaptation in Semantic Segmentation

In this work, we address the task of unsupervised domain adaptation (UDA...

Label Refinement Network for Coarse-to-Fine Semantic Segmentation

We consider the problem of semantic image segmentation using deep convol...

Unsupervised Domain Adaptation for Video Semantic Segmentation

Unsupervised Domain Adaptation for semantic segmentation has gained imme...

Learning with Style: Continual Semantic Segmentation Across Tasks and Domains

Deep learning models dealing with image understanding in real-world sett...

Domain Adaptation for Semantic Segmentation with Maximum Squares Loss

Deep neural networks for semantic segmentation always require a large nu...

Continual Unsupervised Domain Adaptation for Semantic Segmentation using a Class-Specific Transfer

In recent years, there has been tremendous progress in the field of sema...

Please sign up or login with your details

Forgot password? Click here to reset