Curriculum Model Adaptation with Synthetic and Real Data for Semantic Foggy Scene Understanding

01/05/2019
by   Dengxin Dai, et al.
2

This work addresses the problem of semantic scene understanding under fog. Although marked progress has been made in semantic scene understanding, it is mainly concentrated on clear-weather scenes. Extending semantic segmentation methods to adverse weather conditions such as fog is crucial for outdoor applications. In this paper, we propose a novel method, named Curriculum Model Adaptation (CMAda), which gradually adapts a semantic segmentation model from light synthetic fog to dense real fog in multiple steps, using both labeled synthetic foggy data and unlabeled real foggy data. The method is based on the fact that the results of semantic segmentation in moderately adverse conditions (light fog) can be bootstrapped to solve the same problem in highly adverse conditions (dense fog). CMAda is extensible to other adverse conditions and provides a new paradigm for learning with synthetic data and unlabeled real data. In addition, we present three other main stand-alone contributions: 1) a novel method to add synthetic fog to real, clear-weather scenes using semantic input; 2) a new fog density estimator; 3) a novel fog densification method to densify the fog in real foggy scenes without using depth; and 4) the Foggy Zurich dataset comprising 3808 real foggy images, with pixel-level semantic annotations for 40 images under dense fog. Our experiments show that 1) our fog simulation and fog density estimator outperform their state-of-the-art counterparts with respect to the task of semantic foggy scene understanding (SFSU); 2) CMAda improves the performance of state-of-the-art models for SFSU significantly, benefiting both from our synthetic and real foggy data. The datasets and code are available at the project website.

READ FULL TEXT

page 2

page 5

page 6

page 10

page 11

page 14

page 16

page 18

research
08/03/2018

Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding

This work addresses the problem of semantic scene understanding under de...
research
10/09/2019

Semantic Understanding of Foggy Scenes with Purely Synthetic Data

This work addresses the problem of semantic scene understanding under fo...
research
08/25/2017

Semantic Foggy Scene Understanding with Synthetic Data

This work addresses the problem of semantic foggy scene understanding (S...
research
01/07/2022

Leveraging Scale-Invariance and Uncertainity with Self-Supervised Domain Adaptation for Semantic Segmentation of Foggy Scenes

This paper presents FogAdapt, a novel approach for domain adaptation of ...
research
12/01/2021

Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic Foggy Scene Understanding

Although considerable progress has been made in semantic scene understan...
research
02/11/2021

ZeroScatter: Domain Transfer for Long Distance Imaging and Vision through Scattering Media

Adverse weather conditions, including snow, rain, and fog pose a challen...
research
04/04/2022

FIFO: Learning Fog-invariant Features for Foggy Scene Segmentation

Robust visual recognition under adverse weather conditions is of great i...

Please sign up or login with your details

Forgot password? Click here to reset