Semantic Nighttime Image Segmentation with Synthetic Stylized Data, Gradual Adaptation and Uncertainty-Aware Evaluation

01/17/2019
by   Christos Sakaridis, et al.
6

This work addresses the problem of semantic segmentation of nighttime images. The main direction of recent progress in semantic segmentation pertains to daytime scenes with favorable illumination conditions. We focus on improving the performance of state-of-the-art methods on the nighttime domain by adapting them to nighttime data without extra annotations, and designing a new evaluation framework to address the uncertainty of semantics in nighttime images. To this end, we make the following contributions: 1) a novel pipeline for dataset-scale guided style transfer to generate synthetic nighttime images from real daytime input; 2) a framework to gradually adapt semantic segmentation models from day to night via stylized and real images of progressively increasing darkness; 3) a novel uncertainty-aware annotation and evaluation framework and metric for semantic segmentation in adverse conditions; 4) the Dark Zurich dataset with 2416 nighttime and 2920 twilight unlabeled images plus 20 nighttime images with pixel-level annotations that conform to our newly-proposed evaluation. Our experiments evidence that both our stylized data per se and our gradual adaptation significantly boost performance at nighttime both for standard evaluation metrics and our metric. Moreover, our new evaluation reveals that state-of-the-art segmentation models output overly confident predictions at indiscernible regions compared to visible ones.

READ FULL TEXT

page 3

page 5

page 7

research
05/28/2020

Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation

We address the problem of semantic nighttime image segmentation and impr...
research
03/15/2020

Night-time Semantic Segmentation with a Large Real Dataset

Although huge progress has been made on semantic segmentation in recent ...
research
07/21/2021

Weighted Intersection over Union (wIoU): A New Evaluation Metric for Image Segmentation

In this paper, we propose a novel evaluation metric for performance eval...
research
10/05/2018

Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttime

This work addresses the problem of semantic image segmentation of nightt...
research
08/16/2019

See Clearer at Night: Towards Robust Nighttime Semantic Segmentation through Day-Night Image Conversion

Currently, semantic segmentation shows remarkable efficiency and reliabi...
research
04/12/2022

NightLab: A Dual-level Architecture with Hardness Detection for Segmentation at Night

The semantic segmentation of nighttime scenes is a challenging problem t...
research
07/18/2023

Disentangle then Parse:Night-time Semantic Segmentation with Illumination Disentanglement

Most prior semantic segmentation methods have been developed for day-tim...

Please sign up or login with your details

Forgot password? Click here to reset