Boosting Night-time Scene Parsing with Learnable Frequency

08/30/2022
by   Zhifeng Xie, et al.
1

Night-Time Scene Parsing (NTSP) is essential to many vision applications, especially for autonomous driving. Most of the existing methods are proposed for day-time scene parsing. They rely on modeling pixel intensity-based spatial contextual cues under even illumination. Hence, these methods do not perform well in night-time scenes as such spatial contextual cues are buried in the over-/under-exposed regions in night-time scenes. In this paper, we first conduct an image frequency-based statistical experiment to interpret the day-time and night-time scene discrepancies. We find that image frequency distributions differ significantly between day-time and night-time scenes, and understanding such frequency distributions is critical to NTSP problem. Based on this, we propose to exploit the image frequency distributions for night-time scene parsing. First, we propose a Learnable Frequency Encoder (LFE) to model the relationship between different frequency coefficients to measure all frequency components dynamically. Second, we propose a Spatial Frequency Fusion module (SFF) that fuses both spatial and frequency information to guide the extraction of spatial context features. Extensive experiments show that our method performs favorably against the state-of-the-art methods on the NightCity, NightCity+ and BDD100K-night datasets. In addition, we demonstrate that our method can be applied to existing day-time scene parsing methods and boost their performance on night-time scenes.

READ FULL TEXT

page 1

page 4

page 7

page 9

page 10

research
12/04/2016

Pyramid Scene Parsing Network

Scene parsing is challenging for unrestricted open vocabulary and divers...
research
10/17/2017

Scene Parsing with Global Context Embedding

We present a scene parsing method that utilizes global context informati...
research
03/06/2023

Traffic Scene Parsing through the TSP6K Dataset

Traffic scene parsing is one of the most important tasks to achieve inte...
research
08/08/2017

FoveaNet: Perspective-aware Urban Scene Parsing

Parsing urban scene images benefits many applications, especially self-d...
research
06/19/2018

MoE-SPNet: A Mixture-of-Experts Scene Parsing Network

Scene parsing is an indispensable component in understanding the semanti...
research
09/16/2019

Boosting Real-Time Driving Scene Parsing with Shared Semantics

Real-time scene parsing is a fundamental feature for autonomous driving ...
research
02/24/2018

Spatially Constrained Location Prior for Scene Parsing

Semantic context is an important and useful cue for scene parsing in com...

Please sign up or login with your details

Forgot password? Click here to reset