Beyond Monocular Deraining: Parallel Stereo Deraining Network Via Semantic Prior

05/09/2021
by   Kaihao Zhang, et al.
9

Rain is a common natural phenomenon. Taking images in the rain however often results in degraded quality of images, thus compromises the performance of many computer vision systems. Most existing de-rain algorithms use only one single input image and aim to recover a clean image. Few work has exploited stereo images. Moreover, even for single image based monocular deraining, many current methods fail to complete the task satisfactorily because they mostly rely on per pixel loss functions and ignore semantic information. In this paper, we present a Paired Rain Removal Network (PRRNet), which exploits both stereo images and semantic information. Specifically, we develop a Semantic-Aware Deraining Module (SADM) which solves both tasks of semantic segmentation and deraining of scenes, and a Semantic-Fusion Network (SFNet) and a View-Fusion Network (VFNet) which fuse semantic information and multi-view information respectively. In addition, we also introduce an Enhanced Paired Rain Removal Network (EPRRNet) which exploits semantic prior to remove rain streaks from stereo images. We first use a coarse deraining network to reduce the rain streaks on the input images, and then adopt a pre-trained semantic segmentation network to extract semantic features from the coarse derained image. Finally, a parallel stereo deraining network fuses semantic and multi-view information to restore finer results. We also propose new stereo based rainy datasets for benchmarking. Experiments on both monocular and the newly proposed stereo rainy datasets demonstrate that the proposed method achieves the state-of-the-art performance.

READ FULL TEXT

page 5

page 7

page 9

page 10

page 11

page 12

07/15/2022

Learning Parallax Transformer Network for Stereo Image JPEG Artifacts Removal

Under stereo settings, the performance of image JPEG artifacts removal c...
03/21/2021

Deep Dense Multi-scale Network for Snow Removal Using Semantic and Geometric Priors

Images captured in snowy days suffer from noticeable degradation of scen...
10/13/2021

Semantic Image Fusion

Image fusion methods and metrics for their evaluation have conventionall...
04/16/2018

Semantic Single-Image Dehazing

Single-image haze-removal is challenging due to limited information cont...
04/17/2019

Material Segmentation of Multi-View Satellite Imagery

Material recognition methods use image context and local cues for pixel-...
06/22/2021

Deep Stereo Image Compression with Decoder Side Information using Wyner Common Information

We present a novel deep neural network (DNN) architecture for compressin...
07/18/2022

A Semantic-aware Attention and Visual Shielding Network for Cloth-changing Person Re-identification

Cloth-changing person reidentification (ReID) is a newly emerging resear...