From Rain Removal to Rain Generation

08/08/2020
by   Hong Wang, et al.
36

Single image deraining is an important yet challenging issue due to the complex and diverse rain structures in real scenes. Currently, the state-of-the-art performance on this task is achieved by deep learning (DL)-based methods that mainly benefit from abundant pre-collected paired rainy-clean samples either manually synthesized or semi-automatically generated under human supervision. This tends to bring a large labor for data collection and more importantly, such manner neglects to elaborately explore the intrinsic generative mechanism of rain streaks which should be related to the most insightful understanding of the task. Against this issue, we investigate the generative process of rainy image and construct a full Bayesian generative model for generating rains from automatically extracted latent variables that represent physical structural factors for depicting rains, like direction, scale, and thickness. To solve this model, we propose an algorithm where the posteriors of latent variables are parameterized as CNNs and all the involved parameters can be inferred under a concise variational inference framework in a data-driven manner. Especially, the rain layer is modeled as an implicit distribution, parameterized as a generator, which avoids subjective prior assumptions on rains as in traditional model-based methods. More practically, from the learned generator, rain patches can be automatically generated and utilized to simulate diverse training pairs so as to enrich and augment the existing benchmark datasets. Comprehensive experiments substantiate that the proposed model has fine capability of generating plausible samples that not only helps significantly improve the deraining performance of current DL-based single image derainers, but also largely loosens the requirement of large training sample pre-collection for the task.

READ FULL TEXT

page 3

page 8

page 9

page 14

page 15

page 16

05/04/2020

A Model-driven Deep Neural Network for Single Image Rain Removal

Deep learning (DL) methods have achieved state-of-the-art performance in...
05/19/2020

Structural Residual Learning for Single Image Rain Removal

To alleviate the adverse effect of rain streaks in image processing task...
08/25/2020

Variational Image Restoration Network

Deep neural networks (DNNs) have achieved significant success in image r...
06/30/2020

You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing Neural Network

In this paper, we study two challenging and less-touched problems in sin...
07/29/2018

Semi-supervised CNN for Single Image Rain Removal

Single image rain removal is a typical inverse problem in computer visio...
06/05/2021

A Deep Variational Bayesian Framework for Blind Image Deblurring

Blind image deblurring is an important yet very challenging problem in l...

Code Repositories

VRGNet

【CVPR 2021, Variational Inference Framework, PyTorch】 From Rain Generation to Rain Removal


view repo