DeepAI AI Chat
Log In Sign Up

From Rain Removal to Rain Generation

by   Hong Wang, et al.

Single image deraining is an important yet challenging issue due to the complex and diverse rain structures in real scenes. Currently, the state-of-the-art performance on this task is achieved by deep learning (DL)-based methods that mainly benefit from abundant pre-collected paired rainy-clean samples either manually synthesized or semi-automatically generated under human supervision. This tends to bring a large labor for data collection and more importantly, such manner neglects to elaborately explore the intrinsic generative mechanism of rain streaks which should be related to the most insightful understanding of the task. Against this issue, we investigate the generative process of rainy image and construct a full Bayesian generative model for generating rains from automatically extracted latent variables that represent physical structural factors for depicting rains, like direction, scale, and thickness. To solve this model, we propose an algorithm where the posteriors of latent variables are parameterized as CNNs and all the involved parameters can be inferred under a concise variational inference framework in a data-driven manner. Especially, the rain layer is modeled as an implicit distribution, parameterized as a generator, which avoids subjective prior assumptions on rains as in traditional model-based methods. More practically, from the learned generator, rain patches can be automatically generated and utilized to simulate diverse training pairs so as to enrich and augment the existing benchmark datasets. Comprehensive experiments substantiate that the proposed model has fine capability of generating plausible samples that not only helps significantly improve the deraining performance of current DL-based single image derainers, but also largely loosens the requirement of large training sample pre-collection for the task.


page 3

page 8

page 9

page 14

page 15

page 16


A Model-driven Deep Neural Network for Single Image Rain Removal

Deep learning (DL) methods have achieved state-of-the-art performance in...

Structural Residual Learning for Single Image Rain Removal

To alleviate the adverse effect of rain streaks in image processing task...

Variational Image Restoration Network

Deep neural networks (DNNs) have achieved significant success in image r...

You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing Neural Network

In this paper, we study two challenging and less-touched problems in sin...

Semi-supervised CNN for Single Image Rain Removal

Single image rain removal is a typical inverse problem in computer visio...

Unsupervised Single Image Super-resolution Under Complex Noise

While the researches on single image super-resolution (SISR), especially...

Patchwise Generative ConvNet: Training Energy-Based Models from a Single Natural Image for Internal Learning

Exploiting internal statistics of a single natural image has long been r...

Code Repositories


【CVPR 2021, Variational Inference Framework, PyTorch】 From Rain Generation to Rain Removal

view repo