Memory-guided Image De-raining Using Time-Lapse Data

01/06/2022
by   Jaehoon Cho, et al.
8

This paper addresses the problem of single image de-raining, that is, the task of recovering clean and rain-free background scenes from a single image obscured by a rainy artifact. Although recent advances adopt real-world time-lapse data to overcome the need for paired rain-clean images, they are limited to fully exploit the time-lapse data. The main cause is that, in terms of network architectures, they could not capture long-term rain streak information in the time-lapse data during training owing to the lack of memory components. To address this problem, we propose a novel network architecture based on a memory network that explicitly helps to capture long-term rain streak information in the time-lapse data. Our network comprises the encoder-decoder networks and a memory network. The features extracted from the encoder are read and updated in the memory network that contains several memory items to store rain streak-aware feature representations. With the read/update operation, the memory network retrieves relevant memory items in terms of the queries, enabling the memory items to represent the various rain streaks included in the time-lapse data. To boost the discriminative power of memory features, we also present a novel background selective whitening (BSW) loss for capturing only rain streak information in the memory network by erasing the background information. Experimental results on standard benchmarks demonstrate the effectiveness and superiority of our approach.

READ FULL TEXT

page 1

page 4

page 5

page 8

page 9

page 10

page 11

page 12

research
04/12/2021

Memory-guided Unsupervised Image-to-image Translation

We present a novel unsupervised framework for instance-level image-to-im...
research
10/17/2022

Keep Me Updated! Memory Management in Long-term Conversations

Remembering important information from the past and continuing to talk a...
research
07/25/2019

Learning Transparent Object Matting

This paper addresses the problem of image matting for transparent object...
research
05/08/2018

The Effectiveness of Instance Normalization: a Strong Baseline for Single Image Dehazing

We propose a novel deep neural network architecture for the challenging ...
research
03/13/2018

TOM-Net: Learning Transparent Object Matting from a Single Image

This paper addresses the problem of transparent object matting. Existing...
research
04/22/2021

Aerial Scene Understanding in The Wild: Multi-Scene Recognition via Prototype-based Memory Networks

Aerial scene recognition is a fundamental visual task and has attracted ...
research
01/29/2021

Re Learning Memory Guided Normality for Anomaly Detection

The authors have introduced a novel method for unsupervised anomaly dete...

Please sign up or login with your details

Forgot password? Click here to reset