Multi-Stage Raw Video Denoising with Adversarial Loss and Gradient Mask

03/04/2021
by   Avinash Paliwal, et al.
0

In this paper, we propose a learning-based approach for denoising raw videos captured under low lighting conditions. We propose to do this by first explicitly aligning the neighboring frames to the current frame using a convolutional neural network (CNN). We then fuse the registered frames using another CNN to obtain the final denoised frame. To avoid directly aligning the temporally distant frames, we perform the two processes of alignment and fusion in multiple stages. Specifically, at each stage, we perform the denoising process on three consecutive input frames to generate the intermediate denoised frames which are then passed as the input to the next stage. By performing the process in multiple stages, we can effectively utilize the information of neighboring frames without directly aligning the temporally distant frames. We train our multi-stage system using an adversarial loss with a conditional discriminator. Specifically, we condition the discriminator on a soft gradient mask to prevent introducing high-frequency artifacts in smooth regions. We show that our system is able to produce temporally coherent videos with realistic details. Furthermore, we demonstrate through extensive experiments that our approach outperforms state-of-the-art image and video denoising methods both numerically and visually.

READ FULL TEXT

page 4

page 5

page 6

page 7

page 8

research
05/10/2022

Efficient Burst Raw Denoising with Variance Stabilization and Multi-frequency Denoising Network

With the growing popularity of smartphones, capturing high-quality image...
research
06/12/2017

A filter based approach for inbetweening

We present a filter based approach for inbetweening. We train a convolut...
research
07/01/2023

Unsupervised Coordinate-Based Video Denoising

In this paper, we introduce a novel unsupervised video denoising deep le...
research
07/27/2021

Multi-Scale Local-Temporal Similarity Fusion for Continuous Sign Language Recognition

Continuous sign language recognition (cSLR) is a public significant task...
research
04/21/2019

Automatic Temporally Coherent Video Colorization

Greyscale image colorization for applications in image restoration has s...
research
07/13/2021

Developmental Stage Classification of Embryos Using Two-Stream Neural Network with Linear-Chain Conditional Random Field

The developmental process of embryos follows a monotonic order. An embry...
research
06/04/2021

Temporally coherent video anonymization through GAN inpainting

This work tackles the problem of temporally coherent face anonymization ...

Please sign up or login with your details

Forgot password? Click here to reset