Copy-and-Paste Networks for Deep Video Inpainting

08/30/2019
by   Sungho Lee, et al.
13

We present a novel deep learning based algorithm for video inpainting. Video inpainting is a process of completing corrupted or missing regions in videos. Video inpainting has additional challenges compared to image inpainting due to the extra temporal information as well as the need for maintaining the temporal coherency. We propose a novel DNN-based framework called the Copy-and-Paste Networks for video inpainting that takes advantage of additional information in other frames of the video. The network is trained to copy corresponding contents in reference frames and paste them to fill the holes in the target frame. Our network also includes an alignment network that computes affine matrices between frames for the alignment, enabling the network to take information from more distant frames for robustness. Our method produces visually pleasing and temporally coherent results while running faster than the state-of-the-art optimization-based method. In addition, we extend our framework for enhancing over/under exposed frames in videos. Using this enhancement technique, we were able to significantly improve the lane detection accuracy on road videos.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 8

research
05/05/2019

Deep Video Inpainting

Video inpainting aims to fill spatio-temporal holes with plausible conte...
research
05/08/2019

Frame-Recurrent Video Inpainting by Robust Optical Flow Inference

In this paper, we present a new inpainting framework for recovering miss...
research
05/30/2019

Align-and-Attend Network for Globally and Locally Coherent Video Inpainting

We propose a novel feed-forward network for video inpainting. We use a s...
research
07/02/2019

Learnable Gated Temporal Shift Module for Deep Video Inpainting

How to efficiently utilize temporal information to recover videos in a c...
research
08/23/2019

Onion-Peel Networks for Deep Video Completion

We propose the onion-peel networks for video completion. Given a set of ...
research
07/21/2022

Error Compensation Framework for Flow-Guided Video Inpainting

The key to video inpainting is to use correlation information from as ma...
research
06/04/2021

Temporally coherent video anonymization through GAN inpainting

This work tackles the problem of temporally coherent face anonymization ...

Please sign up or login with your details

Forgot password? Click here to reset