The DEVIL is in the Details: A Diagnostic Evaluation Benchmark for Video Inpainting

by   Ryan Szeto, et al.
University of Michigan
Stevens Institute of Technology

Quantitative evaluation has increased dramatically among recent video inpainting work, but the video and mask content used to gauge performance has received relatively little attention. Although attributes such as camera and background scene motion inherently change the difficulty of the task and affect methods differently, existing evaluation schemes fail to control for them, thereby providing minimal insight into inpainting failure modes. To address this gap, we propose the Diagnostic Evaluation of Video Inpainting on Landscapes (DEVIL) benchmark, which consists of two contributions: (i) a novel dataset of videos and masks labeled according to several key inpainting failure modes, and (ii) an evaluation scheme that samples slices of the dataset characterized by a fixed content attribute, and scores performance on each slice according to reconstruction, realism, and temporal consistency quality. By revealing systematic changes in performance induced by particular characteristics of the input content, our challenging benchmark enables more insightful analysis into video inpainting methods and serves as an invaluable diagnostic tool for the field. Our code is available at .


page 2

page 5

page 7

page 8

page 11


Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN

Free-form video inpainting is a very challenging task that could be wide...

Towards An End-to-End Framework for Flow-Guided Video Inpainting

Optical flow, which captures motion information across frames, is exploi...

Deep Long Audio Inpainting

Long (> 200 ms) audio inpainting, to recover a long missing part in an a...

Resolution-robust Large Mask Inpainting with Fourier Convolutions

Modern image inpainting systems, despite the significant progress, often...

Feature Refinement to Improve High Resolution Image Inpainting

In this paper, we address the problem of degradation in inpainting quali...

Video Inpainting of Complex Scenes

We propose an automatic video inpainting algorithm which relies on the o...

Reconstructing Existing Levels through Level Inpainting

Procedural Content Generation (PCG) and Procedural Content Generation vi...

Please sign up or login with your details

Forgot password? Click here to reset