Temporal Kernel Consistency for Blind Video Super-Resolution

08/18/2021
by   Lichuan Xiang, et al.
6

Deep learning-based blind super-resolution (SR) methods have recently achieved unprecedented performance in upscaling frames with unknown degradation. These models are able to accurately estimate the unknown downscaling kernel from a given low-resolution (LR) image in order to leverage the kernel during restoration. Although these approaches have largely been successful, they are predominantly image-based and therefore do not exploit the temporal properties of the kernels across multiple video frames. In this paper, we investigated the temporal properties of the kernels and highlighted its importance in the task of blind video super-resolution. Specifically, we measured the kernel temporal consistency of real-world videos and illustrated how the estimated kernels might change per frame in videos of varying dynamicity of the scene and its objects. With this new insight, we revisited previous popular video SR approaches, and showed that previous assumptions of using a fixed kernel throughout the restoration process can lead to visual artifacts when upscaling real-world videos. In order to counteract this, we tailored existing single-image and video SR techniques to leverage kernel consistency during both kernel estimation and video upscaling processes. Extensive experiments on synthetic and real-world videos show substantial restoration gains quantitatively and qualitatively, achieving the new state-of-the-art in blind video SR and underlining the potential of exploiting kernel temporal consistency.

READ FULL TEXT

page 4

page 5

page 8

page 13

page 14

page 15

page 16

research
01/19/2022

Self-Supervised Deep Blind Video Super-Resolution

Existing deep learning-based video super-resolution (SR) methods usually...
research
11/09/2020

DynaVSR: Dynamic Adaptive Blind Video Super-Resolution

Most conventional supervised super-resolution (SR) algorithms assume tha...
research
12/14/2021

Kernel-aware Raw Burst Blind Super-Resolution

Burst super-resolution (SR) provides a possibility of restoring rich det...
research
10/09/2017

UG^2: a Video Benchmark for Assessing the Impact of Image Restoration andEnhancement on Automatic Visual Recognition

Advances in image restoration and enhancement techniques have led to dis...
research
03/29/2021

Flow-based Kernel Prior with Application to Blind Super-Resolution

Kernel estimation is generally one of the key problems for blind image s...
research
12/15/2022

Meta-Learned Kernel For Blind Super-Resolution Kernel Estimation

Recent image degradation estimation methods have enabled single-image su...
research
11/19/2020

Recursive Deep Prior Video: a Super Resolution algorithm for Time-Lapse Microscopy of organ-on-chip experiments

Biological experiments based on organ-on-chips (OOCs) exploit light Time...

Please sign up or login with your details

Forgot password? Click here to reset