Investigating Tradeoffs in Real-World Video Super-Resolution

11/24/2021
by   Kelvin C. K. Chan, et al.
24

The diversity and complexity of degradations in real-world video super-resolution (VSR) pose non-trivial challenges in inference and training. First, while long-term propagation leads to improved performance in cases of mild degradations, severe in-the-wild degradations could be exaggerated through propagation, impairing output quality. To balance the tradeoff between detail synthesis and artifact suppression, we found an image pre-cleaning stage indispensable to reduce noises and artifacts prior to propagation. Equipped with a carefully designed cleaning module, our RealBasicVSR outperforms existing methods in both quality and efficiency. Second, real-world VSR models are often trained with diverse degradations to improve generalizability, requiring increased batch size to produce a stable gradient. Inevitably, the increased computational burden results in various problems, including 1) speed-performance tradeoff and 2) batch-length tradeoff. To alleviate the first tradeoff, we propose a stochastic degradation scheme that reduces up to 40% of training time without sacrificing performance. We then analyze different training settings and suggest that employing longer sequences rather than larger batches during training allows more effective uses of temporal information, leading to more stable performance during inference. To facilitate fair comparisons, we propose the new VideoLQ dataset, which contains a large variety of real-world low-quality video sequences containing rich textures and patterns. Our dataset can serve as a common ground for benchmarking. Code, models, and the dataset will be made publicly available.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 8

page 12

page 13

page 14

research
12/16/2021

Stable Long-Term Recurrent Video Super-Resolution

Recurrent models have gained popularity in deep learning (DL) based vide...
research
05/06/2022

VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution

Most of the existing video face super-resolution (VFSR) methods are trai...
research
05/04/2023

Expanding Synthetic Real-World Degradations for Blind Video Super Resolution

Video super-resolution (VSR) techniques, especially deep-learning-based ...
research
06/14/2022

AnimeSR: Learning Real-World Super-Resolution Models for Animation Videos

This paper studies the problem of real-world video super-resolution (VSR...
research
05/24/2023

NegVSR: Augmenting Negatives for Generalized Noise Modeling in Real-World Video Super-Resolution

The capability of video super-resolution (VSR) to synthesize high-resolu...
research
04/01/2019

Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model

Most of the existing learning-based single image superresolution (SISR) ...
research
07/02/2019

A Single Video Super-Resolution GAN for Multiple Downsampling Operators based on Pseudo-Inverse Image Formation Models

The popularity of high and ultra-high definition displays has led to the...

Please sign up or login with your details

Forgot password? Click here to reset