Real-Time Super-Resolution System of 4K-Video Based on Deep Learning

07/12/2021
by   Yanpeng Cao, et al.
0

Video super-resolution (VSR) technology excels in reconstructing low-quality video, avoiding unpleasant blur effect caused by interpolation-based algorithms. However, vast computation complexity and memory occupation hampers the edge of deplorability and the runtime inference in real-life applications, especially for large-scale VSR task. This paper explores the possibility of real-time VSR system and designs an efficient and generic VSR network, termed EGVSR. The proposed EGVSR is based on spatio-temporal adversarial learning for temporal coherence. In order to pursue faster VSR processing ability up to 4K resolution, this paper tries to choose lightweight network structure and efficient upsampling method to reduce the computation required by EGVSR network under the guarantee of high visual quality. Besides, we implement the batch normalization computation fusion, convolutional acceleration algorithm and other neural network acceleration techniques on the actual hardware platform to optimize the inference process of EGVSR network. Finally, our EGVSR achieves the real-time processing capacity of 4K@29.61FPS. Compared with TecoGAN, the most advanced VSR network at present, we achieve 85.04 computation density and 7.92x performance speedups. In terms of visual quality, the proposed EGVSR tops the list of most metrics (such as LPIPS, tOF, tLP, etc.) on the public test dataset Vid4 and surpasses other state-of-the-art methods in overall performance score. The source code of this project can be found on https://github.com/Thmen/EGVSR.

READ FULL TEXT

page 1

page 5

page 6

08/06/2023

NNVISR: Bring Neural Network Video Interpolation and Super Resolution into Video Processing Framework

We present NNVISR - an open-source filter plugin for the VapourSynth vid...
12/06/2017

Real-time Video Processing in Web Applications

The OpenGL ES standard is implemented in modern desktop and mobile brows...
05/05/2023

AsConvSR: Fast and Lightweight Super-Resolution Network with Assembled Convolutions

In recent years, videos and images in 720p (HD), 1080p (FHD) and 4K (UHD...
09/28/2021

An Efficient Network Design for Face Video Super-resolution

Face video super-resolution algorithm aims to reconstruct realistic face...
03/16/2023

A High-Performance Accelerator for Super-Resolution Processing on Embedded GPU

Recent years have witnessed impressive progress in super-resolution (SR)...
05/17/2021

Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report

Video super-resolution has recently become one of the most important mob...
08/01/2017

Real-time Deep Video Deinterlacing

Interlacing is a widely used technique, for television broadcast and vid...

Please sign up or login with your details

Forgot password? Click here to reset