Prediction-assistant Frame Super-Resolution for Video Streaming

03/17/2021
by   Wang Shen, et al.
0

Video frame transmission delay is critical in real-time applications such as online video gaming, live show, etc. The receiving deadline of a new frame must catch up with the frame rendering time. Otherwise, the system will buffer a while, and the user will encounter a frozen screen, resulting in unsatisfactory user experiences. An effective approach is to transmit frames in lower-quality under poor bandwidth conditions, such as using scalable video coding. In this paper, we propose to enhance video quality using lossy frames in two situations. First, when current frames are too late to receive before rendering deadline (i.e., lost), we propose to use previously received high-resolution images to predict the future frames. Second, when the quality of the currently received frames is low (i.e., lossy), we propose to use previously received high-resolution frames to enhance the low-quality current ones. For the first case, we propose a small yet effective video frame prediction network. For the second case, we improve the video prediction network to a video enhancement network to associate current frames as well as previous frames to restore high-quality images. Extensive experimental results demonstrate that our method performs favorably against state-of-the-art algorithms in the lossy video streaming environment.

READ FULL TEXT

page 2

page 3

page 7

research
09/28/2019

Frame and Feature-Context Video Super-Resolution

For video super-resolution, current state-of-the-art approaches either p...
research
03/05/2022

A Novel Dual Dense Connection Network for Video Super-resolution

Video super-resolution (VSR) refers to the reconstruction of high-resolu...
research
08/23/2022

Wavelet-Based Fast Decoding of 360-Degree Videos

In this paper, we propose a wavelet-based video codec specifically desig...
research
07/15/2023

Neural Video Recovery for Cloud Gaming

Cloud gaming is a multi-billion dollar industry. A client in cloud gamin...
research
11/11/2013

Stitched Panoramas from Toy Airborne Video Cameras

Effective panoramic photographs are taken from vantage points that are h...
research
02/21/2015

Intra-and-Inter-Constraint-based Video Enhancement based on Piecewise Tone Mapping

Video enhancement plays an important role in various video applications....

Please sign up or login with your details

Forgot password? Click here to reset