Frame and Feature-Context Video Super-Resolution

by   Bo Yan, et al.

For video super-resolution, current state-of-the-art approaches either process multiple low-resolution (LR) frames to produce each output high-resolution (HR) frame separately in a sliding window fashion or recurrently exploit the previously estimated HR frames to super-resolve the following frame. The main weaknesses of these approaches are: 1) separately generating each output frame may obtain high-quality HR estimates while resulting in unsatisfactory flickering artifacts, and 2) combining previously generated HR frames can produce temporally consistent results in the case of short information flow, but it will cause significant jitter and jagged artifacts because the previous super-resolving errors are constantly accumulated to the subsequent frames. In this paper, we propose a fully end-to-end trainable frame and feature-context video super-resolution (FFCVSR) network that consists of two key sub-networks: local network and context network, where the first one explicitly utilizes a sequence of consecutive LR frames to generate local feature and local SR frame, and the other combines the outputs of local network and the previously estimated HR frames and features to super-resolve the subsequent frame. Our approach takes full advantage of the inter-frame information from multiple LR frames and the context information from previously predicted HR frames, producing temporally consistent high-quality results while maintaining real-time speed by directly reusing previous features and frames. Extensive evaluations and comparisons demonstrate that our approach produces state-of-the-art results on a standard benchmark dataset, with advantages in terms of accuracy, efficiency, and visual quality over the existing approaches.


page 4

page 6

page 7


Frame-Recurrent Video Super-Resolution

Recent advances in video super-resolution have shown that convolutional ...

Prediction-assistant Frame Super-Resolution for Video Streaming

Video frame transmission delay is critical in real-time applications suc...

Detail-revealing Deep Video Super-resolution

Previous CNN-based video super-resolution approaches need to align multi...

Omniscient Video Super-Resolution

Most recent video super-resolution (SR) methods either adopt an iterativ...

Look Back and Forth: Video Super-Resolution with Explicit Temporal Difference Modeling

Temporal modeling is crucial for video super-resolution. Most of the vid...

Video Face Super-Resolution with Motion-Adaptive Feedback Cell

Video super-resolution (VSR) methods have recently achieved a remarkable...

Self-Enhanced Convolutional Network for Facial Video Hallucination

As a domain-specific super-resolution problem, facial image hallucinatio...

Please sign up or login with your details

Forgot password? Click here to reset