Video Face Super-Resolution with Motion-Adaptive Feedback Cell

02/15/2020
by   Jingwei Xin, et al.
6

Video super-resolution (VSR) methods have recently achieved a remarkable success due to the development of deep convolutional neural networks (CNN). Current state-of-the-art CNN methods usually treat the VSR problem as a large number of separate multi-frame super-resolution tasks, at which a batch of low resolution (LR) frames is utilized to generate a single high resolution (HR) frame, and running a slide window to select LR frames over the entire video would obtain a series of HR frames. However, duo to the complex temporal dependency between frames, with the number of LR input frames increase, the performance of the reconstructed HR frames become worse. The reason is in that these methods lack the ability to model complex temporal dependencies and hard to give an accurate motion estimation and compensation for VSR process. Which makes the performance degrade drastically when the motion in frames is complex. In this paper, we propose a Motion-Adaptive Feedback Cell (MAFC), a simple but effective block, which can efficiently capture the motion compensation and feed it back to the network in an adaptive way. Our approach efficiently utilizes the information of the inter-frame motion, the dependence of the network on motion estimation and compensation method can be avoid. In addition, benefiting from the excellent nature of MAFC, the network can achieve better performance in the case of extremely complex motion scenarios. Extensive evaluations and comparisons validate the strengths of our approach, and the experimental results demonstrated that the proposed framework is outperform the state-of-the-art methods.

READ FULL TEXT

page 1

page 2

page 5

page 7

research
01/14/2018

Frame-Recurrent Video Super-Resolution

Recent advances in video super-resolution have shown that convolutional ...
research
11/23/2019

Self-Enhanced Convolutional Network for Facial Video Hallucination

As a domain-specific super-resolution problem, facial image hallucinatio...
research
11/23/2016

Multiframe Motion Coupling for Video Super Resolution

The idea of video super resolution is to use different view points of a ...
research
01/05/2020

End-To-End Trainable Video Super-Resolution Based on a New Mechanism for Implicit Motion Estimation and Compensation

Video super-resolution aims at generating a high-resolution video from i...
research
09/28/2019

Frame and Feature-Context Video Super-Resolution

For video super-resolution, current state-of-the-art approaches either p...
research
04/10/2017

Detail-revealing Deep Video Super-resolution

Previous CNN-based video super-resolution approaches need to align multi...
research
08/01/2017

Real-time Deep Video Deinterlacing

Interlacing is a widely used technique, for television broadcast and vid...

Please sign up or login with your details

Forgot password? Click here to reset