Quality-Gated Convolutional LSTM for Enhancing compressed video

03/11/2019
by   Ren Yang, et al.
0

The past decade has witnessed great success in applying deep learning to enhance the quality of compressed video. However, the existing approaches aim at quality enhancement on a single frame, or only using fixed neighboring frames. Thus they fail to take full advantage of the inter-frame correlation in the video. This paper proposes the Quality-Gated Convolutional Long Short-Term Memory (QG-ConvLSTM) network with bi-directional recurrent structure to fully exploit the advantageous information in a large range of frames. More importantly, due to the obvious quality fluctuation among compressed frames, higher quality frames can provide more useful information for other frames to enhance quality. Therefore, we propose learning the "forget" and "input" gates in the ConvLSTM cell from quality-related features. As such, the frames with various quality contribute to the memory in ConvLSTM with different importance, making the information of each frame reasonably and adequately used. Finally, the experiments validate the effectiveness of our QG-ConvLSTM approach in advancing the state-of-the-art quality enhancement of compressed video, and the ablation study shows that our QG-ConvLSTM approach is learnt to make a trade-off between quality and correlation when leveraging multi-frame information.

READ FULL TEXT

page 2

page 3

page 6

research
02/26/2019

MFQE 2.0: A New Approach for Multi-frame Quality Enhancement on Compressed Video

The past few years have witnessed great success in applying deep learnin...
research
11/22/2018

MGANet: A Robust Model for Quality Enhancement of Compressed Video

In video compression, most of the existing deep learning approaches conc...
research
03/13/2018

Multi-Frame Quality Enhancement for Compressed Video

The past few years have witnessed great success in applying deep learnin...
research
06/16/2022

PeQuENet: Perceptual Quality Enhancement of Compressed Video with Adaptation- and Attention-based Network

In this paper we propose a generative adversarial network (GAN) framewor...
research
02/21/2015

Intra-and-Inter-Constraint-based Video Enhancement based on Piecewise Tone Mapping

Video enhancement plays an important role in various video applications....
research
11/29/2018

AdaFrame: Adaptive Frame Selection for Fast Video Recognition

We present AdaFrame, a framework that adaptively selects relevant frames...
research
01/27/2022

Multi-Frame Quality Enhancement On Compressed Video Using Quantised Data of Deep Belief Networks

In the age of streaming and surveillance compressed video enhancement ha...

Please sign up or login with your details

Forgot password? Click here to reset