Combining Progressive Rethinking and Collaborative Learning: A Deep Framework for In-Loop Filtering

01/16/2020
by   Dezhao Wang, et al.
15

In this paper, we aim to address two critical issues in deep-learning based in-loop filter of modern codecs: 1) how to model spatial and temporal redundancies more effectively in the coding scenario; 2) what kinds of side information (side-info) can be inferred from the codecs to benefit in-loop filter models and how this side-info is injected. For the first issue, we design a deep network with both progressive rethinking and collaborative learning mechanisms to improve quality of the reconstructed intra-frames and inter-frames, respectively. For intra coding, a Progressive Rethinking Block (PRB) and its stacked Progressive Rethinking Network (PRN) are designed to simulate the human decision mechanism for effective spatial modeling. The typical cascaded deep network utilizes a bottleneck module at the end of each block to reduce the dimension size of the feature to generate the summarization of past experiences. Our designed block rethinks progressively, namely introducing an additional inter-block connection to bypass a high-dimensional informative feature across blocks to review the complete past memorized experiences. For inter coding, the model learns collaboratively for temporal modeling. The current reconstructed frame interacts with reference frames (peak quality frame and the nearest adjacent frame) progressively at the feature level. For the second issue, side-info utilization, we extract both intra-frame and interframe side-info for a better context modeling. A coarse-tofine partition map based on HEVC partition trees is built as the intra-frame side-info. Furthermore, the warped features of the reference frames are offered as the inter-frame side-info. Benefiting from our subtle design, under All-Intra (AI), Low-Delay B (LDB), Low-Delay P (LDP) and Random Access (RA) configuration, our PRNs provide 9.0 average respectively.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 11

research
09/25/2017

Spatial-Temporal Residue Network Based In-Loop Filter for Video Coding

Deep learning has demonstrated tremendous break through in the area of i...
research
03/05/2019

A DenseNet Based Approach for Multi-Frame In-Loop Filter in HEVC

High efficiency video coding (HEVC) has brought outperforming efficiency...
research
05/16/2019

Deep Reference Generation with Multi-Domain Hierarchical Constraints for Inter Prediction

Inter prediction is an important module in video coding for temporal red...
research
07/30/2022

Consistent Quality Oriented Rate Control in HEVC via Balancing Intra and Inter Frame Coding

Consistent quality oriented rate control in video coding has attracted m...
research
07/06/2018

Progressive Spatial Recurrent Neural Network for Intra Prediction

Intra prediction is an important component of modern video codecs, which...
research
07/09/2020

Neural Video Coding using Multiscale Motion Compensation and Spatiotemporal Context Model

Over the past two decades, traditional block-based video coding has made...

Please sign up or login with your details

Forgot password? Click here to reset