Deep Learning-Based Video Coding: A Review and A Case Study

04/29/2019
by   Dong Liu, et al.
0

The past decade has witnessed great success of deep learning technology in many disciplines, especially in computer vision and image processing. However, deep learning-based video coding remains in its infancy. This paper reviews the representative works about using deep learning for image/video coding, which has been an actively developing research area since the year of 2015. We divide the related works into two categories: new coding schemes that are built primarily upon deep networks (deep schemes), and deep network-based coding tools (deep tools) that shall be used within traditional coding schemes or together with traditional coding tools. For deep schemes, pixel probability modeling and auto-encoder are the two approaches, that can be viewed as predictive coding scheme and transform coding scheme, respectively. For deep tools, there have been several proposed techniques using deep learning to perform intra-picture prediction, inter-picture prediction, cross-channel prediction, probability distribution prediction, transform, post- or in-loop filtering, down- and up-sampling, as well as encoding optimizations. In the hope of advocating the research of deep learning-based video coding, we present a case study of our developed prototype video codec, namely Deep Learning Video Coding (DLVC). DLVC features two deep tools that are both based on convolutional neural network (CNN), namely CNN-based in-loop filter (CNN-ILF) and CNN-based block adaptive resolution coding (CNN-BARC). Both tools help improve the compression efficiency by a significant margin. With the two deep tools as well as other non-deep coding tools, DLVC is able to achieve on average 39.6% and 33.0% bits saving than HEVC, under random-access and low-delay configurations, respectively. The source code of DLVC has been released for future researches.

READ FULL TEXT
research
02/22/2017

Convolutional Neural Network-Based Block Up-sampling for Intra Frame Coding

Inspired by the recent advances of image super-resolution using convolut...
research
12/25/2020

Study On Coding Tools Beyond Av1

The Alliance for Open Media has recently initiated coding tool explorati...
research
03/30/2020

BVI-DVC: A Training Database for Deep Video Compression

Deep learning methods are increasingly being applied in the optimisation...
research
11/22/2019

Dual Learning-based Video Coding with Inception Dense Blocks

In this paper, a dual learning-based method in intra coding is introduce...
research
06/16/2021

Improved CNN-based Learning of Interpolation Filters for Low-Complexity Inter Prediction in Video Coding

The versatility of recent machine learning approaches makes them ideal f...
research
07/14/2020

MFRNet: A New CNN Architecture for Post-Processing and In-loop Filtering

In this paper, we propose a novel convolutional neural network (CNN) arc...
research
03/10/2017

A Convolutional Neural Network Approach for Half-Pel Interpolation in Video Coding

Motion compensation is a fundamental technology in video coding to remov...

Please sign up or login with your details

Forgot password? Click here to reset