Log In Sign Up

Improved CNN-based Learning of Interpolation Filters for Low-Complexity Inter Prediction in Video Coding

by   Luka Murn, et al.

The versatility of recent machine learning approaches makes them ideal for improvement of next generation video compression solutions. Unfortunately, these approaches typically bring significant increases in computational complexity and are difficult to interpret into explainable models, affecting their potential for implementation within practical video coding applications. This paper introduces a novel explainable neural network-based inter-prediction scheme, to improve the interpolation of reference samples needed for fractional precision motion compensation. The approach requires a single neural network to be trained from which a full quarter-pixel interpolation filter set is derived, as the network is easily interpretable due to its linear structure. A novel training framework enables each network branch to resemble a specific fractional shift. This practical solution makes it very efficient to use alongside conventional video coding schemes. When implemented in the context of the state-of-the-art Versatile Video Coding (VVC) test model, 0.77 2.25 under the random access, low-delay B and low-delay P configurations, respectively, while the complexity of the learned interpolation schemes is significantly reduced compared to the interpolation with full CNNs.


page 1

page 7

page 11


Interpreting CNN for Low Complexity Learned Sub-pixel Motion Compensation in Video Coding

Deep learning has shown great potential in image and video compression t...

An Efficient Four-Parameter Affine Motion Model for Video Coding

In this paper, we study a simplified affine motion model based coding fr...

A Convolutional Neural Network Approach for Half-Pel Interpolation in Video Coding

Motion compensation is a fundamental technology in video coding to remov...

A Group Variational Transformation Neural Network for Fractional Interpolation of Video Coding

Motion compensation is an important technology in video coding to remove...

Deep Learning-Based Video Coding: A Review and A Case Study

The past decade has witnessed great success of deep learning technology ...

Utilising Low Complexity CNNs to Lift Non-Local Redundancies in Video Coding

Digital media is ubiquitous and produced in ever-growing quantities. Thi...

Memory Assessment of Versatile Video Coding

This paper presents a memory assessment of the next-generation Versatile...

Code Repositories


The GitHub open source software repository on interpreting super-resolution CNNs for sub-pixel motion compensation in video coding

view repo