DeepAI AI Chat
Log In Sign Up

Boosting neural video codecs by exploiting hierarchical redundancy

by   Reza Pourreza, et al.

In video compression, coding efficiency is improved by reusing pixels from previously decoded frames via motion and residual compensation. We define two levels of hierarchical redundancy in video frames: 1) first-order: redundancy in pixel space, i.e., similarities in pixel values across neighboring frames, which is effectively captured using motion and residual compensation, 2) second-order: redundancy in motion and residual maps due to smooth motion in natural videos. While most of the existing neural video coding literature addresses first-order redundancy, we tackle the problem of capturing second-order redundancy in neural video codecs via predictors. We introduce generic motion and residual predictors that learn to extrapolate from previously decoded data. These predictors are lightweight, and can be employed with most neural video codecs in order to improve their rate-distortion performance. Moreover, while RGB is the dominant colorspace in neural video coding literature, we introduce general modifications for neural video codecs to embrace the YUV420 colorspace and report YUV420 results. Our experiments show that using our predictors with a well-known neural video codec leads to 38 dataset.


page 4

page 5


Neural Residual Flow Fields for Efficient Video Representations

Implicit neural representation (INR) has emerged as a powerful paradigm ...

Spatiotemporal Entropy Model is All You Need for Learned Video Compression

The framework of dominant learned video compression methods is usually c...

Learned Video Compression with Residual Prediction and Loop Filter

In this paper, we propose a learned video codec with a residual predicti...

Ray-Space Motion Compensation for Lenslet Plenoptic Video Coding

Plenoptic images and videos bearing rich information demand a tremendous...

A Group Variational Transformation Neural Network for Fractional Interpolation of Video Coding

Motion compensation is an important technology in video coding to remove...

Intra Prediction Using In-Loop Residual Coding for the post-HEVC Standard

A few years after standardization of the High Efficiency Video Coding (H...