Log In Sign Up

End-to-End Rate-Distortion Optimization for Bi-Directional Learned Video Compression

by   M. Akin Yilmaz, et al.

Conventional video compression methods employ a linear transform and block motion model, and the steps of motion estimation, mode and quantization parameter selection, and entropy coding are optimized individually due to combinatorial nature of the end-to-end optimization problem. Learned video compression allows end-to-end rate-distortion optimized training of all nonlinear modules, quantization parameter and entropy model simultaneously. While previous work on learned video compression considered training a sequential video codec based on end-to-end optimization of cost averaged over pairs of successive frames, it is well-known in conventional video compression that hierarchical, bi-directional coding outperforms sequential compression. In this paper, we propose for the first time end-to-end optimization of a hierarchical, bi-directional motion compensated learned codec by accumulating cost function over fixed-size groups of pictures (GOP). Experimental results show that the rate-distortion performance of our proposed learned bi-directional GOP coder outperforms the state-of-the-art end-to-end optimized learned sequential compression as expected.


page 1

page 2

page 3

page 4


End-to-End Rate-Distortion Optimized Learned Hierarchical Bi-Directional Video Compression

Conventional video compression (VC) methods are based on motion compensa...

Lossy Compression with Distortion Constrained Optimization

When training end-to-end learned models for lossy compression, one has t...

Learning End-to-End Lossy Image Compression: A Benchmark

Image compression is one of the most fundamental techniques and commonly...

Characterizing Generalized Rate-Distortion Performance of Video Coding: An Eigen Analysis Approach

Rate-distortion (RD) theory is at the heart of lossy data compression. H...

AIVC: Artificial Intelligence based Video Codec

This paper introduces AIVC, an end-to-end neural video codec. It is base...

Parallelized Rate-Distortion Optimized Quantization Using Deep Learning

Rate-Distortion Optimized Quantization (RDOQ) has played an important ro...