ELF-VC: Efficient Learned Flexible-Rate Video Coding

04/29/2021
by   Oren Rippel, et al.
12

While learned video codecs have demonstrated great promise, they have yet to achieve sufficient efficiency for practical deployment. In this work, we propose several novel ideas for learned video compression which allow for improved performance for the low-latency mode (I- and P-frames only) along with a considerable increase in computational efficiency. In this setting, for natural videos our approach compares favorably across the entire R-D curve under metrics PSNR, MS-SSIM and VMAF against all mainstream video standards (H.264, H.265, AV1) and all ML codecs. At the same time, our approach runs at least 5x faster and has fewer parameters than all ML codecs which report these figures. Our contributions include a flexible-rate framework allowing a single model to cover a large and dense range of bitrates, at a negligible increase in computation and parameter count; an efficient backbone optimized for ML-based codecs; and a novel in-loop flow prediction scheme which leverages prior information towards more efficient compression. We benchmark our method, which we call ELF-VC (Efficient, Learned and Flexible Video Coding) on popular video test sets UVG and MCL-JCV under metrics PSNR, MS-SSIM and VMAF. For example, on UVG under PSNR, it reduces the BD-rate by 44 current best ML codec. At the same time, on an NVIDIA Titan V GPU our approach encodes/decodes VGA at 49/91 FPS, HD 720 at 19/35 FPS, and HD 1080 at 10/18 FPS.

READ FULL TEXT

page 3

page 4

page 5

page 12

research
11/16/2018

Learned Video Compression

We present a new algorithm for video coding, learned end-to-end for the ...
research
12/17/2021

End-to-End Rate-Distortion Optimized Learned Hierarchical Bi-Directional Video Compression

Conventional video compression (VC) methods are based on motion compensa...
research
06/27/2022

Flexible-Rate Learned Hierarchical Bi-Directional Video Compression With Motion Refinement and Frame-Level Bit Allocation

This paper presents improvements and novel additions to our recent work ...
research
06/29/2020

OpenDVC: An Open Source Implementation of the DVC Video Compression Method

We introduce an open source Tensorflow implementation of the Deep Video ...
research
04/09/2020

Feedback Recurrent Autoencoder for Video Compression

Recent advances in deep generative modeling have enabled efficient model...
research
11/05/2021

Versatile Learned Video Compression

Learned video compression methods have demonstrated great promise in cat...

Please sign up or login with your details

Forgot password? Click here to reset