Video Compression With Rate-Distortion Autoencoders

08/14/2019
by   Amirhossein Habibian, et al.
7

In this paper we present a a deep generative model for lossy video compression. We employ a model that consists of a 3D autoencoder with a discrete latent space and an autoregressive prior used for entropy coding. Both autoencoder and prior are trained jointly to minimize a rate-distortion loss, which is closely related to the ELBO used in variational autoencoders. Despite its simplicity, we find that our method outperforms the state-of-the-art learned video compression networks based on motion compensation or interpolation. We systematically evaluate various design choices, such as the use of frame-based or spatio-temporal autoencoders, and the type of autoregressive prior. In addition, we present three extensions of the basic method that demonstrate the benefits over classical approaches to compression. First, we introduce semantic compression, where the model is trained to allocate more bits to objects of interest. Second, we study adaptive compression, where the model is adapted to a domain with limited variability, e.g., videos taken from an autonomous car, to achieve superior compression on that domain. Finally, we introduce multimodal compression, where we demonstrate the effectiveness of our model in joint compression of multiple modalities captured by non-standard imaging sensors, such as quad cameras. We believe that this opens up novel video compression applications, which have not been feasible with classical codecs.

READ FULL TEXT

page 5

page 8

page 14

research
10/05/2018

Deep Probabilistic Video Compression

We propose a variational inference approach to deep probabilistic video ...
research
02/01/2018

Variational image compression with a scale hyperprior

We describe an end-to-end trainable model for image compression based on...
research
11/17/2021

End-to-end optimized image compression with competition of prior distributions

Convolutional autoencoders are now at the forefront of image compression...
research
09/08/2018

Joint Autoregressive and Hierarchical Priors for Learned Image Compression

Recent models for learned image compression are based on autoencoders, l...
research
04/09/2020

Feedback Recurrent Autoencoder for Video Compression

Recent advances in deep generative modeling have enabled efficient model...
research
01/21/2021

Overfitting for Fun and Profit: Instance-Adaptive Data Compression

Neural data compression has been shown to outperform classical methods i...
research
07/28/2021

Insights from Generative Modeling for Neural Video Compression

While recent machine learning research has revealed connections between ...

Please sign up or login with your details

Forgot password? Click here to reset