DeepAI AI Chat
Log In Sign Up

Video Compression With Rate-Distortion Autoencoders

by   Amirhossein Habibian, et al.

In this paper we present a a deep generative model for lossy video compression. We employ a model that consists of a 3D autoencoder with a discrete latent space and an autoregressive prior used for entropy coding. Both autoencoder and prior are trained jointly to minimize a rate-distortion loss, which is closely related to the ELBO used in variational autoencoders. Despite its simplicity, we find that our method outperforms the state-of-the-art learned video compression networks based on motion compensation or interpolation. We systematically evaluate various design choices, such as the use of frame-based or spatio-temporal autoencoders, and the type of autoregressive prior. In addition, we present three extensions of the basic method that demonstrate the benefits over classical approaches to compression. First, we introduce semantic compression, where the model is trained to allocate more bits to objects of interest. Second, we study adaptive compression, where the model is adapted to a domain with limited variability, e.g., videos taken from an autonomous car, to achieve superior compression on that domain. Finally, we introduce multimodal compression, where we demonstrate the effectiveness of our model in joint compression of multiple modalities captured by non-standard imaging sensors, such as quad cameras. We believe that this opens up novel video compression applications, which have not been feasible with classical codecs.


page 5

page 8

page 14


Deep Probabilistic Video Compression

We propose a variational inference approach to deep probabilistic video ...

Variational image compression with a scale hyperprior

We describe an end-to-end trainable model for image compression based on...

End-to-end optimized image compression with competition of prior distributions

Convolutional autoencoders are now at the forefront of image compression...

Joint Autoregressive and Hierarchical Priors for Learned Image Compression

Recent models for learned image compression are based on autoencoders, l...

Feedback Recurrent Autoencoder for Video Compression

Recent advances in deep generative modeling have enabled efficient model...

Overfitting for Fun and Profit: Instance-Adaptive Data Compression

Neural data compression has been shown to outperform classical methods i...

Feedback Recurrent AutoEncoder

In this work, we propose a new recurrent autoencoder architecture, terme...