A Tutorial on VAEs: From Bayes' Rule to Lossless Compression

06/18/2020

∙

The Variational Auto-Encoder (VAE) is a simple, efficient, and popular deep maximum likelihood model. Though usage of VAEs is widespread, the derivation of the VAE is not as widely understood. In this tutorial, we will provide an overview of the VAE and a tour through various derivations and interpretations of the VAE objective. From a probabilistic standpoint, we will examine the VAE through the lens of Bayes' Rule, importance sampling, and the change-of-variables formula. From an information theoretic standpoint, we will examine the VAE through the lens of lossless compression and transmission through a noisy channel. We will then identify two common misconceptions over the VAE formulation and their practical consequences. Finally, we will visualize the capabilities and limitations of VAEs using a code example (with an accompanying Jupyter notebook) on toy 2D data.

READ FULL TEXT

A Tutorial on VAEs: From Bayes' Rule to Lossless Compression

Learning Energy-Based Model with Variational Auto-Encoder as Amortized Sampler

A Variational Auto-Encoder Model for Stochastic Point Processes

Bi-fidelity Variational Auto-encoder for Uncertainty Quantification

Latent Variables on Spheres for Sampling and Spherical Inference

Chaotic Variational Auto encoder-based Adversarial Machine Learning

Generalization Gap in Amortized Inference

β-VAEs can retain label information even at high compression

A Tutorial on VAEs: From Bayes' Rule to Lossless Compression

Related Research

Learning Energy-Based Model with Variational Auto-Encoder as Amortized Sampler

A Variational Auto-Encoder Model for Stochastic Point Processes

Bi-fidelity Variational Auto-encoder for Uncertainty Quantification

Latent Variables on Spheres for Sampling and Spherical Inference

Chaotic Variational Auto encoder-based Adversarial Machine Learning

Generalization Gap in Amortized Inference

β-VAEs can retain label information even at high compression