Variational Autoencoder with Implicit Optimal Priors

by   Hiroshi Takahashi, et al.

The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the standard Gaussian distribution is usually used for the prior, this simple prior incurs over-regularization. As a sophisticated prior, the aggregated posterior has been introduced, which is the expectation of the posterior over the data distribution. This prior is optimal for the VAE in terms of maximizing the training objective function. However, KL divergence with the aggregated posterior cannot be calculated in a closed form, which prevents us from using this optimal prior. With the proposed method, we introduce the density ratio trick to estimate this KL divergence without modeling the aggregated posterior explicitly. Since the density ratio trick does not work well in high dimensions, we rewrite this KL divergence that contains the high-dimensional density ratio into the sum of the analytically calculable term and the low-dimensional density ratio term, to which the density ratio trick is applied. Experiments on various datasets show that the VAE with this implicit optimal prior achieves high density estimation performance.


page 1

page 2

page 3

page 4


VAE-KRnet and its applications to variational Bayes

In this work, we have proposed a generative model for density estimation...

Cauchy-Schwarz Regularized Autoencoder

Recent work in unsupervised learning has focused on efficient inference ...

Deep Data Density Estimation through Donsker-Varadhan Representation

Estimating the data density is one of the challenging problems in deep l...

Biadversarial Variational Autoencoder

In the original version of the Variational Autoencoder, Kingma et al. as...

Posterior Ratio Estimation for Latent Variables

Density Ratio Estimation has attracted attention from machine learning c...

The Usual Suspects? Reassessing Blame for VAE Posterior Collapse

In narrow asymptotic settings Gaussian VAE models of continuous data hav...

Learning Posterior and Prior for Uncertainty Modeling in Person Re-Identification

Data uncertainty in practical person reID is ubiquitous, hence it requir...

Please sign up or login with your details

Forgot password? Click here to reset