VAE Approximation Error: ELBO and Conditional Independence

02/18/2021
by   Dmitrij Schlesinger, et al.
0

The importance of Variational Autoencoders reaches far beyond standalone generative models – the approach is also used for learning latent representations and can be generalized to semi-supervised learning. This requires a thorough analysis of their commonly known shortcomings: posterior collapse and approximation errors. This paper analyzes VAE approximation errors caused by the combination of the ELBO objective with the choice of the encoder probability family, in particular under conditional independence assumptions. We identify the subclass of generative models consistent with the encoder family. We show that the ELBO optimizer is pulled from the likelihood optimizer towards this consistent subset. Furthermore, this subset can not be enlarged, and the respective error cannot be decreased, by only considering deeper encoder networks.

READ FULL TEXT
research
01/31/2020

On Implicit Regularization in β-VAEs

While the impact of variational inference (VI) on posterior inference in...
research
05/07/2019

Adversarial Variational Embedding for Robust Semi-supervised Learning

Semi-supervised learning is sought for leveraging the unlabelled data wh...
research
06/01/2017

Learning Disentangled Representations with Semi-Supervised Deep Generative Models

Variational autoencoders (VAEs) learn representations of data by jointly...
research
09/09/2021

Supervising the Decoder of Variational Autoencoders to Improve Scientific Utility

Probabilistic generative models are attractive for scientific modeling b...
research
12/03/2020

Lookahead optimizer improves the performance of Convolutional Autoencoders for reconstruction of natural images

Autoencoders are a class of artificial neural networks which have gained...
research
11/21/2020

SHOT-VAE: Semi-supervised Deep Generative Models With Label-aware ELBO Approximations

Semi-supervised variational autoencoders (VAEs) have obtained strong res...
research
07/19/2023

Symmetric Equilibrium Learning of VAEs

We view variational autoencoders (VAE) as decoder-encoder pairs, which m...

Please sign up or login with your details

Forgot password? Click here to reset