Learning in Variational Autoencoders with Kullback-Leibler and Renyi Integral Bounds

07/05/2018
by   Septimia Sârbu, et al.
0

In this paper we propose two novel bounds for the log-likelihood based on Kullback-Leibler and the Rényi divergences, which can be used for variational inference and in particular for the training of Variational AutoEncoders. Our proposal is motivated by the difficulties encountered in training VAEs on continuous datasets with high contrast images, such as those with handwritten digits and characters, where numerical issues often appear unless noise is added, either to the dataset during training or to the generative model given by the decoder. The new bounds we propose, which are obtained from the maximization of the likelihood of an interval for the observations, allow numerically stable training procedures without the necessity of adding any extra source of noise to the data.

READ FULL TEXT

page 5

page 6

research
02/27/2019

Training Variational Autoencoders with Buffered Stochastic Variational Inference

The recognition network in deep latent variable models such as variation...
research
02/06/2016

Ladder Variational Autoencoders

Variational Autoencoders are powerful models for unsupervised learning. ...
research
06/09/2020

Super-resolution Variational Auto-Encoders

The framework of variational autoencoders (VAEs) provides a principled m...
research
05/03/2023

Tensorizing flows: a tool for variational inference

Fueled by the expressive power of deep neural networks, normalizing flow...
research
07/22/2017

Adversarial Variational Optimization of Non-Differentiable Simulators

Complex computer simulators are increasingly used across fields of scien...
research
07/09/2021

InfoVAEGAN : learning joint interpretable representations by information maximization and maximum likelihood

Learning disentangled and interpretable representations is an important ...
research
11/04/2021

Variational Inference with Holder Bounds

The recent introduction of thermodynamic integration techniques has prov...

Please sign up or login with your details

Forgot password? Click here to reset