A Generalised Linear Model Framework for Variational Autoencoders based on Exponential Dispersion Families

06/11/2020
by   Robert Sicks, et al.
0

Although variational autoencoders (VAE) are successfully used to obtain meaningful low-dimensional representations for high-dimensional data, aspects of their loss function are not yet fully understood. We introduce a theoretical framework that is based on a connection between VAE and generalized linear models (GLM). The equality between the activation function of a VAE and the inverse of the link function of a GLM enables us to provide a systematic generalization of the loss analysis for VAE based on the assumption that the distribution of the decoder belongs to an exponential dispersion family (EDF). As a further result, we can initialize VAE nets by maximum likelihood estimates (MLE) that enhance the training performance on both synthetic and real world data sets.

READ FULL TEXT

page 19

page 20

page 21

page 22

page 23

page 25

page 26

page 27

research
12/13/2021

Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias

Variational Autoencoders (VAEs) are one of the most commonly used genera...
research
08/02/2018

Variational Information Bottleneck on Vector Quantized Autoencoders

In this paper, we provide an information-theoretic interpretation of the...
research
07/16/2019

The continuous Bernoulli: fixing a pervasive error in variational autoencoders

Variational autoencoders (VAE) have quickly become a central tool in mac...
research
01/13/2022

Reproducible, incremental representation learning with Rosetta VAE

Variational autoencoders are among the most popular methods for distilli...
research
05/19/2022

Closing the gap: Exact maximum likelihood training of generative autoencoders using invertible layers

In this work, we provide an exact likelihood alternative to the variatio...
research
10/31/2021

PIE: Pseudo-Invertible Encoder

We consider the problem of information compression from high dimensional...
research
05/18/2020

Learning and Inference in Imaginary Noise Models

Inspired by recent developments in learning smoothed densities with empi...

Please sign up or login with your details

Forgot password? Click here to reset