Batch norm with entropic regularization turns deterministic autoencoders into generative models

02/25/2020
by   Amur Ghose, et al.
16

The variational autoencoder is a well defined deep generative model that utilizes an encoder-decoder framework where an encoding neural network outputs a non-deterministic code for reconstructing an input. The encoder achieves this by sampling from a distribution for every input, instead of outputting a deterministic code per input. The great advantage of this process is that it allows the use of the network as a generative model for sampling from the data distribution beyond provided samples for training. We show in this work that utilizing batch normalization as a source for non-determinism suffices to turn deterministic autoencoders into generative models on par with variational ones, so long as we add a suitable entropic regularization to the training objective.

READ FULL TEXT VIEW PDF

Authors

page 7

page 8

page 9

page 13

page 14

06/05/2018

Generative Reversible Networks

Generative models with an encoding component such as autoencoders curren...
02/09/2016

Discriminative Regularization for Generative Models

We explore the question of whether the representations learned by classi...
07/09/2021

The Effects of Invertibility on the Representational Complexity of Encoders in Variational Autoencoders

Training and using modern neural-network based latent-variable generativ...
09/28/2016

Variational Autoencoder for Deep Learning of Images, Labels and Captions

A novel variational autoencoder is developed to model images, as well as...
10/13/2021

The deep generative decoder: Using MAP estimates of representations

A deep generative model is characterized by a representation space, its ...
06/05/2018

Training Generative Reversible Networks

Generative models with an encoding component such as autoencoders curren...
03/29/2019

From Variational to Deterministic Autoencoders

Variational Autoencoders (VAEs) provide a theoretically-backed framework...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction