Learning Representations by Maximizing Mutual Information in Variational Autoencoder

12/21/2019
by   Ali Lotfi Rezaabad, et al.
0

Variational autoencoders (VAE) have ushered in a new era of unsupervised learning methods for complex distributions. Although these techniques are elegant in their approach, they are typically not useful for representation learning. In this work, we propose a simple yet powerful class of VAEs that simultaneously result in meaningful learned representations. Our solution is to combine traditional VAEs with mutual information maximization, with the goal to enhance amortized inference in VAEs using Information Theoretic techniques. We call this approach InfoMax-VAE, and such an approach can significantly boost the quality of learned high-level representations. We realize this through the explicit maximization of information measures associated with the representation. Using extensive experiments on varied datasets and setups, we show that InfoMax-VAE outperforms contemporary popular approaches, including Info-VAE and β-VAE.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

05/28/2020

VMI-VAE: Variational Mutual Information Maximization Framework for VAE With Discrete and Continuous Priors

Variational Autoencoder is a scalable method for learning latent variabl...
02/16/2018

Auto-Encoding Total Correlation Explanation

Advances in unsupervised learning enable reconstruction and generation o...
05/27/2019

Wyner VAE: Joint and Conditional Generation with Succinct Common Representation Learning

A new variational autoencoder (VAE) model is proposed that learns a succ...
02/24/2020

Progressive Learning and Disentanglement of Hierarchical Representations

Learning rich representation from data is an important task for deep gen...
07/06/2021

InfoNCE is a variational autoencoder

We show that a popular self-supervised learning method, InfoNCE, is a sp...
01/25/2019

Unsupervised speech representation learning using WaveNet autoencoders

We consider the task of unsupervised extraction of meaningful latent rep...
04/07/2020

PatchVAE: Learning Local Latent Codes for Recognition

Unsupervised representation learning holds the promise of exploiting lar...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.