Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations

09/05/2019
by   Jan Stühmer, et al.
24

Recently there has been an increased interest in unsupervised learning of disentangled representations using the Variational Autoencoder (VAE) framework. Most of the existing work has focused largely on modifying the variational cost function to achieve this goal. We first show that these modifications, e.g. beta-VAE, simplify the tendency of variational inference to underfit causing pathological over-pruning and over-orthogonalization of learned components. Second we propose a complementary approach: to modify the probabilistic model with a structured latent prior. This prior allows to discover latent variable representations that are structured into a hierarchy of independent vector spaces. The proposed prior has three major advantages: First, in contrast to the standard VAE normal prior the proposed prior is not rotationally invariant. This resolves the problem of unidentifiability of the standard VAE normal prior. Second, we demonstrate that the proposed prior encourages a disentangled latent representation which facilitates learning of disentangled representations. Third, extensive quantitative experiments demonstrate that the prior significantly mitigates the trade-off between reconstruction loss and disentanglement over the state of the art.

READ FULL TEXT

page 13

page 15

page 16

page 17

page 18

page 19

page 20

page 21

research
07/26/2023

Learning Disentangled Discrete Representations

Recent successes in image generation, model-based reinforcement learning...
research
12/11/2019

A Closer Look at Disentangling in β-VAE

In many data analysis tasks, it is beneficial to learn representations w...
research
01/08/2020

Disentangling Representations using Gaussian Processes in Variational Autoencoders for Video Prediction

We introduce MGP-VAE, a variational autoencoder which uses Gaussian proc...
research
11/02/2017

Variational Inference of Disentangled Latent Concepts from Unlabeled Observations

Disentangled representations, where the higher level data generative fac...
research
07/19/2023

Impact of Disentanglement on Pruning Neural Networks

Deploying deep learning neural networks on edge devices, to accomplish t...
research
04/11/2019

Variational AutoEncoder For Regression: Application to Brain Aging Analysis

While unsupervised variational autoencoders (VAE) have become a powerful...
research
05/24/2017

Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations

We would like to learn a representation of the data which decomposes an ...

Please sign up or login with your details

Forgot password? Click here to reset