GM-VAE: Representation Learning with VAE on Gaussian Manifold

09/30/2022
by   Seunghyuk Cho, et al.
0

We propose a Gaussian manifold variational auto-encoder (GM-VAE) whose latent space consists of a set of diagonal Gaussian distributions. It is known that the set of the diagonal Gaussian distributions with the Fisher information metric forms a product hyperbolic space, which we call a Gaussian manifold. To learn the VAE endowed with the Gaussian manifold, we first propose a pseudo Gaussian manifold normal distribution based on the Kullback-Leibler divergence, a local approximation of the squared Fisher-Rao distance, to define a density over the latent space. With the newly proposed distribution, we introduce geometric transformations at the last and the first of the encoder and the decoder of VAE, respectively to help the transition between the Euclidean and Gaussian manifolds. Through the empirical experiments, we show competitive generalization performance of GM-VAE against other variants of hyperbolic- and Euclidean-VAEs. Our model achieves strong numerical stability, which is a common limitation reported with previous hyperbolic-VAEs.

READ FULL TEXT
research
04/03/2018

Hyperspherical Variational Auto-Encoders

The Variational Auto-Encoder (VAE) is one of the most used unsupervised ...
research
06/12/2020

Disentangled Representation Learning and Generation with Manifold Optimization

Disentanglement is an enjoyable property in representation learning whic...
research
10/07/2019

Increasing Expressivity of a Hyperspherical VAE

Learning suitable latent representations for observed, high-dimensional ...
research
12/22/2018

Disentangling Latent Space for VAE by Label Relevant/Irrelevant Dimensions

VAE requires the standard Gaussian distribution as a prior in the latent...
research
01/28/2019

Lie Group Auto-Encoder

In this paper, we propose an auto-encoder based generative neural networ...
research
04/30/2020

Maximum likelihood estimation of the Fisher-Bingham distribution via efficient calculation of its normalizing constant

This paper proposes an efficient numerical integration formula to comput...
research
11/27/2020

Representation of 2D frame less visual space as a neural manifold and its information geometric interpretation

Representation of 2D frame less visual space as neural manifold and its ...

Please sign up or login with your details

Forgot password? Click here to reset