Lie Group Auto-Encoder

01/28/2019
by   Liyu Gong, et al.
0

In this paper, we propose an auto-encoder based generative neural network model whose encoder compresses the inputs into vectors in the tangent space of a special Lie group manifold: upper triangular positive definite affine transform matrices (UTDATs). UTDATs are representations of Gaussian distributions and can straightforwardly generate Gaussian distributed samples. Therefore, the encoder is trained together with a decoder (generator) which takes Gaussian distributed latent vectors as input. Compared with related generative models such as variational auto-encoder, the proposed model incorporates the information on geometric properties of Gaussian distributions. As a special case, we derive an exponential mapping layer for diagonal Gaussian UTDATs which eliminates matrix exponential operator compared with general exponential mapping in Lie group theory. Moreover, we derive an intrinsic loss for UTDAT Lie group which can be calculated as l-2 loss in the tangent space. Furthermore, inspired by the Lie group theory, we propose to use the Lie algebra vectors rather than the raw parameters (e.g. mean) of Gaussian distributions as compressed representations of original inputs. Experimental results verity the effectiveness of the proposed new generative model and the benefits gained from the Lie group structural information of UTDATs.

READ FULL TEXT

page 8

page 9

research
11/12/2018

Gaussian Auto-Encoder

Evaluating distance between sample distribution and the wanted one, usua...
research
04/03/2013

Lie Algebrized Gaussians for Image Representation

We present an image representation method which is derived from analyzin...
research
09/30/2022

GM-VAE: Representation Learning with VAE on Gaussian Manifold

We propose a Gaussian manifold variational auto-encoder (GM-VAE) whose l...
research
09/12/2017

Transform Invariant Auto-encoder

The auto-encoder method is a type of dimensionality reduction method. A ...
research
05/18/2017

Spatial Variational Auto-Encoding via Matrix-Variate Normal Distributions

The key idea of variational auto-encoders (VAEs) resembles that of tradi...
research
02/10/2016

A Theory of Generative ConvNet

We show that a generative random field model, which we call generative C...
research
12/07/2020

The Neural Coding Framework for Learning Generative Models

Neural generative models can be used to learn complex probability distri...

Please sign up or login with your details

Forgot password? Click here to reset