The Power Spherical distribution

06/08/2020
by   Nicola De Cao, et al.
18

There is a growing interest in probabilistic models defined in hyper-spherical spaces, be it to accommodate observed data or latent structure. The von Mises-Fisher (vMF) distribution, often regarded as the Normal distribution on the hyper-sphere, is a standard modeling choice: it is an exponential family and thus enjoys important statistical results, for example, known Kullback-Leibler (KL) divergence from other vMF distributions. Sampling from a vMF distribution, however, requires a rejection sampling procedure which besides being slow poses difficulties in the context of stochastic backpropagation via the reparameterization trick. Moreover, this procedure is numerically unstable for certain vMFs, e.g., those with high concentration and/or in high dimensions. We propose a novel distribution, the Power Spherical distribution, which retains some of the important aspects of the vMF (e.g., support on the hyper-sphere, symmetry about its mean direction parameter, known KL from other vMF distributions) while addressing its main drawbacks (i.e., scalability and numerical stability). We demonstrate the stability of Power Spherical distributions with a numerical experiment and further apply it to a variational auto-encoder trained on MNIST. Code at: https://github.com/nicola-decao/power_spherical

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/21/2019

Latent Variables on Spheres for Sampling and Spherical Inference

Variational inference is a fundamental problem in Variational Auto-Encod...
research
02/25/2015

A Note on the Kullback-Leibler Divergence for the von Mises-Fisher distribution

We present a derivation of the Kullback Leibler (KL)-Divergence (also kn...
research
08/31/2018

Spherical Latent Spaces for Stable Variational Autoencoders

A hallmark of variational autoencoders (VAEs) for text processing is the...
research
07/12/2020

Fisher Auto-Encoders

It has been conjectured that the Fisher divergence is more robust to mod...
research
04/08/2019

DeepSphere: towards an equivariant graph-based spherical CNN

Spherical data is found in many applications. By modeling the discretize...
research
06/20/2019

The spherical ensemble and quasi-Monte-Carlo designs

The spherical ensemble is a well-known ensemble of N repulsive points on...
research
10/05/2020

Improving Relational Regularized Autoencoders with Spherical Sliced Fused Gromov Wasserstein

Relational regularized autoencoder (RAE) is a framework to learn the dis...

Please sign up or login with your details

Forgot password? Click here to reset