Sliced Wasserstein Distance for Learning Gaussian Mixture Models

11/15/2017
by   Soheil Kolouri, et al.
0

Gaussian mixture models (GMM) are powerful parametric tools with many applications in machine learning and computer vision. Expectation maximization (EM) is the most popular algorithm for estimating the GMM parameters. However, EM guarantees only convergence to a stationary point of the log-likelihood function, which could be arbitrarily worse than the optimal solution. Inspired by the relationship between the negative log-likelihood function and the Kullback-Leibler (KL) divergence, we propose an alternative formulation for estimating the GMM parameters using the sliced Wasserstein distance, which gives rise to a new algorithm. Specifically, we propose minimizing the sliced-Wasserstein distance between the mixture model and the data distribution with respect to the GMM parameters. In contrast to the KL-divergence, the energy landscape for the sliced-Wasserstein distance is more well-behaved and therefore more suitable for a stochastic gradient descent scheme to obtain the optimal GMM parameters. We show that our formulation results in parameter estimates that are more robust to random initializations and demonstrate that it can estimate high-dimensional data distributions more faithfully than the EM algorithm.

READ FULL TEXT

page 4

page 7

page 8

page 13

research
11/28/2021

Schema matching using Gaussian mixture models with Wasserstein distance

Gaussian mixture models find their place as a powerful tool, mostly in t...
research
01/04/2023

Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow

Gaussian mixture models form a flexible and expressive parametric family...
research
08/29/2023

Bridging Distribution Learning and Image Clustering in High-dimensional Space

Distribution learning focuses on learning the probability density functi...
research
06/12/2023

Deep Gaussian Mixture Ensembles

This work introduces a novel probabilistic deep learning technique calle...
research
12/18/2019

Gradient-based training of Gaussian Mixture Models in High-Dimensional Spaces

We present an approach for efficiently training Gaussian Mixture Models ...
research
07/13/2023

Cramer Type Distances for Learning Gaussian Mixture Models by Gradient Descent

The learning of Gaussian Mixture Models (also referred to simply as GMMs...
research
09/24/2020

A Rigorous Link Between Self-Organizing Maps and Gaussian Mixture Models

This work presents a mathematical treatment of the relation between Self...

Please sign up or login with your details

Forgot password? Click here to reset