DeepAI AI Chat
Log In Sign Up

An Alternative to EM for Gaussian Mixture Models: Batch and Stochastic Riemannian Optimization

by   Reshad Hosseini, et al.

We consider maximum likelihood estimation for Gaussian Mixture Models (Gmms). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of key importance. We propose an alternative to EM by appealing to the rich Riemannian geometry of positive definite matrices, using which we cast Gmm parameter estimation as a Riemannian optimization problem. Surprisingly, such an out-of-the-box Riemannian formulation completely fails and proves much inferior to EM. This motivates us to take a closer look at the problem geometry, and derive a better formulation that is much more amenable to Riemannian optimization. We then develop (Riemannian) batch and stochastic gradient algorithms that outperform EM, often substantially. We provide a non-asymptotic convergence analysis for our stochastic method, which is also the first (to our knowledge) such global analysis for Riemannian stochastic gradient. Numerous empirical results are included to demonstrate the effectiveness of our methods.


page 1

page 2

page 3

page 4


Manifold Optimization for Gaussian Mixture Models

We take a new look at parameter estimation for Gaussian Mixture Models (...

Likelihood Maximization and Moment Matching in Low SNR Gaussian Mixture Models

We derive an asymptotic expansion for the log likelihood of Gaussian mix...

Stochastic First-Order Learning for Large-Scale Flexibly Tied Gaussian Mixture Model

Gaussian Mixture Models (GMM) are one of the most potent parametric dens...

Riemannian Optimization for Variance Estimation in Linear Mixed Models

Variance parameter estimation in linear mixed models is a challenge for ...

Analysis of Gradient-Based Expectation-Maximization-Like Algorithms via Integral Quadratic Constraints

The Expectation-Maximization (EM) algorithm is one of the most popular m...

Continuous Color Transfer

Color transfer, which plays a key role in image editing, has attracted n...

Geometric Mean Metric Learning

We revisit the task of learning a Euclidean metric from data. We approac...