Global analysis of Expectation Maximization for mixtures of two Gaussians

08/26/2016
by   Ji Xu, et al.
0

Expectation Maximization (EM) is among the most popular algorithms for estimating parameters of statistical models. However, EM, which is an iterative algorithm based on the maximum likelihood principle, is generally only guaranteed to find stationary points of the likelihood objective, and these points may be far from any maximizer. This article addresses this disconnect between the statistical principles behind EM and its algorithmic properties. Specifically, it provides a global analysis of EM for specific models in which the observations comprise an i.i.d. sample from a mixture of two Gaussians. This is achieved by (i) studying the sequence of parameters from idealized execution of EM in the infinite sample limit, and fully characterizing the limit points of the sequence in terms of the initial parameters; and then (ii) based on this convergence analysis, establishing statistical consistency (or lack thereof) for the actual sequence of parameters produced by EM.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/03/2013

An Improved EM algorithm

In this paper, we firstly give a brief introduction of expectation maxim...
research
02/23/2022

An expectation-maximization algorithm for estimating the parameters of the correlated binomial distribution

The correlated binomial (CB) distribution was proposed by Luceño (Comput...
research
04/26/2017

Estimating the coefficients of a mixture of two linear regressions by expectation maximization

We give convergence guarantees for estimating the coefficients of a symm...
research
06/30/2020

Sinkhorn EM: An Expectation-Maximization algorithm based on entropic optimal transport

We study Sinkhorn EM (sEM), a variant of the expectation maximization (E...
research
05/24/2019

A view of Estimation of Distribution Algorithms through the lens of Expectation-Maximization

We show that under mild conditions, Estimation of Distribution Algorithm...
research
03/15/2015

Statistical Estimation and Clustering of Group-invariant Orientation Parameters

We treat the problem of estimation of orientation parameters whose value...
research
06/20/2018

An Asynchronous Distributed Expectation Maximization Algorithm For Massive Data: The DEM Algorithm

The family of Expectation-Maximization (EM) algorithms provides a genera...

Please sign up or login with your details

Forgot password? Click here to reset