On the Convergence of EM for truncated mixtures of two Gaussians
share
Motivated by a recent result of Daskalakis et al. DGTZ18, we analyze the population version of Expectation-Maximization (EM) algorithm for the case of truncated mixtures of two Gaussians. Truncated samples from a d-dimensional mixture of two Gaussians 1/2N(μ⃗, Σ⃗)+ 1/2N(-μ⃗, Σ⃗) means that a sample is only revealed if it falls in some subset S ⊂R^d of positive (Lebesgue) measure. We show that for d=1, EM converges almost surely (under random initialization) to the true mean (variance σ^2 is known) for any measurable set S. Moreover, for d>1 we show EM almost surely converges to the true mean μ⃗ for any measurable set S, under the assumption that the map of EM has only three fixed points, namely -μ⃗, 0⃗, μ⃗ (covariance matrix Σ⃗ is known). Our techniques deviate from those of Daskalakis et al. DTZ17, which heavily depend on symmetry that the untruncated problem exhibits. The fact that the truncated set S may not be symmetric around 0⃗ makes it impossible to compute a closed form of the update rule of EM. We circumvent this fact by using techniques from dynamical systems, in particular implicit function theorem and stability analysis around the fixed points of the update rule of EM.
READ FULL TEXT