The EM Algorithm is Adaptively-Optimal for Unbalanced Symmetric Gaussian Mixtures
share
This paper studies the problem of estimating the means ±θ_*∈ℝ^d of a symmetric two-component Gaussian mixture δ_*· N(θ_*,I)+(1-δ_*)· N(-θ_*,I) where the weights δ_* and 1-δ_* are unequal. Assuming that δ_* is known, we show that the population version of the EM algorithm globally converges if the initial estimate has non-negative inner product with the mean of the larger weight component. This can be achieved by the trivial initialization θ_0=0. For the empirical iteration based on n samples, we show that when initialized at θ_0=0, the EM algorithm adaptively achieves the minimax error rate Õ(min{1/(1-2δ_*)√(d/n),1/θ_*√(d/n),(d/n)^1/4}) in no more than O(1/θ_*(1-2δ_*)) iterations (with high probability). We also consider the EM iteration for estimating the weight δ_*, assuming a fixed mean θ (which is possibly mismatched to θ_*). For the empirical iteration of n samples, we show that the minimax error rate Õ(1/θ_*√(d/n)) is achieved in no more than O(1/θ_*^2) iterations. These results robustify and complement recent results of Wu and Zhou obtained for the equal weights case δ_*=1/2.
READ FULL TEXT