Optimal estimation of high-dimensional Gaussian mixtures

by   Natalie Doss, et al.

This paper studies the optimal rate of estimation in a finite Gaussian location mixture model in high dimensions without separation conditions. We assume that the number of components k is bounded and that the centers lie in a ball of bounded radius, while allowing the dimension d to be as large as the sample size n. Extending the one-dimensional result of Heinrich and Kahn <cit.>, we show that the minimax rate of estimating the mixing distribution in Wasserstein distance is Θ((d/n)^1/4 + n^-1/(4k-2)), achieved by an estimator computable in time O(nd^2+n^5/4). Furthermore, we show that the mixture density can be estimated at the optimal parametric rate Θ(√(d/n)) in Hellinger distance; however, no computationally efficient algorithm is known to achieve the optimal rate. Both the theoretical and methodological development rely on a careful application of the method of moments. Central to our results is the observation that the information geometry of finite Gaussian mixtures is characterized by the moment tensors of the mixing distribution, whose low-rank structure can be exploited to obtain a sharp local entropy bound.



page 1

page 2

page 3

page 4


Optimal Bayesian estimation of Gaussian mixtures with growing number of components

We study posterior concentration properties of Bayesian procedures for e...

Minimax Theory for High-dimensional Gaussian Mixtures with Sparse Mean Separation

While several papers have investigated computationally and statistically...

Minimax bounds for estimating multivariate Gaussian location mixtures

We prove minimax bounds for estimating Gaussian location mixtures on ℝ^d...

Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models

Introduced by Kiefer and Wolfowitz <cit.>, the nonparametric maximum lik...

Identifiability and optimal rates of convergence for parameters of multiple types in finite mixtures

This paper studies identifiability and convergence behaviors for paramet...

Learning Mixtures of Spherical Gaussians via Fourier Analysis

Suppose that we are given independent, identically distributed samples x...

Sharp Statistical Guarantees for Adversarially Robust Gaussian Classification

Adversarial robustness has become a fundamental requirement in modern ma...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.