Clustering a Mixture of Gaussians with Unknown Covariance

10/04/2021
by   Damek Davis, et al.
0

We investigate a clustering problem with data from a mixture of Gaussians that share a common but unknown, and potentially ill-conditioned, covariance matrix. We start by considering Gaussian mixtures with two equally-sized components and derive a Max-Cut integer program based on maximum likelihood estimation. We prove its solutions achieve the optimal misclassification rate when the number of samples grows linearly in the dimension, up to a logarithmic factor. However, solving the Max-cut problem appears to be computationally intractable. To overcome this, we develop an efficient spectral algorithm that attains the optimal rate but requires a quadratic sample size. Although this sample complexity is worse than that of the Max-cut problem, we conjecture that no polynomial-time method can perform better. Furthermore, we gather numerical and theoretical evidence that supports the existence of a statistical-computational gap. Finally, we generalize the Max-Cut program to a k-means program that handles multi-component mixtures with possibly unequal weights. It enjoys similar optimality guarantees for mixtures of distributions that satisfy a transportation-cost inequality, encompassing Gaussian and strongly log-concave distributions.

READ FULL TEXT
research
09/01/2016

Ten Steps of EM Suffice for Mixtures of Two Gaussians

The Expectation-Maximization (EM) algorithm is a widely used method for ...
research
03/22/2020

Efficient Clustering for Stretched Mixtures: Landscape and Optimality

This paper considers a canonical clustering problem where one receives u...
research
05/23/2022

Beyond EM Algorithm on Over-specified Two-Component Location-Scale Gaussian Mixtures

The Expectation-Maximization (EM) algorithm has been predominantly used ...
research
01/01/2023

Semidefinite programming on population clustering: a global analysis

In this paper, we consider the problem of partitioning a small data samp...
research
12/10/2021

Beyond Parallel Pancakes: Quasi-Polynomial Time Guarantees for Non-Spherical Gaussian Mixtures

We consider mixtures of k≥ 2 Gaussian components with unknown means and ...
research
06/25/2019

Spectral Properties of Radial Kernels and Clustering in High Dimensions

In this paper, we study the spectrum and the eigenvectors of radial kern...
research
10/28/2019

Testing Equivalence of Clustering

In this paper, we test whether two datasets share a common clustering st...

Please sign up or login with your details

Forgot password? Click here to reset