Log In Sign Up

Differentially Private (Gradient) Expectation Maximization Algorithm with Statistical Guarantees

by   Di Wang, et al.

(Gradient) Expectation Maximization (EM) is a widely used algorithm for estimating the maximum likelihood of mixture models or incomplete data problems. A major challenge facing this popular technique is how to effectively preserve the privacy of sensitive data. Previous research on this problem has already lead to the discovery of some Differentially Private (DP) algorithms for (Gradient) EM. However, unlike in the non-private case, existing techniques are not yet able to provide finite sample statistical guarantees. To address this issue, we propose in this paper the first DP version of (Gradient) EM algorithm with statistical guarantees. Moreover, we apply our general framework to three canonical models: Gaussian Mixture Model (GMM), Mixture of Regressions Model (MRM) and Linear Regression with Missing Covariates (RMC). Specifically, for GMM in the DP model, our estimation error is near optimal in some cases. For the other two models, we provide the first finite sample statistical guarantees. Our theory is supported by thorough numerical experiments.


page 1

page 2

page 3

page 4


DP-EM: Differentially Private Expectation Maximization

The iterative nature of the expectation maximization (EM) algorithm pres...

Statistical guarantees for the EM algorithm: From population to sample-based analysis

We develop a general framework for proving rigorous guarantees on the pe...

Robust High Dimensional Expectation Maximization Algorithm via Trimmed Hard Thresholding

In this paper, we study the problem of estimating latent variable models...

Edge differentially private estimation in the β-model via jittering and method of moments

A standing challenge in data privacy is the trade-off between the level ...

Near Optimal Private and Robust Linear Regression

We study the canonical statistical estimation problem of linear regressi...

Regularized EM Algorithms: A Unified Framework and Statistical Guarantees

Latent variable models are a fundamental modeling tool in machine learni...

Noise Benefits in Expectation-Maximization Algorithms

This dissertation shows that careful injection of noise into sample data...