The Cost of Privacy: Optimal Rates of Convergence for Parameter Estimation with Differential Privacy

by   T. Tony Cai, et al.

Privacy-preserving data analysis is a rising challenge in contemporary statistics, as the privacy guarantees of statistical methods are often achieved at the expense of accuracy. In this paper, we investigate the tradeoff between statistical accuracy and privacy in mean estimation and linear regression, under both the classical low-dimensional and modern high-dimensional settings. A primary focus is to establish minimax optimality for statistical estimation with the (ε,δ)-differential privacy constraint. To this end, we find that classical lower bound arguments fail to yield sharp results, and new technical tools are called for. We first develop a general lower bound argument for estimation problems with differential privacy constraints, and then apply the lower bound argument to mean estimation and linear regression. For these statistical problems, we also design computationally efficient algorithms that match the minimax lower bound up to a logarithmic factor. In particular, for the high-dimensional linear regression, a novel private iterative hard thresholding pursuit algorithm is proposed, based on a privately truncated version of stochastic gradient descent. The numerical performance of these algorithms is demonstrated by simulation studies and applications to real data containing sensitive information, for which privacy-preserving statistical methods are necessary.


Score Attack: A Lower Bound Technique for Optimal Differentially Private Learning

Achieving optimal statistical performance while ensuring the privacy of ...

Median regression with differential privacy

Median regression analysis has robustness properties which make it attra...

The Cost of Privacy in Generalized Linear Models: Algorithms and Minimax Lower Bounds

We propose differentially private algorithms for parameter estimation in...

On robustness and local differential privacy

It is of soaring demand to develop statistical analysis tools that are r...

Distributed Gaussian Mean Estimation under Communication Constraints: Optimal Rates and Communication-Efficient Algorithms

We study distributed estimation of a Gaussian mean under communication c...

Differentially private sliced inverse regression in the federated paradigm

We extend the celebrated sliced inverse regression to address the challe...

Differential privacy and robust statistics in high dimensions

We introduce a universal framework for characterizing the statistical ef...

Please sign up or login with your details

Forgot password? Click here to reset