DeepAI AI Chat
Log In Sign Up

The Cost of Privacy: Optimal Rates of Convergence for Parameter Estimation with Differential Privacy

by   T. Tony Cai, et al.
University of Pennsylvania

Privacy-preserving data analysis is a rising challenge in contemporary statistics, as the privacy guarantees of statistical methods are often achieved at the expense of accuracy. In this paper, we investigate the tradeoff between statistical accuracy and privacy in mean estimation and linear regression, under both the classical low-dimensional and modern high-dimensional settings. A primary focus is to establish minimax optimality for statistical estimation with the (ε,δ)-differential privacy constraint. To this end, we find that classical lower bound arguments fail to yield sharp results, and new technical tools are called for. We first develop a general lower bound argument for estimation problems with differential privacy constraints, and then apply the lower bound argument to mean estimation and linear regression. For these statistical problems, we also design computationally efficient algorithms that match the minimax lower bound up to a logarithmic factor. In particular, for the high-dimensional linear regression, a novel private iterative hard thresholding pursuit algorithm is proposed, based on a privately truncated version of stochastic gradient descent. The numerical performance of these algorithms is demonstrated by simulation studies and applications to real data containing sensitive information, for which privacy-preserving statistical methods are necessary.


Score Attack: A Lower Bound Technique for Optimal Differentially Private Learning

Achieving optimal statistical performance while ensuring the privacy of ...

Median regression with differential privacy

Median regression analysis has robustness properties which make it attra...

The Cost of Privacy in Generalized Linear Models: Algorithms and Minimax Lower Bounds

We propose differentially private algorithms for parameter estimation in...

On robustness and local differential privacy

It is of soaring demand to develop statistical analysis tools that are r...

Local differential privacy: Elbow effect in optimal density estimation and adaptation over Besov ellipsoids

We address the problem of non-parametric density estimation under the ad...

Network change point localisation under local differential privacy

Network data are ubiquitous in our daily life, containing rich but often...