Log In Sign Up

Õptimal Differentially Private Learning of Thresholds and Quasi-Concave Optimization

by   Edith Cohen, et al.

The problem of learning threshold functions is a fundamental one in machine learning. Classical learning theory implies sample complexity of O(ξ^-1log(1/β)) (for generalization error ξ with confidence 1-β). The private version of the problem, however, is more challenging and in particular, the sample complexity must depend on the size |X| of the domain. Progress on quantifying this dependence, via lower and upper bounds, was made in a line of works over the past decade. In this paper, we finally close the gap for approximate-DP and provide a nearly tight upper bound of Õ(log^* |X|), which matches a lower bound by Alon et al (that applies even with improper learning) and improves over a prior upper bound of Õ((log^* |X|)^1.5) by Kaplan et al. We also provide matching upper and lower bounds of Θ̃(2^log^*|X|) for the additive error of private quasi-concave optimization (a related and more general problem). Our improvement is achieved via the novel Reorder-Slice-Compute paradigm for private data analysis which we believe will have further applications.


page 1

page 2

page 3

page 4


Privately Learning Thresholds: Closing the Exponential Gap

We study the sample complexity of learning threshold functions under the...

Private Query Release Assisted by Public Data

We study the problem of differentially private query release assisted by...

Tight Lower Bounds for Locally Differentially Private Selection

We prove a tight lower bound (up to constant factors) on the sample comp...

The Sample Complexities of Global Lipschitz Optimization

We study the problem of black-box optimization of a Lipschitz function f...

Private Learning and Sanitization: Pure vs. Approximate Differential Privacy

We compare the sample complexity of private learning [Kasiviswanathan et...

Curse of Dimensionality in Unconstrained Private Convex ERM

We consider the lower bounds of differentially private empirical risk mi...

Learning-based Support Estimation in Sublinear Time

We consider the problem of estimating the number of distinct elements in...