Õptimal Differentially Private Learning of Thresholds and Quasi-Concave Optimization

11/11/2022
by   Edith Cohen, et al.
0

The problem of learning threshold functions is a fundamental one in machine learning. Classical learning theory implies sample complexity of O(ξ^-1log(1/β)) (for generalization error ξ with confidence 1-β). The private version of the problem, however, is more challenging and in particular, the sample complexity must depend on the size |X| of the domain. Progress on quantifying this dependence, via lower and upper bounds, was made in a line of works over the past decade. In this paper, we finally close the gap for approximate-DP and provide a nearly tight upper bound of Õ(log^* |X|), which matches a lower bound by Alon et al (that applies even with improper learning) and improves over a prior upper bound of Õ((log^* |X|)^1.5) by Kaplan et al. We also provide matching upper and lower bounds of Θ̃(2^log^*|X|) for the additive error of private quasi-concave optimization (a related and more general problem). Our improvement is achieved via the novel Reorder-Slice-Compute paradigm for private data analysis which we believe will have further applications.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/22/2019

Privately Learning Thresholds: Closing the Exponential Gap

We study the sample complexity of learning threshold functions under the...
research
07/14/2023

Smooth Lower Bounds for Differentially Private Algorithms via Padding-and-Permuting Fingerprinting Codes

Fingerprinting arguments, first introduced by Bun, Ullman, and Vadhan (S...
research
04/23/2020

Private Query Release Assisted by Public Data

We study the problem of differentially private query release assisted by...
research
02/07/2018

Tight Lower Bounds for Locally Differentially Private Selection

We prove a tight lower bound (up to constant factors) on the sample comp...
research
02/03/2021

The Sample Complexities of Global Lipschitz Optimization

We study the problem of black-box optimization of a Lipschitz function f...
research
05/28/2021

Curse of Dimensionality in Unconstrained Private Convex ERM

We consider the lower bounds of differentially private empirical risk mi...
research
08/29/2011

Learning Valuation Functions

In this paper we study the approximate learnability of valuations common...

Please sign up or login with your details

Forgot password? Click here to reset