Empirical Risk Minimization in the Non-interactive Local Model of Differential Privacy

by   Di Wang, et al.

In this paper, we study the Empirical Risk Minimization (ERM) problem in the non-interactive Local Differential Privacy (LDP) model. Previous research on this problem <cit.> indicates that the sample complexity, to achieve error α, needs to be exponentially depending on the dimensionality p for general loss functions. In this paper, we make two attempts to resolve this issue by investigating conditions on the loss functions that allow us to remove such a limit. In our first attempt, we show that if the loss function is (∞, T)-smooth, by using the Bernstein polynomial approximation we can avoid the exponential dependency in the term of α. We then propose player-efficient algorithms with 1-bit communication complexity and O(1) computation cost for each player. The error bound of these algorithms is asymptotically the same as the original one. With some additional assumptions, we also give an algorithm which is more efficient for the server. In our second attempt, we show that for any 1-Lipschitz generalized linear convex loss function, there is an (ϵ, δ)-LDP algorithm whose sample complexity for achieving error α is only linear in the dimensionality p. Our results use a polynomial of inner product approximation technique. Finally, motivated by the idea of using polynomial approximation and based on different types of polynomial approximations, we propose (efficient) non-interactive locally differentially private algorithms for learning the set of k-way marginal queries and the set of smooth queries.


page 1

page 2

page 3

page 4


Efficient Empirical Risk Minimization with Smooth Loss Functions in Non-interactive Local Differential Privacy

In this paper, we study the Empirical Risk Minimization problem in the n...

Differentially Private Empirical Risk Minimization in Non-interactive Local Model via Polynomial of Inner Product Approximation

In this paper, we study the Empirical Risk Minimization problem in the n...

Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds

In this paper, we initiate a systematic investigation of differentially ...

Estimating Smooth GLM in Non-interactive Local Differential Privacy Model with Public Unlabeled Data

In this paper, we study the problem of estimating smooth Generalized Lin...

Private Empirical Risk Minimization Beyond the Worst Case: The Effect of the Constraint Set Geometry

Empirical Risk Minimization (ERM) is a standard technique in machine lea...

Interaction is necessary for distributed learning with privacy or communication constraints

Local differential privacy (LDP) is a model where users send privatized ...

A Convex Surrogate Operator for General Non-Modular Loss Functions

Empirical risk minimization frequently employs convex surrogates to unde...

Please sign up or login with your details

Forgot password? Click here to reset