Learning discrete distributions: user vs item-level privacy

07/27/2020
by   Yuhan Liu, et al.
5

Much of the literature on differential privacy focuses on item-level privacy, where loosely speaking, the goal is to provide privacy per item or training example. However, recently many practical applications such as federated learning require preserving privacy for all items of a single user, which is much harder to achieve. Therefore understanding the theoretical limit of user-level privacy becomes crucial. We study the fundamental problem of learning discrete distributions over k symbols with user-level differential privacy. If each user has m samples, we show that straightforward applications of Laplace or Gaussian mechanisms require the number of users to be 𝒪(k/(mα^2) + k/ϵα) to achieve an ℓ_1 distance of α between the true and estimated distributions, with the privacy-induced penalty k/ϵα independent of the number of samples per user m. Moreover, we show that any mechanism that only operates on the final aggregate should require a user complexity of the same order. We then propose a mechanism such that the number of users scales as 𝒪̃(k/(mα^2) + k/√(m)ϵα) and further show that it is nearly-optimal under certain regimes. Thus the privacy penalty is 𝒪(√(m)) times smaller compared to the standard mechanisms. We also propose general techniques for obtaining lower bounds on restricted differentially private estimators and a lower bound on the total variation between binomial distributions, both of which might be of independent interest.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/03/2018

Differentially Private Obfuscation Mechanisms for Hiding Probability Distributions

We propose a formal model for the privacy of user attributes in terms of...
research
02/14/2022

Robust Estimation of Discrete Distributions under Local Differential Privacy

Although robust learning and local differential privacy are both widely ...
research
10/22/2021

Tight and Robust Private Mean Estimation with Few Users

In this work, we study high-dimensional mean estimation under user-level...
research
10/13/2021

Infinitely Divisible Noise in the Low Privacy Regime

Federated learning, in which training data is distributed among users an...
research
06/07/2022

Histogram Estimation under User-level Privacy with Heterogeneous Data

We study the problem of histogram estimation under user-level differenti...
research
12/02/2022

Integer Subspace Differential Privacy

We propose new differential privacy solutions for when external invarian...
research
02/22/2020

Differentially Private Set Union

We study the basic operation of set union in the global model of differe...

Please sign up or login with your details

Forgot password? Click here to reset