Almost Tight Error Bounds on Differentially Private Continual Counting

11/09/2022
by   Monika Henzinger, et al.
0

The first large-scale deployment of private federated learning uses differentially private counting in the continual release model as a subroutine (Google AI blog titled "Federated Learning with Formal Differential Privacy Guarantees"). In this case, a concrete bound on the error is very relevant to reduce the privacy parameter. The standard mechanism for continual counting is the binary mechanism. We present a novel mechanism and show that its mean squared error is both asymptotically optimal and a factor 10 smaller than the error of the binary mechanism. We also show that the constants in our analysis are almost tight by giving non-asymptotic lower and upper bounds that differ only in the constants of lower-order terms. Our algorithm is a matrix mechanism for the counting matrix and takes constant time per release. We also use our explicit factorization of the counting matrix to give an upper bound on the excess risk of the private learning algorithm of Denisov et al. (NeurIPS 2022). Our lower bound for any continual counting mechanism is the first tight lower bound on continual counting under approximate differential privacy. It is achieved using a new lower bound on a certain factorization norm, denoted by γ_F(·), in terms of the singular values of the matrix. In particular, we show that for any complex matrix, A ∈ℂ^m × n, γ_F(A) ≥1/√(m)A_1, where · denotes the Schatten-1 norm. We believe this technique will be useful in proving lower bounds for a larger class of linear queries. To illustrate the power of this technique, we show the first lower bound on the mean squared error for answering parity queries.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/18/2023

A Unifying Framework for Differentially Private Sums under Continual Observation

We study the problem of maintaining a differentially private decaying su...
research
12/01/2021

The Price of Differential Privacy under Continual Observation

We study the accuracy of differentially private mechanisms in the contin...
research
02/23/2022

Constant matters: Fine-grained Complexity of Differentially Private Continual Observation

We study fine-grained error bounds for differentially private algorithms...
research
06/17/2023

Differentially Private Histogram, Predecessor, and Set Cardinality under Continual Observation

Differential privacy is the de-facto privacy standard in data analysis. ...
research
10/04/2020

Privately Answering Counting Queries with Generalized Gaussian Mechanisms

We consider the problem of answering k counting (i.e. sensitivity-1) que...
research
06/12/2020

Tight Approximate Differential Privacy for Discrete-Valued Mechanisms Using FFT

We propose a numerical accountant for evaluating the tight (ε,δ)-privacy...
research
11/24/2019

Four accuracy bounds and one estimator for frequency estimation under local differential privacy

We present four lower bounds on the mean squared error of both frequency...

Please sign up or login with your details

Forgot password? Click here to reset