The Poisson binomial mechanism for secure and private federated learning

07/09/2022
by   Wei-Ning Chen, et al.
0

We introduce the Poisson Binomial mechanism (PBM), a discrete differential privacy mechanism for distributed mean estimation (DME) with applications to federated learning and analytics. We provide a tight analysis of its privacy guarantees, showing that it achieves the same privacy-accuracy trade-offs as the continuous Gaussian mechanism. Our analysis is based on a novel bound on the Rényi divergence of two Poisson binomial distributions that may be of independent interest. Unlike previous discrete DP schemes based on additive noise, our mechanism encodes local information into a parameter of the binomial distribution, and hence the output distribution is discrete with bounded support. Moreover, the support does not increase as the privacy budget ε→ 0 as in the case of additive schemes which require the addition of more noise to achieve higher privacy; on the contrary, the support becomes smaller as ε→ 0. The bounded support enables us to combine our mechanism with secure aggregation (SecAgg), a multi-party cryptographic protocol, without the need of performing modular clipping which results in an unbiased estimator of the sum of the local vectors. This in turn allows us to apply it in the private FL setting and provide an upper bound on the convergence rate of the SGD algorithm. Moreover, since the support of the output distribution becomes smaller as ε→ 0, the communication cost of our scheme decreases with the privacy constraint ε, outperforming all previous distributed DP schemes based on additive noise in the high privacy or low communication regimes.

READ FULL TEXT
research
10/11/2021

The Skellam Mechanism for Differentially Private Federated Learning

We introduce the multi-dimensional Skellam mechanism, a discrete differe...
research
02/12/2021

The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation

We consider training models on private data that is distributed across u...
research
06/20/2023

Randomized Quantization is All You Need for Differential Privacy in Federated Learning

Federated learning (FL) is a common and practical framework for learning...
research
06/01/2021

Wireless Federated Learning with Limited Communication and Differential Privacy

This paper investigates the role of dimensionality reduction in efficien...
research
07/13/2020

Privacy Amplification via Random Check-Ins

Differentially Private Stochastic Gradient Descent (DP-SGD) forms a fund...
research
05/29/2023

Federated Learning of Gboard Language Models with Differential Privacy

We train language models (LMs) with federated learning (FL) and differen...
research
06/15/2023

Private Federated Frequency Estimation: Adapting to the Hardness of the Instance

In federated frequency estimation (FFE), multiple clients work together ...

Please sign up or login with your details

Forgot password? Click here to reset