Multi-Message Shuffled Privacy in Federated Learning

02/22/2023
by   Antonious M. Girgis, et al.
0

We study differentially private distributed optimization under communication constraints. A server using SGD for optimization aggregates the client-side local gradients for model updates using distributed mean estimation (DME). We develop a communication-efficient private DME, using the recently developed multi-message shuffled (MMS) privacy framework. We analyze our proposed DME scheme to show that it achieves the order-optimal privacy-communication-performance tradeoff resolving an open question in [1], whether the shuffled models can improve the tradeoff obtained in Secure Aggregation. This also resolves an open question on the optimal trade-off for private vector sum in the MMS model. We achieve it through a novel privacy mechanism that non-uniformly allocates privacy at different resolutions of the local gradient vectors. These results are directly applied to give guarantees on private distributed learning algorithms using this for private gradient aggregation iteratively. We also numerically evaluate the private DME algorithms.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/08/2022

The Interpolated MVU Mechanism For Communication-efficient Private Federated Learning

We consider private federated learning (FL), where a server aggregates d...
research
02/22/2022

Differential Secrecy for Distributed Data and Applications to Robust Differentially Secure Vector Summation

Computing the noisy sum of real-valued vectors is an important primitive...
research
06/08/2021

Private Multi-Group Aggregation

We study the differentially private multi group aggregation (PMGA) probl...
research
10/15/2022

Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerability

Sketching is one of the most fundamental tools in large-scale machine le...
research
07/11/2018

Differentially-Private "Draw and Discard" Machine Learning

In this work, we propose a novel framework for privacy-preserving client...
research
04/10/2023

Differentially Private Numerical Vector Analyses in the Local and Shuffle Model

Numerical vector aggregation plays a crucial role in privacy-sensitive a...
research
02/28/2023

Collaborative Mean Estimation over Intermittently Connected Networks with Peer-To-Peer Privacy

This work considers the problem of Distributed Mean Estimation (DME) ove...

Please sign up or login with your details

Forgot password? Click here to reset