FeO2: Federated Learning with Opt-Out Differential Privacy

10/28/2021
by   Nasser Aldaghri, et al.
0

Federated learning (FL) is an emerging privacy-preserving paradigm, where a global model is trained at a central server while keeping client data local. However, FL can still indirectly leak private client information through model updates during training. Differential privacy (DP) can be employed to provide privacy guarantees within FL, typically at the cost of degraded final trained model. In this work, we consider a heterogeneous DP setup where clients are considered private by default, but some might choose to opt out of DP. We propose a new algorithm for federated learning with opt-out DP, referred to as FeO2, along with a discussion on its advantages compared to the baselines of private and personalized FL algorithms. We prove that the server-side and client-side procedures in FeO2 are optimal for a simplified linear problem. We also analyze the incentive for opting out of DP in terms of performance gain. Through numerical experiments, we show that FeO2 provides up to 9.27% performance gain in the global model compared to the baseline DP FL for the considered datasets. Additionally, we show a gap in the average performance of personalized models between non-private and private clients of up to 3.49%, empirically illustrating an incentive for clients to opt out.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/25/2021

Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy

Providing privacy protection has been one of the primary motivations of ...
research
08/16/2022

FedPerm: Private and Robust Federated Learning by Parameter Permutation

Federated Learning (FL) is a distributed learning paradigm that enables ...
research
02/22/2023

Personalized Privacy-Preserving Framework for Cross-Silo Federated Learning

Federated learning (FL) is recently surging as a promising decentralized...
research
08/22/2021

Personalised Federated Learning: A Combinational Approach

Federated learning (FL) is a distributed machine learning approach invol...
research
07/14/2023

Population Expansion for Training Language Models with Private Federated Learning

Federated learning (FL) combined with differential privacy (DP) offers m...
research
09/26/2022

Taming Client Dropout for Distributed Differential Privacy in Federated Learning

Federated learning (FL) is increasingly deployed among multiple clients ...
research
09/23/2022

Privacy-Preserving Online Content Moderation: A Federated Learning Use Case

Users are daily exposed to a large volume of harmful content on various ...

Please sign up or login with your details

Forgot password? Click here to reset