Federated Learning with Differential Privacy: Algorithms and Performance Analysis

11/01/2019
by   Kang Wei, et al.
0

In this paper, to effectively prevent information leakage, we propose a novel framework based on the concept of differential privacy (DP), in which artificial noises are added to the parameters at the clients side before aggregating, namely, noising before model aggregation FL (NbAFL). First, we prove that the NbAFL can satisfy DP under distinct protection levels by properly adapting different variances of artificial noises. Then we develop a theoretical convergence bound of the loss function of the trained FL model in the NbAFL. Specifically, the theoretical bound reveals the following three key properties: 1) There is a tradeoff between the convergence performance and privacy protection levels, i.e., a better convergence performance leads to a lower protection level; 2) Given a fixed privacy protection level, increasing the number N of overall clients participating in FL can improve the convergence performance; 3) There is an optimal number of maximum aggregation times (communication rounds) in terms of convergence performance for a given protection level. Furthermore, we propose a K-random scheduling strategy, where K (1<K<N) clients are randomly selected from the N overall clients to participate in each aggregation. We also develop the corresponding convergence bound of the loss function in this case and the K-random scheduling strategy can also retain the above three properties. Moreover, we find that there is an optimal K that achieves the best convergence performance at a fixed privacy level. Evaluations demonstrate that our theoretical results are consistent with simulations, thereby facilitating the designs on various privacy-preserving FL algorithms with different tradeoff requirements on convergence performance and privacy levels.

READ FULL TEXT
11/01/2019

Performance Analysis on Federated Learning with Differential Privacy

In this paper, to effectively prevent the differential attack, we propos...
02/29/2020

Performance Analysis and Optimization in Privacy-Preserving Federated Learning

As a means of decentralized machine learning, federated learning (FL) ha...
01/31/2021

Gain without Pain: Offsetting DP-injected Nosies Stealthily in Cross-device Federated Learning

Federated Learning (FL) is an emerging paradigm through which decentrali...
12/02/2020

Blockchain Assisted Decentralized Federated Learning (BLADE-FL) with Lazy Clients

Federated learning (FL), as a distributed machine learning approach, has...
01/11/2021

On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times

In spite that Federated Learning (FL) is well known for its privacy prot...
04/14/2020

Differentially Private AirComp Federated Learning with Power Adaptation Harnessing Receiver Noise

Over-the-air computation (AirComp)-based federated learning (FL) enables...
03/23/2022

Contextual Model Aggregation for Fast and Robust Federated Learning in Edge Computing

Federated learning is a prime candidate for distributed machine learning...