Confined Gradient Descent: Privacy-preserving Optimization for Federated Learning

04/27/2021
by   Yanjun Zhang, et al.
0

Federated learning enables multiple participants to collaboratively train a model without aggregating the training data. Although the training data are kept within each participant and the local gradients can be securely synthesized, recent studies have shown that such privacy protection is insufficient. The global model parameters that have to be shared for optimization are susceptible to leak information about training data. In this work, we propose Confined Gradient Descent (CGD) that enhances privacy of federated learning by eliminating the sharing of global model parameters. CGD exploits the fact that a gradient descent optimization can start with a set of discrete points and converges to another set at the neighborhood of the global minimum of the objective function. It lets the participants independently train on their local data, and securely share the sum of local gradients to benefit each other. We formally demonstrate CGD's privacy enhancement over traditional FL. We prove that less information is exposed in CGD compared to that of traditional FL. CGD also guarantees desired model accuracy. We theoretically establish a convergence rate for CGD. We prove that the loss of the proprietary models learned for each participant against a model learned by aggregated training data is bounded. Extensive experimental results on two real-world datasets demonstrate the performance of CGD is comparable with the centralized learning, with marginal differences on validation loss (mostly within 0.05) and accuracy (mostly within 1

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

09/18/2020

Federated Learning with Nesterov Accelerated Gradient Momentum Method

Federated learning (FL) is a fast-developing technique that allows multi...
12/17/2021

From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimization

In the setting of federated optimization, where a global model is aggreg...
02/03/2022

Comparative assessment of federated and centralized machine learning

Federated Learning (FL) is a privacy preserving machine learning scheme,...
10/07/2021

Neural Tangent Kernel Empowered Federated Learning

Federated learning (FL) is a privacy-preserving paradigm where multiple ...
03/05/2022

Acceleration of Federated Learning with Alleviated Forgetting in Local Training

Federated learning (FL) enables distributed optimization of machine lear...
12/01/2020

MYSTIKO : : Cloud-Mediated, Private, Federated Gradient Descent

Federated learning enables multiple, distributed participants (potential...
09/16/2020

FedSmart: An Auto Updating Federated Learning Optimization Mechanism

Federated learning has made an important contribution to data privacy-pr...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.