Federated Two-stage Learning with Sign-based Voting

12/10/2021
by   Zichen Ma, et al.
0

Federated learning is a distributed machine learning mechanism where local devices collaboratively train a shared global model under the orchestration of a central server, while keeping all private data decentralized. In the system, model parameters and its updates are transmitted instead of raw data, and thus the communication bottleneck has become a key challenge. Besides, recent larger and deeper machine learning models also pose more difficulties in deploying them in a federated environment. In this paper, we design a federated two-stage learning framework that augments prototypical federated learning with a cut layer on devices and uses sign-based stochastic gradient descent with the majority vote method on model updates. Cut layer on devices learns informative and low-dimension representations of raw data locally, which helps reduce global model parameters and prevents data leakage. Sign-based SGD with the majority vote method for model updates also helps alleviate communication limitations. Empirically, we show that our system is an efficient and privacy preserving federated learning scheme and suits for general application scenarios.

READ FULL TEXT
research
04/14/2021

Privacy-preserving Federated Learning based on Multi-key Homomorphic Encryption

With the advance of machine learning and the internet of things (IoT), s...
research
01/06/2020

Think Locally, Act Globally: Federated Learning with Local and Global Representations

Federated learning is an emerging research paradigm to train models on p...
research
05/05/2020

Information-Theoretic Bounds on the Generalization Error and Privacy Leakage in Federated Learning

Machine learning algorithms operating on mobile networks can be characte...
research
06/05/2023

Over-the-Air Federated Learning in Satellite systems

Federated learning in satellites offers several advantages. Firstly, it ...
research
03/24/2020

FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selection

As massive data are produced from small gadgets, federated learning on m...
research
02/06/2023

z-SignFedAvg: A Unified Stochastic Sign-based Compression for Federated Learning

Federated Learning (FL) is a promising privacy-preserving distributed le...
research
07/09/2020

Federated Learning of User Authentication Models

Machine learning-based User Authentication (UA) models have been widely ...

Please sign up or login with your details

Forgot password? Click here to reset