Information-Theoretic Bounds on the Generalization Error and Privacy Leakage in Federated Learning

05/05/2020
by   Semih Yagli, et al.
0

Machine learning algorithms operating on mobile networks can be characterized into three different categories. First is the classical situation in which the end-user devices send their data to a central server where this data is used to train a model. Second is the distributed setting in which each device trains its own model and send its model parameters to a central server where these model parameters are aggregated to create one final model. Third is the federated learning setting in which, at any given time t, a certain number of active end users train with their own local data along with feedback provided by the central server and then send their newly estimated model parameters to the central server. The server, then, aggregates these new parameters, updates its own model, and feeds the updated parameters back to all the end users, continuing this process until it converges. The main objective of this work is to provide an information-theoretic framework for all of the aforementioned learning paradigms. Moreover, using the provided framework, we develop upper and lower bounds on the generalization error together with bounds on the privacy leakage in the classical, distributed and federated learning settings. Keywords: Federated Learning, Distributed Learning, Machine Learning, Model Aggregation.

READ FULL TEXT
research
02/14/2022

FLHub: a Federated Learning model sharing service

As easy-to-use deep learning libraries such as Tensorflow and Pytorch ar...
research
10/30/2022

PHY-Fed: An Information-Theoretic Secure Aggregation in Federated Learning in Wireless Communications

Federated learning (FL) is a type of distributed machine learning at the...
research
09/30/2021

Federated Learning in ASR: Not as Easy as You Think

With the growing availability of smart devices and cloud services, perso...
research
12/10/2021

Federated Two-stage Learning with Sign-based Voting

Federated learning is a distributed machine learning mechanism where loc...
research
10/11/2019

Central Server Free Federated Learning over Single-sided Trust Social Networks

Federated learning has become increasingly important for modern machine ...
research
09/24/2019

Matrix Sketching for Secure Collaborative Machine Learning

Collaborative machine learning (ML), also known as federated ML, allows ...
research
02/05/2023

Federated Privacy-preserving Collaborative Filtering for On-Device Next App Prediction

In this study, we propose a novel SeqMF model to solve the problem of pr...

Please sign up or login with your details

Forgot password? Click here to reset