DeepAI AI Chat
Log In Sign Up

A Bayesian Federated Learning Framework with Multivariate Gaussian Product

02/03/2021
by   Liangxi Liu, et al.
0

Federated learning (FL) allows multiple clients to collaboratively learn a globally shared model through cycles of model aggregation and local model training without the need to share data. In this paper, we comprehensively study a new problem named aggregation error (AE), arising from the model aggregation stage on a server, which is mainly induced by the heterogeneity of the client data. Due to the large discrepancies between local models, the accompanying large AE generally results in a slow convergence and an expected reduction of accuracy for FL. In order to reduce AE, we propose a novel federated learning framework from a Bayesian perspective, in which a multivariate Gaussian product mechanism is employed to aggregate the local models. It is worth noting that the product of Gaussians is still a Gaussian. This property allows us to directly aggregate local expectations and covariances in a definitely convex form, thereby greatly reducing the AE. Accordingly, on the clients, we develop a new Federated Online Laplace Approximation (FOLA) method, which can estimate the parameters of the local posterior by repeatedly accumulating priors. Specifically, in every round, the global posterior distributed from the server can be treated as the priors, and thus the local posterior can also be effectively approximated by a Gaussian using FOLA. Experimental results on benchmarks reach state-of-the-arts performance and clearly demonstrate the advantages of the proposed method.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/19/2022

Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning

Federated learning (FL) is a collaborative machine learning framework th...
08/16/2022

FedMR: Fedreated Learning via Model Recombination

As a promising privacy-preserving machine learning method, Federated Lea...
11/23/2020

LINDT: Tackling Negative Federated Learning with Local Adaptation

Federated Learning (FL) is a promising distributed learning paradigm, wh...
06/20/2022

Robust One Round Federated Learning with Predictive Space Bayesian Inference

Making predictions robust is an important challenge. A separate challeng...
08/11/2020

FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning

Federated learning (FL) is a distributed learning protocol in which a se...
12/06/2020

Probabilistic Federated Learning of Neural Networks Incorporated with Global Posterior Information

In federated learning, models trained on local clients are distilled int...
02/13/2023

Multi-Carrier NOMA-Empowered Wireless Federated Learning with Optimal Power and Bandwidth Allocation

Wireless federated learning (WFL) undergoes a communication bottleneck i...