DeepAI AI Chat
Log In Sign Up

A Bayesian Federated Learning Framework with Multivariate Gaussian Product

by   Liangxi Liu, et al.

Federated learning (FL) allows multiple clients to collaboratively learn a globally shared model through cycles of model aggregation and local model training without the need to share data. In this paper, we comprehensively study a new problem named aggregation error (AE), arising from the model aggregation stage on a server, which is mainly induced by the heterogeneity of the client data. Due to the large discrepancies between local models, the accompanying large AE generally results in a slow convergence and an expected reduction of accuracy for FL. In order to reduce AE, we propose a novel federated learning framework from a Bayesian perspective, in which a multivariate Gaussian product mechanism is employed to aggregate the local models. It is worth noting that the product of Gaussians is still a Gaussian. This property allows us to directly aggregate local expectations and covariances in a definitely convex form, thereby greatly reducing the AE. Accordingly, on the clients, we develop a new Federated Online Laplace Approximation (FOLA) method, which can estimate the parameters of the local posterior by repeatedly accumulating priors. Specifically, in every round, the global posterior distributed from the server can be treated as the priors, and thus the local posterior can also be effectively approximated by a Gaussian using FOLA. Experimental results on benchmarks reach state-of-the-arts performance and clearly demonstrate the advantages of the proposed method.


page 1

page 2

page 3

page 4


Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning

Federated learning (FL) is a collaborative machine learning framework th...

FedMR: Fedreated Learning via Model Recombination

As a promising privacy-preserving machine learning method, Federated Lea...

LINDT: Tackling Negative Federated Learning with Local Adaptation

Federated Learning (FL) is a promising distributed learning paradigm, wh...

Robust One Round Federated Learning with Predictive Space Bayesian Inference

Making predictions robust is an important challenge. A separate challeng...

FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning

Federated learning (FL) is a distributed learning protocol in which a se...

Probabilistic Federated Learning of Neural Networks Incorporated with Global Posterior Information

In federated learning, models trained on local clients are distilled int...

Multi-Carrier NOMA-Empowered Wireless Federated Learning with Optimal Power and Bandwidth Allocation

Wireless federated learning (WFL) undergoes a communication bottleneck i...