QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning

06/01/2021
by   Maxime Vono, et al.
0

Federated learning aims at conducting inference when data are decentralised and locally stored on several clients, under two main constraints: data ownership and communication overhead. In this paper, we address these issues under the Bayesian paradigm. To this end, we propose a novel Markov chain Monte Carlo algorithm coined built upon quantised versions of stochastic gradient Langevin dynamics. To improve performance in a big data regime, we introduce variance-reduced alternatives of our methodology referred to as ^⋆ and ^++. We provide both non-asymptotic and asymptotic convergence guarantees for the proposed algorithms and illustrate their benefits on several federated learning benchmarks.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

05/05/2022

Communication-Efficient Adaptive Federated Learning

Federated learning is a machine learning training paradigm that enables ...
05/03/2022

Efficient and Convergent Federated Learning

Federated learning has shown its advances over the last few years but is...
09/20/2020

Stochastic Gradient Langevin Dynamics Algorithms with Adaptive Drifts

Bayesian deep learning offers a principled way to address many issues co...
11/29/2021

SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learning

Efficient federated learning is one of the key challenges for training a...
12/09/2020

Accurate and Fast Federated Learning via IID and Communication-Aware Grouping

Federated learning has emerged as a new paradigm of collaborative machin...
11/27/2021

Resource-Aware Asynchronous Online Federated Learning for Nonlinear Regression

Many assumptions in the federated learning literature present a best-cas...
12/06/2020

TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecture

Federated learning has emerged as a new paradigm of collaborative machin...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.