A Decentralized Approach to Bayesian Learning

07/14/2020
by   Anjaly Parayil, et al.
19

Motivated by decentralized approaches to machine learning, we propose a collaborative Bayesian learning algorithm taking the form of decentralized Langevin dynamics in a non-convex setting. Our analysis show that the initial KL-divergence between the Markov Chain and the target posterior distribution is exponentially decreasing while the error contributions to the overall KL-divergence from the additive noise is decreasing in polynomial time. We further show that the polynomial-term experiences speed-up with number of agents and provide sufficient conditions on the time-varying step-sizes to guarantee convergence to the desired distribution. The performance of the proposed algorithm is evaluated on a wide variety of machine learning tasks. The empirical results show that the performance of individual agents with locally available data is on par with the centralized setting with considerable improvement in the convergence rate.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

08/18/2020

On the Convergence of Consensus Algorithms with Markovian Noise and Gradient Bias

This paper presents a finite time convergence analysis for a decentraliz...
09/23/2019

Decentralized Markov Chain Gradient Descent

Decentralized stochastic gradient method emerges as a promising solution...
07/02/2019

Minimum Power to Maintain a Nonequilibrium Distribution of a Markov Chain

Biological systems use energy to maintain non-equilibrium distributions ...
12/04/2020

Decentralized State-Dependent Markov Chain Synthesis for Swarm Guidance

This paper introduces a decentralized state-dependent Markov chain synth...
04/02/2016

Centralized and Decentralized Global Outer-synchronization of Asymmetric Recurrent Time-varying Neural Network by Data-sampling

In this paper, we discuss the outer-synchronization of the asymmetricall...
07/16/2021

Efficient Bayesian Sampling Using Normalizing Flows to Assist Markov Chain Monte Carlo Methods

Normalizing flows can generate complex target distributions and thus sho...

Code Repositories

d-ula

Decentralized Langevin Dynamics for Bayesian Learning


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.