Parallel Stochastic Gradient Markov Chain Monte Carlo for Matrix Factorisation Models

06/03/2015
by   Umut Şimşekli, et al.
0

For large matrix factorisation problems, we develop a distributed Markov Chain Monte Carlo (MCMC) method based on stochastic gradient Langevin dynamics (SGLD) that we call Parallel SGLD (PSGLD). PSGLD has very favourable scaling properties with increasing data size and is comparable in terms of computational requirements to optimisation methods based on stochastic gradient descent. PSGLD achieves high performance by exploiting the conditional independence structure of the MF models to sub-sample data in a systematic manner as to allow parallelisation and distributed computation. We provide a convergence proof of the algorithm and verify its superior performance on various architectures such as Graphics Processing Units, shared memory multi-core systems and multi-computer clusters.

READ FULL TEXT
research
09/14/2016

Relativistic Monte Carlo

Hamiltonian Monte Carlo (HMC) is a popular Markov chain Monte Carlo (MCM...
research
05/09/2018

Stochastic Modelling of Urban Structure

The building of mathematical and computer models of cities has a long hi...
research
07/06/2020

Stochastic Stein Discrepancies

Stein discrepancies (SDs) monitor convergence and non-convergence in app...
research
10/14/2019

Parallelized Training of Restricted Boltzmann Machines using Markov-Chain Monte Carlo Methods

Restricted Boltzmann Machine (RBM) is a generative stochastic neural net...
research
01/28/2014

Tempering by Subsampling

In this paper we demonstrate that tempering Markov chain Monte Carlo sam...
research
07/25/2019

Probabilistic Approximate Logic and its Implementation in the Logical Imagination Engine

In spite of the rapidly increasing number of applications of machine lea...
research
06/12/2017

Fractional Langevin Monte Carlo: Exploring Lévy Driven Stochastic Differential Equations for Markov Chain Monte Carlo

Along with the recent advances in scalable Markov Chain Monte Carlo meth...

Please sign up or login with your details

Forgot password? Click here to reset