From Monte Carlo to Las Vegas: Improving Restricted Boltzmann Machine Training Through Stopping Sets

11/22/2017
by   Pedro H. P. Savarese, et al.
0

We propose a Las Vegas transformation of Markov Chain Monte Carlo (MCMC) estimators of Restricted Boltzmann Machines (RBMs). We denote our approach Markov Chain Las Vegas (MCLV). MCLV gives statistical guarantees in exchange for random running times. MCLV uses a stopping set built from the training data and has maximum number of Markov chain steps K (referred as MCLV-K). We present a MCLV-K gradient estimator (LVS-K) for RBMs and explore the correspondence and differences between LVS-K and Contrastive Divergence (CD-K), with LVS-K significantly outperforming CD-K training RBMs over the MNIST dataset, indicating MCLV to be a promising direction in learning generative models.

READ FULL TEXT
research
10/14/2019

Parallelized Training of Restricted Boltzmann Machines using Markov-Chain Monte Carlo Methods

Restricted Boltzmann Machine (RBM) is a generative stochastic neural net...
research
11/03/2022

Can RBMs be trained with zero step contrastive divergence?

Restricted Boltzmann Machines (RBMs) are probabilistic generative models...
research
03/21/2019

Generative Models For Deep Learning with Very Scarce Data

The goal of this paper is to deal with a data scarcity scenario where de...
research
07/24/2015

A Neighbourhood-Based Stopping Criterion for Contrastive Divergence Learning

Restricted Boltzmann Machines (RBMs) are general unsupervised learning d...
research
02/25/2012

Training Restricted Boltzmann Machines on Word Observations

The restricted Boltzmann machine (RBM) is a flexible tool for modeling c...
research
02/13/2013

Bayesian Learning of Loglinear Models for Neural Connectivity

This paper presents a Bayesian approach to learning the connectivity str...
research
10/03/2019

Efficient training of energy-based models via spin-glass control

We present an efficient method for unsupervised learning using Boltzmann...

Please sign up or login with your details

Forgot password? Click here to reset