Three Learning Stages and Accuracy-Efficiency Tradeoff of Restricted Boltzmann Machines

09/02/2022
by   Lennart Dabelow, et al.
3

Restricted Boltzmann Machines (RBMs) offer a versatile architecture for unsupervised machine learning that can in principle approximate any target probability distribution with arbitrary accuracy. However, the RBM model is usually not directly accessible due to its computational complexity, and Markov-chain sampling is invoked to analyze the learned probability distribution. For training and eventual applications, it is thus desirable to have a sampler that is both accurate and efficient. We highlight that these two goals generally compete with each other and cannot be achieved simultaneously. More specifically, we identify and quantitatively characterize three regimes of RBM learning: independent learning, where the accuracy improves without losing efficiency; correlation learning, where higher accuracy entails lower efficiency; and degradation, where both accuracy and efficiency no longer improve or even deteriorate. These findings are based on numerical experiments and heuristic arguments.

READ FULL TEXT

page 5

page 7

page 20

page 21

research
12/21/2013

Relaxations for inference in restricted Boltzmann machines

We propose a relaxation-based approximate inference algorithm that sampl...
research
05/21/2018

Transductive Boltzmann Machines

We present transductive Boltzmann machines (TBMs), which firstly achieve...
research
04/23/2018

Boltzmann Encoded Adversarial Machines

Restricted Boltzmann Machines (RBMs) are a class of generative neural ne...
research
02/28/2017

Can Boltzmann Machines Discover Cluster Updates ?

Boltzmann machines are physics informed generative models with wide appl...
research
01/15/2020

Mode-Assisted Unsupervised Learning of Restricted Boltzmann Machines

Restricted Boltzmann machines (RBMs) are a powerful class of generative ...
research
02/25/2012

Training Restricted Boltzmann Machines on Word Observations

The restricted Boltzmann machine (RBM) is a flexible tool for modeling c...
research
12/18/2013

On the Challenges of Physical Implementations of RBMs

Restricted Boltzmann machines (RBMs) are powerful machine learning model...

Please sign up or login with your details

Forgot password? Click here to reset