On the equivalence of Hopfield Networks and Boltzmann Machines

05/13/2011
by   Adriano Barra, et al.
0

A specific type of neural network, the Restricted Boltzmann Machine (RBM), is implemented for classification and feature detection in machine learning. RBM is characterized by separate layers of visible and hidden units, which are able to learn efficiently a generative model of the observed data. We study a "hybrid" version of RBM's, in which hidden units are analog and visible units are binary, and we show that thermodynamics of visible units are equivalent to those of a Hopfield network, in which the N visible units are the neurons and the P hidden units are the learned patterns. We apply the method of stochastic stability to derive the thermodynamics of the model, by considering a formal extension of this technique to the case of multiple sets of stored patterns, which may act as a benchmark for the study of correlated sets. Our results imply that simulating the dynamics of a Hopfield network, requiring the update of N neurons and the storage of N(N-1)/2 synapses, can be accomplished by a hybrid Boltzmann Machine, requiring the update of N+P neurons but the storage of only NP synapses. In addition, the well known glass transition of the Hopfield network has a counterpart in the Boltzmann Machine: It corresponds to an optimum criterion for selecting the relative sizes of the hidden and visible layers, resolving the trade-off between flexibility and generality of the model. The low storage phase of the Hopfield model corresponds to few hidden units and hence a overly constrained RBM, while the spin-glass phase (too many hidden units) corresponds to unconstrained RBM prone to overfitting of the observed data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/30/2010

Sparse Group Restricted Boltzmann Machines

Since learning is typically very slow in Boltzmann machines, there is a ...
research
04/10/2017

Unsupervised prototype learning in an associative-memory network

Unsupervised learning in a generalized Hopfield associative-memory netwo...
research
01/24/2020

Effectively Trainable Semi-Quantum Restricted Boltzmann Machine

We propose a novel quantum model for the restricted Boltzmann machine (R...
research
06/16/2023

Structural Restricted Boltzmann Machine for image denoising and classification

Restricted Boltzmann Machines are generative models that consist of a la...
research
04/20/2018

Sampling the Riemann-Theta Boltzmann Machine

We show that the visible sector probability density function of the Riem...
research
02/20/2017

Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors

Restricted Boltzmann Machines are described by the Gibbs measure of a bi...
research
12/20/2017

Riemann-Theta Boltzmann Machine

A general Boltzmann machine with continuous visible and discrete integer...

Please sign up or login with your details

Forgot password? Click here to reset