What is a Boltzmann Machine?
A Boltzmann machine is a neural network of symmetrically connected nodes that make their own decisions whether to activate. Boltzmann machines use a straightforward stochastic learning algorithm to discover “interesting” features that represent complex patterns in the database.
While this program is quite slow in networks with extensive feature detection layers, it is fast in networks with a single layer of feature detectors, called “restricted Boltzmann machines.” Multiple hidden layers can be processed and trained on efficiently by using the feature activations of one restricted Boltzmann machine as the training dataset for the next.
How are Boltzmann Machines used?
Boltzmann machines solve two separate but crucial deep learning problems:
Search queries: The weighting on each layer’s connections are fixed and represent some form of a cost function. The Boltzmann machine’s stochastic rules allow it to sample any binary state vectors that have the lowest cost function values.
Training problems: Given a set of binary data vectors, the machine must learn to predict the output vectors with high probability. The first step is to determine which layer connection weights have the lowest cost function values, relative to all the other possible binary vectors. The Boltzmann technique accomplishes this by continuously updating its own weights as each feature is processed, instead of treating the weights as a fixed value.
