Self-regularizing restricted Boltzmann machines

12/09/2019
by   Orestis Loukas, et al.
13

Focusing on the grand-canonical extension of the ordinary restricted Boltzmann machine, we suggest an energy-based model for feature extraction that uses a layer of hidden units with varying size. By an appropriate choice of the chemical potential and given a sufficiently large number of hidden resources the generative model is able to efficiently deduce the optimal number of hidden units required to learn the target data with exceedingly small generalization error. The formal simplicity of the grand-canonical ensemble combined with a rapidly converging ansatz in mean-field theory enable us to recycle well-established numerical algothhtims during training, like contrastive divergence, with only minor changes. As a proof of principle and to demonstrate the novel features of grand-canonical Boltzmann machines, we train our generative models on data from the Ising theory and MNIST.

READ FULL TEXT

page 16

page 21

page 22

research
11/30/2018

Restricted Boltzmann Machine with Multivalued Hidden Variables: a model suppressing over-fitting

Generalization is one of the most important issues in machine learning p...
research
06/09/2015

Training Restricted Boltzmann Machines via the Thouless-Anderson-Palmer Free Energy

Restricted Boltzmann machines are undirected neural networks which have ...
research
06/24/2015

A Novel Feature Extraction Method for Scene Recognition Based on Centered Convolutional Restricted Boltzmann Machines

Scene recognition is an important research topic in computer vision, whi...
research
02/09/2015

An Infinite Restricted Boltzmann Machine

We present a mathematical construction for the restricted Boltzmann mach...
research
09/11/2017

On better training the infinite restricted Boltzmann machines

The infinite restricted Boltzmann machine (iRBM) is an extension of the ...
research
10/15/2017

Learning Infinite RBMs with Frank-Wolfe

In this work, we propose an infinite restricted Boltzmann machine (RBM),...
research
09/18/2019

Data Mapping for Restricted Boltzmann Machine

Restricted Boltzmann machine (RBM) is interpreted as a data mapping betw...

Please sign up or login with your details

Forgot password? Click here to reset