Learning Feature Hierarchies with Centered Deep Boltzmann Machines

03/16/2012
by   Grégoire Montavon, et al.
0

Deep Boltzmann machines are in principle powerful models for extracting the hierarchical structure of data. Unfortunately, attempts to train layers jointly (without greedy layer-wise pretraining) have been largely unsuccessful. We propose a modification of the learning algorithm that initially recenters the output of the activation functions to zero. This modification leads to a better conditioned Hessian and thus makes learning easier. We test the algorithm on real data and demonstrate that our suggestion, the centered deep Boltzmann machine, learns a hierarchy of increasingly abstract representations and a better generative model of data.

READ FULL TEXT
research
12/12/2012

Joint Training of Deep Boltzmann Machines

We introduce a new method for training deep Boltzmann machines jointly. ...
research
12/20/2013

Modeling correlations in spontaneous activity of visual cortex with centered Gaussian-binary deep Boltzmann machines

Spontaneous cortical activity -- the ongoing cortical activities in abse...
research
06/24/2015

A Novel Feature Extraction Method for Scene Recognition Based on Centered Convolutional Restricted Boltzmann Machines

Scene recognition is an important research topic in computer vision, whi...
research
03/20/2012

On Training Deep Boltzmann Machines

The deep Boltzmann machine (DBM) has been an important development in th...
research
09/26/2013

Modeling Documents with Deep Boltzmann Machines

We introduce a Deep Boltzmann Machine model suitable for modeling and ex...
research
10/14/2014

Detection of cheating by decimation algorithm

We expand the item response theory to study the case of "cheating studen...
research
05/11/2015

Soft-Deep Boltzmann Machines

We present a layered Boltzmann machine (BM) that can better exploit the ...

Please sign up or login with your details

Forgot password? Click here to reset