Learning Infinite RBMs with Frank-Wolfe

10/15/2017
by   Wei Ping, et al.
0

In this work, we propose an infinite restricted Boltzmann machine (RBM), whose maximum likelihood estimation (MLE) corresponds to a constrained convex optimization. We consider the Frank-Wolfe algorithm to solve the program, which provides a sparse solution that can be interpreted as inserting a hidden unit at each iteration, so that the optimization process takes the form of a sequence of finite models of increasing complexity. As a side benefit, this can be used to easily and efficiently identify an appropriate number of hidden units during the optimization. The resulting model can also be used as an initialization for typical state-of-the-art RBM training algorithms such as contrastive divergence, leading to models with consistently higher test likelihood than random initialization.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/09/2015

An Infinite Restricted Boltzmann Machine

We present a mathematical construction for the restricted Boltzmann mach...
research
05/06/2014

Training Restricted Boltzmann Machine by Perturbation

A new approach to maximum likelihood learning of discrete graphical mode...
research
02/11/2021

Learning Gaussian-Bernoulli RBMs using Difference of Convex Functions Optimization

The Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM) is a useful...
research
12/09/2019

Self-regularizing restricted Boltzmann machines

Focusing on the grand-canonical extension of the ordinary restricted Bol...
research
06/25/2019

Certifiably Optimal Sparse Inverse Covariance Estimation

We consider the maximum likelihood estimation of sparse inverse covarian...
research
03/20/2012

On Training Deep Boltzmann Machines

The deep Boltzmann machine (DBM) has been an important development in th...
research
06/16/2021

Maximum likelihood estimation for mechanistic network models

Mechanistic network models specify the mechanisms by which networks grow...

Please sign up or login with your details

Forgot password? Click here to reset