Conditional Restricted Boltzmann Machines for Structured Output Prediction

02/14/2012
by   Volodymyr Mnih, et al.
0

Conditional Restricted Boltzmann Machines (CRBMs) are rich probabilistic models that have recently been applied to a wide range of problems, including collaborative filtering, classification, and modeling motion capture data. While much progress has been made in training non-conditional RBMs, these algorithms are not applicable to conditional models and there has been almost no work on training and generating predictions from conditional RBMs for structured output problems. We first argue that standard Contrastive Divergence-based learning may not be suitable for training CRBMs. We then identify two distinct types of structured output prediction problems and propose an improved learning algorithm for each. The first problem type is one where the output space has arbitrary structure but the set of likely output configurations is relatively small, such as in multi-label classification. The second problem is one where the output space is arbitrarily structured but where the output space variability is much greater, such as in image denoising or pixel labeling. We show that the new learning algorithms can work much better than Contrastive Divergence on both types of problems.

READ FULL TEXT
research
03/02/2017

Belief Propagation in Conditional RBMs for Structured Prediction

Restricted Boltzmann machines (RBMs) and conditional RBMs (CRBMs) are po...
research
06/09/2015

Training Restricted Boltzmann Machines via the Thouless-Anderson-Palmer Free Energy

Restricted Boltzmann machines are undirected neural networks which have ...
research
12/22/2009

Learning to Predict Combinatorial Structures

The major challenge in designing a discriminative learning algorithm for...
research
01/08/2018

Weighted Contrastive Divergence

Learning algorithms for energy based Boltzmann architectures that rely o...
research
04/28/2018

Learning from multivariate discrete sequential data using a restricted Boltzmann machine model

A restricted Boltzmann machine (RBM) is a generative neural-network mode...
research
12/19/2019

Robust Multi-Output Learning with Highly Incomplete Data via Restricted Boltzmann Machines

In a standard multi-output classification scenario, both features and la...
research
08/25/2020

Towards Structured Prediction in Bioinformatics with Deep Learning

Using machine learning, especially deep learning, to facilitate biologic...

Please sign up or login with your details

Forgot password? Click here to reset