Hierarchical Models as Marginals of Hierarchical Models

08/14/2015
by   Guido Montufar, et al.
0

We investigate the representation of hierarchical models in terms of marginals of other hierarchical models with smaller interactions. We focus on binary variables and marginals of pairwise interaction models whose hidden variables are conditionally independent given the visible variables. In this case the problem is equivalent to the representation of linear subspaces of polynomials by feedforward neural networks with soft-plus computational units. We show that every hidden variable can freely model multiple interactions among the visible variables, which allows us to generalize and improve previous results. In particular, we show that a restricted Boltzmann machine with less than [ 2((v)+1) / (v+1) ] 2^v-1 hidden binary variables can approximate every distribution of v visible binary variables arbitrarily well, compared to 2^v-1-1 from the best previously known result.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/15/2013

Discrete Restricted Boltzmann Machines

We describe discrete restricted Boltzmann machines: probabilistic graphi...
research
10/23/2019

A Hierarchical Mixture Density Network

The relationship among three correlated variables could be very sophisti...
research
11/10/2015

Dimension of Marginals of Kronecker Product Models

A Kronecker product model is the set of visible marginal probability dis...
research
01/27/2021

On the mapping between Hopfield networks and Restricted Boltzmann Machines

Hopfield networks (HNs) and Restricted Boltzmann Machines (RBMs) are two...
research
10/24/2022

Occam learning

We discuss probabilistic neural network models for unsupervised learning...
research
06/16/2023

Structural Restricted Boltzmann Machine for image denoising and classification

Restricted Boltzmann Machines are generative models that consist of a la...
research
12/05/2018

Self-learning Local Supervision Encoding Framework to Constrict and Disperse Feature Distribution for Clustering

To obtain suitable feature distribution is a difficult task in machine l...

Please sign up or login with your details

Forgot password? Click here to reset