Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks

03/24/2015

∙

by   Guido Montufar, et al.

∙

0

∙

share

We establish upper bounds for the minimal number of hidden units for which a binary stochastic feedforward network with sigmoid activation probabilities and a single hidden layer is a universal approximator of Markov kernels. We show that each possible probabilistic assignment of the states of n output units, given the states of k≥1 input units, can be approximated arbitrarily well by a network with 2^k-1(2^n-1-1) hidden units.