On the Relative Expressiveness of Bayesian and Neural Networks

12/21/2018
by   Arthur Choi, et al.
0

A neural network computes a function. A central property of neural networks is that they are "universal approximators:" for a given continuous function, there exists a neural network that can approximate it arbitrarily well, given enough neurons (and some additional assumptions). In contrast, a Bayesian network is a model, but each of its queries can be viewed as computing a function. In this paper, we identify some key distinctions between the functions computed by neural networks and those by marginal Bayesian network queries, showing that the former are more expressive than the latter. Moreover, we propose a simple augmentation to Bayesian networks (a testing operator), which enables their marginal queries to become "universal approximators."

READ FULL TEXT
research
11/18/2022

Universal Property of Convolutional Neural Networks

Universal approximation, whether a set of functions can approximate an a...
research
07/06/2021

Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons

This paper develops simple feed-forward neural networks that achieve the...
research
02/10/2019

An Algorithm for Approximating Continuous Functions on Compact Subsets with a Neural Network with one Hidden Layer

George Cybenko's landmark 1989 paper showed that there exists a feedforw...
research
10/08/2019

Universal Approximation Theorems

The universal approximation theorem established the density of specific ...
research
07/12/2020

Abstract Universal Approximation for Neural Networks

With growing concerns about the safety and robustness of neural networks...
research
11/11/2022

Do Bayesian Neural Networks Need To Be Fully Stochastic?

We investigate the efficacy of treating all the parameters in a Bayesian...
research
02/05/2022

Spelunking the Deep: Guaranteed Queries on General Neural Implicit Surfaces via Range Analysis

Neural implicit representations, which encode a surface as the level set...

Please sign up or login with your details

Forgot password? Click here to reset