Exact posterior distributions of wide Bayesian neural networks

06/18/2020
by   Jiri Hron, et al.
0

Recent work has shown that the prior over functions induced by a deep Bayesian neural network (BNN) behaves as a Gaussian process (GP) as the width of all layers becomes large. However, many BNN applications are concerned with the BNN function space posterior. While some empirical evidence of the posterior convergence was provided in the original works of Neal (1996) and Matthews et al. (2018), it is limited to small datasets or architectures due to the notorious difficulty of obtaining and verifying exactness of BNN posterior approximations. We provide the missing theoretical proof that the exact BNN posterior converges (weakly) to the one induced by the GP limit of the prior. For empirical validation, we show how to generate exact samples from a finite BNN on a small dataset via rejection sampling.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/18/2023

Posterior Inference on Infinitely Wide Bayesian Neural Networks under Weights with Unbounded Variance

From the classical and influential works of Neal (1996), it is known tha...
research
11/01/2017

Deep Neural Networks as Gaussian Processes

A deep fully-connected neural network with an i.i.d. prior over its para...
research
11/29/2019

Richer priors for infinitely wide multi-layer perceptrons

It is well-known that the distribution over functions induced through a ...
research
10/17/2019

Why bigger is not always better: on finite and infinite neural networks

Recent work has shown that the outputs of convolutional neural networks ...
research
07/31/2020

Cold Posteriors and Aleatoric Uncertainty

Recent work has observed that one can outperform exact inference in Baye...
research
04/08/2019

Bayesian Neural Networks at Finite Temperature

We recapitulate the Bayesian formulation of neural network based classif...
research
08/19/2020

Neural Networks and Quantum Field Theory

We propose a theoretical understanding of neural networks in terms of Wi...

Please sign up or login with your details

Forgot password? Click here to reset