The Ridgelet Prior: A Covariance Function Approach to Prior Specification for Bayesian Neural Networks

10/16/2020
by   Takuo Matsubara, et al.
0

Bayesian neural networks attempt to combine the strong predictive performance of neural networks with formal quantification of uncertainty associated with the predictive output in the Bayesian framework. However, it remains unclear how to endow the parameters of the network with a prior distribution that is meaningful when lifted into the output space of the network. A possible solution is proposed that enables the user to posit an appropriate covariance function for the task at hand. Our approach constructs a prior distribution for the parameters of the network, called a ridgelet prior, that approximates the posited covariance structure in the output space of the network. The approach is rooted in the ridgelet transform and we establish both finite-sample-size error bounds and the consistency of the approximation of the covariance function in a limit where the number of hidden units is increased. Our experimental assessment is limited to a proof-of-concept, where we demonstrate that the ridgelet prior can out-perform an unstructured prior on regression problems for which an informative covariance function can be a priori provided.

READ FULL TEXT

page 9

page 23

research
06/01/2022

Asymptotic Properties for Bayesian Neural Network in Besov Space

Neural networks have shown great predictive power when dealing with vari...
research
01/07/2021

Infinitely Wide Tensor Networks as Gaussian Process

Gaussian Process is a non-parametric prior which can be understood as a ...
research
12/10/2017

Finite sample Bernstein - von Mises theorems for functionals and spectral projectors of covariance matrix

We demonstrate that a prior influence on the posterior distribution of c...
research
09/05/2023

Sparse Function-space Representation of Neural Networks

Deep neural networks (NNs) are known to lack uncertainty estimates and s...
research
02/15/2018

Covariance Function Pre-Training with m-Kernels for Accelerated Bayesian Optimisation

The paper presents a novel approach to direct covariance function learni...
research
09/13/2022

Bayesian Functional Emulation of CO2 Emissions on Future Climate Change Scenarios

We propose a statistical emulator for a climate-economy deterministic in...
research
11/15/2022

Neural Bayesian Network Understudy

Bayesian Networks may be appealing for clinical decision-making due to t...

Please sign up or login with your details

Forgot password? Click here to reset