Uncertainty Quantification for Sparse Deep Learning

02/26/2020
by   Yuexi Wang, et al.
0

Deep learning methods continue to have a decided impact on machine learning, both in theory and in practice. Statistical theoretical developments have been mostly concerned with approximability or rates of estimation when recovering infinite dimensional objects (curves or densities). Despite the impressive array of available theoretical results, the literature has been largely silent about uncertainty quantification for deep learning. This paper takes a step forward in this important direction by taking a Bayesian point of view. We study Gaussian approximability of certain aspects of posterior distributions of sparse deep ReLU architectures in non-parametric regression. Building on tools from Bayesian non-parametrics, we provide semi-parametric Bernstein-von Mises theorems for linear and quadratic functionals, which guarantee that implied Bayesian credible regions have valid frequentist coverage. Our results provide new theoretical justifications for (Bayesian) deep learning with ReLU activation functions, highlighting their inferential potential.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/09/2019

On Semi-parametric Bernstein-von Mises Theorems for BART

Few methods in Bayesian non-parametric statistics/ machine learning have...
research
07/31/2020

Statistical guarantees for Bayesian uncertainty quantification in non-linear inverse problems with Gaussian process priors

Bayesian inference and uncertainty quantification in a general class of ...
research
11/09/2018

Bernstein-von Mises theorems and uncertainty quantification for linear inverse problems

We consider the statistical inverse problem of approximating an unknown ...
research
05/24/2023

Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inference

Bayesian approaches for learning deep neural networks (BNN) have been re...
research
10/22/2021

Merging Two Cultures: Deep and Statistical Learning

Merging the two cultures of deep and statistical learning provides insig...
research
07/05/2023

Transgressing the boundaries: towards a rigorous understanding of deep learning and its (non-)robustness

The recent advances in machine learning in various fields of application...

Please sign up or login with your details

Forgot password? Click here to reset