Multilevel Bayesian Deep Neural Networks

03/24/2022
by   Neil K. Chada, et al.
0

In this article we consider Bayesian inference associated to deep neural networks (DNNs) and in particular, trace-class neural network (TNN) priors which were proposed by Sell et al. [39]. Such priors were developed as more robust alternatives to classical architectures in the context of inference problems. For this work we develop multilevel Monte Carlo (MLMC) methods for such models. MLMC is a popular variance reduction technique, with particular applications in Bayesian statistics and uncertainty quantification. We show how a particular advanced MLMC method that was introduced in [4] can be applied to Bayesian inference from DNNs and establish mathematically, that the computational cost to achieve a particular mean square error, associated to posterior expectation computation, can be reduced by several orders, versus more conventional techniques. To verify such results we provide numerous numerical experiments on model problems arising in machine learning. These include Bayesian regression, as well as Bayesian classification and reinforcement learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/12/2019

MOPED: Efficient priors for scalable variational inference in Bayesian deep neural networks

Variational inference for Bayesian deep neural networks (DNNs) requires ...
research
09/06/2023

Learning Active Subspaces for Effective and Scalable Uncertainty Quantification in Deep Neural Networks

Bayesian inference for neural networks, or Bayesian deep learning, has t...
research
11/08/2018

BAR: Bayesian Activity Recognition using variational inference

Uncertainty estimation in deep neural networks is essential for designin...
research
03/08/2021

Bayesian imaging using Plug Play priors: when Langevin meets Tweedie

Since the seminal work of Venkatakrishnan et al. (2013), Plug Play (...
research
07/27/2021

Sparse Bayesian Deep Learning for Dynamic System Identification

This paper proposes a sparse Bayesian treatment of deep neural networks ...
research
01/31/2020

Semi-Exact Control Functionals From Sard's Method

This paper focuses on the numerical computation of posterior expected qu...
research
07/05/2021

Randomized multilevel Monte Carlo for embarrassingly parallel inference

This position paper summarizes a recently developed research program foc...

Please sign up or login with your details

Forgot password? Click here to reset