Improving predictions of Bayesian neural networks via local linearization

08/19/2020
by   Alexander Immer, et al.
0

In this paper we argue that in Bayesian deep learning, the frequently utilized generalized Gauss-Newton (GGN) approximation should be understood as a modification of the underlying probabilistic model and should be considered separately from further approximate inference techniques. Applying the GGN approximation turns a BNN into a locally linearized generalized linear model or, equivalently, a Gaussian process. Because we then use this linearized model for inference, we should also predict using this modified likelihood rather than the original BNN likelihood. This formulation extends previous results to general likelihoods and alleviates underfitting behaviour observed e.g. by Ritter et al. (2018). We demonstrate our approach on several UCI classification datasets as well as CIFAR10.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

page 4

page 10

page 11

page 12

page 13

page 15

07/21/2020

Disentangling the Gauss-Newton Method and Approximate Inference for Neural Networks

In this thesis, we disentangle the generalized Gauss-Newton and approxim...
11/29/2018

Bayesian Adversarial Spheres: Bayesian Inference and Adversarial Examples in a Noiseless Setting

Modern deep neural network models suffer from adversarial examples, i.e....
11/23/2021

Depth induces scale-averaging in overparameterized linear Bayesian neural networks

Inference in deep Bayesian neural networks is only fully understood in t...
11/25/2013

On Approximate Inference for Generalized Gaussian Process Models

A generalized Gaussian process model (GGPM) is a unifying framework that...
12/13/2018

Local Probabilistic Model for Bayesian Classification: a Generalized Local Classification Model

In Bayesian classification, it is important to establish a probabilistic...
05/02/2018

Toward a diagnostic toolkit for linear models with Gaussian-process distributed random effects

Gaussian processes (GPs) are widely used as distributions of random effe...
05/08/2018

Simulation Study on Local Influence Diagnosis for Poisson Mixed-Effect Linear Model

Given that hierarchical count data in many fields are not Normally-distr...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.