Improving predictions of Bayesian neural networks via local linearization

by   Alexander Immer, et al.

In this paper we argue that in Bayesian deep learning, the frequently utilized generalized Gauss-Newton (GGN) approximation should be understood as a modification of the underlying probabilistic model and should be considered separately from further approximate inference techniques. Applying the GGN approximation turns a BNN into a locally linearized generalized linear model or, equivalently, a Gaussian process. Because we then use this linearized model for inference, we should also predict using this modified likelihood rather than the original BNN likelihood. This formulation extends previous results to general likelihoods and alleviates underfitting behaviour observed e.g. by Ritter et al. (2018). We demonstrate our approach on several UCI classification datasets as well as CIFAR10.



There are no comments yet.


page 3

page 4

page 10

page 11

page 12

page 13

page 15


Disentangling the Gauss-Newton Method and Approximate Inference for Neural Networks

In this thesis, we disentangle the generalized Gauss-Newton and approxim...

Bayesian Adversarial Spheres: Bayesian Inference and Adversarial Examples in a Noiseless Setting

Modern deep neural network models suffer from adversarial examples, i.e....

Depth induces scale-averaging in overparameterized linear Bayesian neural networks

Inference in deep Bayesian neural networks is only fully understood in t...

On Approximate Inference for Generalized Gaussian Process Models

A generalized Gaussian process model (GGPM) is a unifying framework that...

Local Probabilistic Model for Bayesian Classification: a Generalized Local Classification Model

In Bayesian classification, it is important to establish a probabilistic...

Toward a diagnostic toolkit for linear models with Gaussian-process distributed random effects

Gaussian processes (GPs) are widely used as distributions of random effe...

Simulation Study on Local Influence Diagnosis for Poisson Mixed-Effect Linear Model

Given that hierarchical count data in many fields are not Normally-distr...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.