Fast Predictive Uncertainty for Classification with Bayesian Deep Networks

03/02/2020
by   Marius Hobbhahn, et al.
10

In Bayesian Deep Learning, distributions over the output of classification neural networks are approximated by first constructing a Gaussian distribution over the weights, then sampling from it to receive a distribution over the categorical output distribution. This is costly. We reconsider old work to construct a Dirichlet approximation of this output distribution, which yields an analytic map between Gaussian distributions in logit space and Dirichlet distributions (the conjugate prior to the categorical) in the output space. We argue that the resulting Dirichlet distribution has theoretical and practical advantages, in particular more efficient computation of the uncertainty estimate, scaling to large datasets and networks like ImageNet and DenseNet. We demonstrate the use of this Dirichlet approximation by using it to construct a lightweight uncertainty-aware output ranking for the ImageNet setup.

READ FULL TEXT

Authors

page 7

11/13/2018

A conjugate prior for the Dirichlet distribution

This note investigates a conjugate class for the Dirichlet distribution ...
06/11/2019

Quantifying Intrinsic Uncertainty in Classification via Deep Dirichlet Mixture Networks

With the widespread success of deep neural networks in science and techn...
02/14/2017

Gaussian-Dirichlet Posterior Dominance in Sequential Learning

We consider the problem of sequential learning from categorical observat...
02/20/2020

The continuous categorical: a novel simplex-valued exponential family

Simplex-valued data appear throughout statistics and machine learning, f...
12/22/2021

Constraining cosmological parameters from N-body simulations with Bayesian Neural Networks

In this paper, we use The Quijote simulations in order to extract the co...
05/01/2014

Fast MLE Computation for the Dirichlet Multinomial

Given a collection of categorical data, we want to find the parameters o...
05/28/2019

A New Distribution on the Simplex with Auto-Encoding Applications

We construct a new distribution for the simplex using the Kumaraswamy di...

Code Repositories

LB_for_BNNs_official

Official repository for the paper "Fast Predictive Uncertainty for Classification with Bayesian Deep Networks". Arxiv: https://arxiv.org/abs/2003.01227


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.