Predictive Uncertainty Quantification with Compound Density Networks

02/04/2019
by   Agustinus Kristiadi, et al.
0

Despite the huge success of deep neural networks (NNs), finding good mechanisms for quantifying their prediction uncertainty is still an open problem. Bayesian neural networks are one of the most popular approaches to uncertainty quantification. On the other hand, it was recently shown that ensembles of NNs, which belong to the class of mixture models, can be used to quantify prediction uncertainty. In this paper, we build upon these two approaches. First, we increase the mixture model's flexibility by replacing the fixed mixing weights by an adaptive, input-dependent distribution (specifying the probability of each component) represented by NNs, and by considering uncountably many mixture components. The resulting class of models can be seen as the continuous counterpart to mixture density networks and is therefore referred to as compound density networks (CDNs). We employ both maximum likelihood and variational Bayesian inference to train CDNs, and empirically show that they yield better uncertainty estimates on out-of-distribution data and are more robust to adversarial examples than the previous approaches.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/07/2020

Investigating maximum likelihood based training of infinite mixtures for uncertainty quantification

Uncertainty quantification in neural networks gained a lot of attention ...
research
06/12/2019

Permutation-based uncertainty quantification about a mixing distribution

Nonparametric estimation of a mixing distribution based on data coming f...
research
12/08/2020

Bayesian Inference for Polycrystalline Materials

Polycrystalline materials, such as metals, are comprised of heterogeneou...
research
07/11/2022

DAUX: a Density-based Approach for Uncertainty eXplanations

Uncertainty quantification (UQ) is essential for creating trustworthy ma...
research
05/14/2020

Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors

Bayesian neural networks (BNNs) demonstrate promising success in improvi...
research
07/23/2018

Approximate Bayesian inference with queueing networks and coupled jump processes

Queueing networks are systems of theoretical interest that give rise to ...
research
11/05/2021

Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learning

Deep neural networks are prone to overconfident predictions on outliers....

Please sign up or login with your details

Forgot password? Click here to reset