DeepAI
Log In Sign Up

On the Practicality of Deterministic Epistemic Uncertainty

07/01/2021
by   Janis Postels, et al.
4

A set of novel approaches for estimating epistemic uncertainty in deep neural networks with a single forward pass has recently emerged as a valid alternative to Bayesian Neural Networks. On the premise of informative representations, these deterministic uncertainty methods (DUMs) achieve strong performance on detecting out-of-distribution (OOD) data while adding negligible computational costs at inference time. However, it remains unclear whether DUMs are well calibrated and can seamlessly scale to real-world applications - both prerequisites for their practical deployment. To this end, we first provide a taxonomy of DUMs, evaluate their calibration under continuous distributional shifts and their performance on OOD detection for image classification tasks. Then, we extend the most promising approaches to semantic segmentation. We find that, while DUMs scale to realistic vision tasks and perform well on OOD detection, the practicality of current methods is undermined by poor calibration under realistic distributional shifts.

READ FULL TEXT

page 15

page 16

page 19

11/08/2021

Evaluating Predictive Uncertainty and Robustness to Distributional Shift Using Real World Data

Most machine learning models operate under the assumption that the train...
06/15/2020

Depth Uncertainty in Neural Networks

Existing methods for estimating uncertainty in deep learning tend to req...
09/01/2021

Evaluating Predictive Uncertainty under Distributional Shift on Dialogue Dataset

In open-domain dialogues, predictive uncertainties are mainly evaluated ...
06/30/2022

Shifts 2.0: Extending The Dataset of Real Distributional Shifts

Distributional shift, or the mismatch between training and deployment da...
10/29/2021

Deep Deterministic Uncertainty for Semantic Segmentation

We extend Deep Deterministic Uncertainty (DDU), a method for uncertainty...
04/16/2021

I Find Your Lack of Uncertainty in Computer Vision Disturbing

Neural networks are used for many real world applications, but often the...
04/02/2021

Misclassification-Aware Gaussian Smoothing improves Robustness against Domain Shifts

Deep neural networks achieve high prediction accuracy when the train and...