Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision

06/04/2019
by   Fredrik K. Gustafsson, et al.
0

While Deep Neural Networks (DNNs) have become the go-to approach in computer vision, the vast majority of these models fail to properly capture the uncertainty inherent in their predictions. Estimating this predictive uncertainty can be crucial, for instance in automotive applications. In Bayesian deep learning, predictive uncertainty is often decomposed into the distinct types of aleatoric and epistemic uncertainty. The former can be estimated by letting a DNN output the parameters of a probability distribution. Epistemic uncertainty estimation is a more challenging problem, and while different scalable methods recently have emerged, no comprehensive comparison has been performed in a real-world setting. We therefore accept this task and propose an evaluation framework for predictive uncertainty estimation that is specifically designed to test the robustness required in real-world computer vision applications. Using the proposed framework, we perform an extensive comparison of the popular ensembling and MC-dropout methods on the tasks of depth completion and street-scene semantic segmentation. Our comparison suggests that ensembling consistently provides more reliable uncertainty estimates. Code is available at https://github.com/fregu856/evaluating_bdl.

READ FULL TEXT

page 2

page 5

page 16

page 17

research
07/31/2020

Learning the Distribution: A Unified Distillation Paradigm for Fast Uncertainty Estimation in Computer Vision

Calibrated estimates of uncertainty are critical for many real-world com...
research
03/15/2017

What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

There are two major types of uncertainty one can model. Aleatoric uncert...
research
10/07/2019

Evaluating Scalable Uncertainty Estimation Methods for DNN-Based Molecular Property Prediction

Advances in deep neural network (DNN) based molecular property predictio...
research
07/20/2022

Latent Discriminant deterministic Uncertainty

Predictive uncertainty estimation is essential for deploying Deep Neural...
research
12/09/2021

The Peril of Popular Deep Learning Uncertainty Estimation Methods

Uncertainty estimation (UE) techniques – such as the Gaussian process (G...
research
03/20/2023

Reliability in Semantic Segmentation: Are We on the Right Track?

Motivated by the increasing popularity of transformers in computer visio...
research
12/22/2019

A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks

Evaluation of Bayesian deep learning (BDL) methods is challenging. We of...

Please sign up or login with your details

Forgot password? Click here to reset