Learning the Distribution: A Unified Distillation Paradigm for Fast Uncertainty Estimation in Computer Vision

07/31/2020
by   Yichen Shen, et al.
8

Calibrated estimates of uncertainty are critical for many real-world computer vision applications of deep learning. While there are several widely-used uncertainty estimation methods, dropout inference stands out for its simplicity and efficacy. This technique, however, requires multiple forward passes through the network during inference and therefore can be too resource-intensive to be deployed in real-time applications. To tackle this issue, we propose a unified distillation paradigm for learning the conditional predictive distribution of a pre-trained dropout model for fast uncertainty estimation of both aleatoric and epistemic uncertainty at the same time. We empirically test the effectiveness of the proposed method on both semantic segmentation and depth estimation tasks, and observe that the student model can well approximate the probability distribution generated by the teacher model, i.e the pre-trained dropout model. In addition to a significant boost in speed, we demonstrate the quality of uncertainty estimates and the overall predictive performance can also be improved with the proposed method.

READ FULL TEXT

page 1

page 5

page 7

page 11

page 12

page 13

page 14

page 15

research
06/04/2019

Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision

While Deep Neural Networks (DNNs) have become the go-to approach in comp...
research
03/08/2017

Dropout Inference in Bayesian Neural Networks with Alpha-divergences

To obtain uncertainty estimates with real-world Bayesian deep learning m...
research
07/14/2022

BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks

High-quality calibrated uncertainty estimates are crucial for numerous r...
research
04/04/2023

Uncertainty estimation in Deep Learning for Panoptic segmentation

As deep learning-based computer vision algorithms continue to improve an...
research
03/06/2020

Scalable Uncertainty for Computer Vision with Functional Variational Inference

As Deep Learning continues to yield successful applications in Computer ...
research
10/21/2021

SLURP: Side Learning Uncertainty for Regression Problems

It has become critical for deep learning algorithms to quantify their ou...
research
03/17/2023

DUDES: Deep Uncertainty Distillation using Ensembles for Semantic Segmentation

Deep neural networks lack interpretability and tend to be overconfident,...

Please sign up or login with your details

Forgot password? Click here to reset