Boosting Uncertainty Estimation for Deep Neural Classifiers

05/21/2018
by   Yonatan Geifman, et al.
0

We consider the problem of uncertainty estimation in the context of (non-Bayesian) deep neural classification. All current methods are based on extracting uncertainty signals from a trained network optimized to solve the classification problem at hand. We demonstrate that such techniques tend to misestimate instances whose predictions are supposed to be highly confident. This deficiency is an artifact of the training process with SGD-like optimizers. Based on this observation, we develop an uncertainty estimation algorithm that "peels away" highly confident points sequentially and estimates their confidence using earlier snapshots of the trained model, before their uncertainty estimates are jittered. We present extensive experiments indicating that the proposed algorithm provides uncertainty estimates that are consistently better than the best known methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/09/2019

BayesOD: A Bayesian Approach for Uncertainty Estimation in Deep Object Detectors

One of the challenging aspects of incorporating deep neural networks int...
research
01/08/2019

Uncertainty-Based Out-of-Distribution Detection in Deep Reinforcement Learning

We consider the problem of detecting out-of-distribution (OOD) samples i...
research
04/17/2023

Collaborative Bearing Estimation Using Set Membership Methods

We consider the problem of collaborative bearing estimation using a meth...
research
10/29/2018

Principled Uncertainty Estimation for Deep Neural Networks

When the cost of misclassifying a sample is high, it is useful to have a...
research
06/17/2020

Sample Efficient Uncertainty Estimation for Deep Learning Safety

Deep Neural Networks (DNNs) are known to make highly overconfident predi...
research
07/11/2012

MOB-ESP and other Improvements in Probability Estimation

A key prerequisite to optimal reasoning under uncertainty in intelligent...
research
05/14/2020

Estimating predictive uncertainty for rumour verification models

The inability to correctly resolve rumours circulating online can have h...

Please sign up or login with your details

Forgot password? Click here to reset