A Geometric Method for Improved Uncertainty Estimation in Real-time

06/23/2022
by   Gabriella Chouraqui, et al.
0

Machine learning classifiers are probabilistic in nature, and thus inevitably involve uncertainty. Predicting the probability of a specific input to be correct is called uncertainty (or confidence) estimation and is crucial for risk management. Post-hoc model calibrations can improve models' uncertainty estimations without the need for retraining, and without changing the model. Our work puts forward a geometric-based approach for uncertainty estimation. Roughly speaking, we use the geometric distance of the current input from the existing training inputs as a signal for estimating uncertainty and then calibrate that signal (instead of the model's estimation) using standard post-hoc calibration techniques. We show that our method yields better uncertainty estimations than recently proposed approaches by extensively evaluating multiple datasets and models. In addition, we also demonstrate the possibility of performing our approach in near real-time applications. Our code is available at our Github https://github.com/NoSleepDeveloper/Geometric-Calibrator.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/11/2023

Uncertainty Estimation based on Geometric Separation

In machine learning, accurately predicting the probability that a specif...
research
06/25/2021

Improving Uncertainty Calibration of Deep Neural Networks via Truth Discovery and Geometric Optimization

Deep Neural Networks (DNNs), despite their tremendous success in recent ...
research
10/06/2021

Post-hoc Models for Performance Estimation of Machine Learning Inference

Estimating how well a machine learning model performs during inference i...
research
10/08/2021

Temperature as Uncertainty in Contrastive Learning

Contrastive learning has demonstrated great capability to learn represen...
research
07/13/2022

Sample-dependent Adaptive Temperature Scaling for Improved Calibration

It is now well known that neural networks can be wrong with high confide...
research
01/13/2023

Local Model Explanations and Uncertainty Without Model Access

We present a model-agnostic algorithm for generating post-hoc explanatio...
research
10/27/2021

A Geometric Perspective towards Neural Calibration via Sensitivity Decomposition

It is well known that vision classification models suffer from poor cali...

Please sign up or login with your details

Forgot password? Click here to reset