Classification Confidence Estimation with Test-Time Data-Augmentation

06/30/2020
by   Yuval Bahat, et al.
0

Machine learning plays an increasingly significant role in many aspects of our lives (including medicine, transportation, security, justice and other domains), making the potential consequences of false predictions increasingly devastating. These consequences may be mitigated if we can automatically flag such false predictions and potentially assign them to alternative, more reliable mechanisms, that are possibly more costly and involve human attention. This suggests the task of detecting errors, which we tackle in this paper for the case of visual classification. To this end, we propose a novel approach for classification confidence estimation. We apply a set of semantics-preserving image transformations to the input image, and show how the resulting image sets can be used to estimate confidence in the classifier's prediction. We demonstrate the potential of our approach by extensively evaluating it on a wide variety of classifier architectures and datasets, including ResNext/ImageNet, achieving state of the art performance. This paper constitutes a significant revision of our earlier work in this direction (Bahat Shakhnarovich, 2018).

READ FULL TEXT
research
04/02/2018

Confidence from Invariance to Image Transformations

We develop a technique for automatically detecting the classification er...
research
10/05/2020

Detecting Misclassification Errors in Neural Networks with a Gaussian Process Model

As neural network classifiers are deployed in real-world applications, i...
research
07/19/2021

Joint Dermatological Lesion Classification and Confidence Modeling with Uncertainty Estimation

Deep learning has played a major role in the interpretation of dermoscop...
research
10/19/2020

Failure Prediction by Confidence Estimation of Uncertainty-Aware Dirichlet Networks

Reliably assessing model confidence in deep learning and predicting erro...
research
04/11/2023

Zoom is what you need: An empirical study of the power of zoom and spatial biases in image classification

Image classifiers are information-discarding machines, by design. Yet, h...
research
02/25/2021

Generalized Adversarial Distances to Efficiently Discover Classifier Errors

Given a black-box classification model and an unlabeled evaluation datas...
research
09/19/2022

Two-stage Modeling for Prediction with Confidence

The use of neural networks has been very successful in a wide variety of...

Please sign up or login with your details

Forgot password? Click here to reset