Cautious Deep Learning

05/24/2018
by   Yotam Hechtlinger, et al.
2

Most classifiers operate by selecting the maximum of an estimate of the conditional distribution p(y|x) where x stands for the features of the instance to be classified and y denotes its label. This often results in a hubristic bias: overconfidence in the assignment of a definite label. Usually, the observations are concentrated on a small volume but the classifier provides definite predictions for the entire space. We propose constructing conformal prediction sets [vovk2005algorithmic] which contain a set of labels rather than a single label. These conformal prediction sets contain the true label with probability 1-α. Our construction is based on p(x|y) rather than p(y|x) which results in a classifier that is very cautious: it outputs the null set - meaning `I don't know' --- when the object does not resemble the training examples. An important property of our approach is that classes can be added or removed without having to retrain the classifier. We demonstrate the performance on the ImageNet ILSVRC dataset using high dimensional features obtained from state of the art convolutional neural networks.

READ FULL TEXT

page 6

page 7

page 8

research
05/01/2017

Regularizing Model Complexity and Label Structure for Multi-Label Text Classification

Multi-label text classification is a popular machine learning task where...
research
08/11/2017

Learning from Noisy Label Distributions

In this paper, we consider a novel machine learning problem, that is, le...
research
09/02/2016

Least Ambiguous Set-Valued Classifiers with Bounded Error Levels

In most classification tasks there are observations that are ambiguous a...
research
05/02/2022

From Noisy Prediction to True Label: Noisy Prediction Calibration via Generative Model

Noisy labels are inevitable yet problematic in machine learning society....
research
10/28/2019

Generative Well-intentioned Networks

We propose Generative Well-intentioned Networks (GWINs), a novel framewo...
research
09/29/2020

Uncertainty Sets for Image Classifiers using Conformal Prediction

Convolutional image classifiers can achieve high predictive accuracy, bu...

Please sign up or login with your details

Forgot password? Click here to reset