A tutorial on conformal prediction

06/21/2007
by   Glenn Shafer, et al.
0

Conformal prediction uses past experience to determine precise levels of confidence in new predictions. Given an error probability ϵ, together with a method that makes a prediction ŷ of a label y, it produces a set of labels, typically containing ŷ, that also contains y with probability 1-ϵ. Conformal prediction can be applied to any method for producing ŷ: a nearest-neighbor method, a support-vector machine, ridge regression, etc. Conformal prediction is designed for an on-line setting in which labels are predicted successively, each one being revealed before the next is predicted. The most novel and valuable feature of conformal prediction is that if the successive examples are sampled independently from the same distribution, then the successive predictions will be right 1-ϵ of the time, even though they are based on an accumulating dataset rather than on independent datasets. In addition to the model under which successive examples are sampled independently, other on-line compression models can also use conformal prediction. The widely used Gaussian linear model is one of these. This tutorial presents a self-contained account of the theory of conformal prediction and works through several numerical examples. A more comprehensive treatment of the topic is provided in "Algorithmic Learning in a Random World", by Vladimir Vovk, Alex Gammerman, and Glenn Shafer (Springer, 2005).

READ FULL TEXT
research
01/30/2013

Learning by Transduction

We describe a method for predicting a classification of an object given ...
research
03/28/2019

Nearest-Neighbor Neural Networks for Geostatistics

Kriging is the predominant method used for spatial prediction, but relie...
research
07/28/2020

Visualizing classification results

Classification is a major tool of statistics and machine learning. A cla...
research
06/22/2019

An enhanced KNN-based twin support vector machine with stable learning rules

Among the extensions of twin support vector machine (TSVM), some scholar...
research
01/22/2021

k-Neighbor Based Curriculum Sampling for Sequence Prediction

Multi-step ahead prediction in language models is challenging due to the...
research
03/13/2022

Learning Maximum Margin Channel Decoders

The problem of learning a channel decoder is considered for two channel ...
research
06/20/2020

Seq2Seq and Joint Learning Based Unix Command Line Prediction System

Despite being an open-source operating system pioneered in the early 90s...

Please sign up or login with your details

Forgot password? Click here to reset