Finite-sample Efficient Conformal Prediction

04/28/2021
by   Yachong Yang, et al.
0

Conformal prediction is a generic methodology for finite-sample valid distribution-free prediction. This technique has garnered a lot of attention in the literature partly because it can be applied with any machine learning algorithm that provides point predictions to yield valid prediction regions. Of course, the efficiency (width/volume) of the resulting prediction region depends on the performance of the machine learning algorithm. In this paper, we consider the problem of obtaining the smallest conformal prediction region given a family of machine learning algorithms. We provide two general-purpose selection algorithms and consider coverage as well as width properties of the final prediction region. The first selection method yields the smallest width prediction region among the family of conformal prediction regions for all sample sizes, but only has an approximate coverage guarantee. The second selection method has a finite sample coverage guarantee but only attains close to the smallest width. The approximate optimal width property of the second method is quantified via an oracle inequality. Asymptotic oracle inequalities are also considered when the family of algorithms is given by ridge regression with different penalty parameters.

READ FULL TEXT
research
04/12/2023

Post-selection Inference for Conformal Prediction: Trading off Coverage for Precision

Conformal inference has played a pivotal role in providing uncertainty q...
research
01/30/2023

Bagging Provides Assumption-free Stability

Bagging is an important technique for stabilizing machine learning model...
research
03/02/2023

Design-based conformal prediction

Conformal prediction is an assumption-lean approach to generating distri...
research
05/09/2019

Conformal prediction for exponential families and generalized linear models

Conformal prediction methods construct prediction regions for iid data t...
research
06/15/2021

Localized Conformal Prediction: A Generalized Inference Framework for Conformal Prediction

We propose a new inference framework called localized conformal predicti...
research
05/10/2019

Prediction and outlier detection: a distribution-free prediction set with a balanced objective

We consider the multi-class classification problem when the training dat...
research
11/04/2021

Conformal prediction for text infilling and part-of-speech prediction

Modern machine learning algorithms are capable of providing remarkably a...

Please sign up or login with your details

Forgot password? Click here to reset