Nested Conformal Prediction and the Generalized Jackknife+

10/23/2019
by   Arun K. Kuchibhotla, et al.
0

We provide an alternate unified framework for conformal prediction, which is a framework to provide assumption-free prediction intervals. Instead of beginning by choosing a conformity score, our framework starts with a sequence of nested sets {F_t(x)}_t∈T for some ordered set T that specifies all potential prediction sets. We show that most proposed conformity scores in the literature, including several based on quantiles, straightforwardly result in nested families. Then, we argue that what conformal prediction does is find a mapping α t(α), meaning that it calibrates or rescales T to [0,1]. Nestedness is a natural and intuitive requirement because the optimal prediction sets (eg: level sets of conditional densities) are also nested, but we also formally prove that nested sets are universal, meaning that any conformal prediction method can be represented in our framework. Finally, to demonstrate its utility, we show how to develop the full conformal, split conformal, cross-conformal and the recent jackknife+ methods within our nested framework, thus immediately generalizing the latter two classes of methods to new settings. Specifically, we prove the validity of the leave-one-out, K-fold, subsampling and bootstrap variants of the latter two methods for any nested family.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/20/2019

On a conjecture by Ben-Akiva and Lerman about the nested logit model

We prove a conjecture of Ben-Akiva and Lerman (1985) regarding the rando...
research
06/27/2019

On nested and 2-nested graphs: two subclasses of graphs between threshold and split graphs

A (0,1)-matrix has the Consecutive Ones Property (C1P) for the rows if t...
research
07/20/2019

A short proof of a conjecture by Ben-Akiva and Lerman about the nested logit model

We provide a short proof of a result by Cardell (1997) on a conjecture o...
research
04/05/2013

Nested Aggregates in Answer Sets: An Application to a Priori Optimization

We allow representing and reasoning in the presence of nested multiple a...
research
01/13/2021

Parametricity for Nested Types and GADTs

This paper considers parametricity and its consequent free theorems for ...
research
10/25/2018

Approximation of trees by self-nested trees

The class of self-nested trees presents remarkable compression propertie...
research
09/08/2018

Ensembles of Nested Dichotomies with Multiple Subset Evaluation

A system of nested dichotomies is a method of decomposing a multi-class ...

Please sign up or login with your details

Forgot password? Click here to reset