Optimized conformal classification using gradient descent approximation

05/24/2021
by   Anthony Bellotti, et al.
0

Conformal predictors are an important class of algorithms that allow predictions to be made with a user-defined confidence level. They are able to do this by outputting prediction sets, rather than simple point predictions. The conformal predictor is valid in the sense that the accuracy of its predictions is guaranteed to meet the confidence level, only assuming exchangeability in the data. Since accuracy is guaranteed, the performance of a conformal predictor is measured through the efficiency of the prediction sets. Typically, a conformal predictor is built on an underlying machine learning algorithm and hence its predictive power is inherited from this algorithm. However, since the underlying machine learning algorithm is not trained with the objective of minimizing predictive efficiency it means that the resulting conformal predictor may be sub-optimal and not aligned sufficiently to this objective. Hence, in this study we consider an approach to train the conformal predictor directly with maximum predictive efficiency as the optimization objective, and we focus specifically on the inductive conformal predictor for classification. To do this, the conformal predictor is approximated by a differentiable objective function and gradient descent used to optimize it. The resulting parameter estimates are then passed to a proper inductive conformal predictor to give valid prediction sets. We test the method on several real world data sets and find that the method is promising and in most cases gives improved predictive efficiency against a baseline conformal predictor.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/22/2022

Algorithms with Prediction Portfolios

The research area of algorithms with predictions has seen recent success...
research
06/17/2021

On Anytime Learning at Macroscale

Classical machine learning frameworks assume access to a possibly large ...
research
12/30/2022

Conformal Prediction Intervals for Remaining Useful Lifetime Estimation

The main objective of Prognostics and Health Management is to estimate t...
research
12/13/2018

Next Hit Predictor - Self-exciting Risk Modeling for Predicting Next Locations of Serial Crimes

Our goal is to predict the location of the next crime in a crime series,...
research
12/03/2019

Improving upon NBA point-differential rankings

For some time, point-differential has been thought to be a better predic...
research
04/08/2014

Efficiency of conformalized ridge regression

Conformal prediction is a method of producing prediction sets that can b...
research
02/10/2021

Novel Techniques to Assess Predictive Systems and Reduce Their Alarm Burden

The performance of a binary classifier ("predictor") depends heavily upo...

Please sign up or login with your details

Forgot password? Click here to reset