DeepAI AI Chat
Log In Sign Up

Optimized conformal classification using gradient descent approximation

by   Anthony Bellotti, et al.

Conformal predictors are an important class of algorithms that allow predictions to be made with a user-defined confidence level. They are able to do this by outputting prediction sets, rather than simple point predictions. The conformal predictor is valid in the sense that the accuracy of its predictions is guaranteed to meet the confidence level, only assuming exchangeability in the data. Since accuracy is guaranteed, the performance of a conformal predictor is measured through the efficiency of the prediction sets. Typically, a conformal predictor is built on an underlying machine learning algorithm and hence its predictive power is inherited from this algorithm. However, since the underlying machine learning algorithm is not trained with the objective of minimizing predictive efficiency it means that the resulting conformal predictor may be sub-optimal and not aligned sufficiently to this objective. Hence, in this study we consider an approach to train the conformal predictor directly with maximum predictive efficiency as the optimization objective, and we focus specifically on the inductive conformal predictor for classification. To do this, the conformal predictor is approximated by a differentiable objective function and gradient descent used to optimize it. The resulting parameter estimates are then passed to a proper inductive conformal predictor to give valid prediction sets. We test the method on several real world data sets and find that the method is promising and in most cases gives improved predictive efficiency against a baseline conformal predictor.


page 1

page 2

page 3

page 4


Algorithms with Prediction Portfolios

The research area of algorithms with predictions has seen recent success...

On Anytime Learning at Macroscale

Classical machine learning frameworks assume access to a possibly large ...

Conformal Prediction Intervals for Remaining Useful Lifetime Estimation

The main objective of Prognostics and Health Management is to estimate t...

Next Hit Predictor - Self-exciting Risk Modeling for Predicting Next Locations of Serial Crimes

Our goal is to predict the location of the next crime in a crime series,...

Improving upon NBA point-differential rankings

For some time, point-differential has been thought to be a better predic...

Efficiency of conformalized ridge regression

Conformal prediction is a method of producing prediction sets that can b...

Novel Techniques to Assess Predictive Systems and Reduce Their Alarm Burden

The performance of a binary classifier ("predictor") depends heavily upo...