Log In Sign Up

Exact Optimization of Conformal Predictors via Incremental and Decremental Learning

by   Giovanni Cherubin, et al.

Conformal Predictors (CP) are wrappers around ML methods, providing error guarantees under weak assumptions on the data distribution. They are suitable for a wide range of problems, from classification and regression to anomaly detection. Unfortunately, their high computational complexity limits their applicability to large datasets. In this work, we show that it is possible to speed up a CP classifier considerably, by studying it in conjunction with the underlying ML method, and by exploiting incremental decremental learning. For methods such as k-NN, KDE, and kernel LS-SVM, our approach reduces the running time by one order of magnitude, whilst producing exact solutions. With similar ideas, we also achieve a linear speed up for the harder case of bootstrapping. Finally, we extend these techniques to improve upon an optimization of k-NN CP for regression. We evaluate our findings empirically, and discuss when methods are suitable for CP optimization.


page 1

page 2

page 3

page 4


A Confidence Machine for Sparse High-Order Interaction Model

In predictive modeling for high-stake decision-making, predictors must b...

Approximating Full Conformal Prediction at Scale via Influence Functions

Conformal prediction (CP) is a wrapper around traditional machine learni...

Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions

Machine learning (ML) applications have been thriving recently, largely ...

Reflections on "Incremental Cardinality Constraints for MaxSAT"

To celebrate the first 25 years of the International Conference on Princ...

Expected Similarity Estimation for Large-Scale Batch and Streaming Anomaly Detection

We present a novel algorithm for anomaly detection on very large dataset...

Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Prediction

Conventional frequentist learning is known to yield poorly calibrated mo...

Code Repositories


Code accompaining the paper "Exact Optimization of Conformal Predictors via Incremental and Decremental Learning"

view repo