Exact Optimization of Conformal Predictors via Incremental and Decremental Learning

02/05/2021
by   Giovanni Cherubin, et al.
0

Conformal Predictors (CP) are wrappers around ML methods, providing error guarantees under weak assumptions on the data distribution. They are suitable for a wide range of problems, from classification and regression to anomaly detection. Unfortunately, their high computational complexity limits their applicability to large datasets. In this work, we show that it is possible to speed up a CP classifier considerably, by studying it in conjunction with the underlying ML method, and by exploiting incremental decremental learning. For methods such as k-NN, KDE, and kernel LS-SVM, our approach reduces the running time by one order of magnitude, whilst producing exact solutions. With similar ideas, we also achieve a linear speed up for the harder case of bootstrapping. Finally, we extend these techniques to improve upon an optimization of k-NN CP for regression. We evaluate our findings empirically, and discuss when methods are suitable for CP optimization.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/28/2022

A Confidence Machine for Sparse High-Order Interaction Model

In predictive modeling for high-stake decision-making, predictors must b...
research
02/02/2022

Approximating Full Conformal Prediction at Scale via Influence Functions

Conformal prediction (CP) is a wrapper around traditional machine learni...
research
05/11/2020

Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions

Machine learning (ML) applications have been thriving recently, largely ...
research
10/10/2019

Reflections on "Incremental Cardinality Constraints for MaxSAT"

To celebrate the first 25 years of the International Conference on Princ...
research
01/25/2016

Expected Similarity Estimation for Large-Scale Batch and Streaming Anomaly Detection

We present a novel algorithm for anomaly detection on very large dataset...
research
08/31/2016

Measuring the Quality of Exercises

This work explores the problem of exercise quality measurement since it ...

Please sign up or login with your details

Forgot password? Click here to reset