DeepAI
Log In Sign Up

Exact Optimization of Conformal Predictors via Incremental and Decremental Learning

02/05/2021
by   Giovanni Cherubin, et al.
0

Conformal Predictors (CP) are wrappers around ML methods, providing error guarantees under weak assumptions on the data distribution. They are suitable for a wide range of problems, from classification and regression to anomaly detection. Unfortunately, their high computational complexity limits their applicability to large datasets. In this work, we show that it is possible to speed up a CP classifier considerably, by studying it in conjunction with the underlying ML method, and by exploiting incremental decremental learning. For methods such as k-NN, KDE, and kernel LS-SVM, our approach reduces the running time by one order of magnitude, whilst producing exact solutions. With similar ideas, we also achieve a linear speed up for the harder case of bootstrapping. Finally, we extend these techniques to improve upon an optimization of k-NN CP for regression. We evaluate our findings empirically, and discuss when methods are suitable for CP optimization.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/28/2022

A Confidence Machine for Sparse High-Order Interaction Model

In predictive modeling for high-stake decision-making, predictors must b...
02/02/2022

Approximating Full Conformal Prediction at Scale via Influence Functions

Conformal prediction (CP) is a wrapper around traditional machine learni...
05/11/2020

Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions

Machine learning (ML) applications have been thriving recently, largely ...
10/10/2019

Reflections on "Incremental Cardinality Constraints for MaxSAT"

To celebrate the first 25 years of the International Conference on Princ...
01/25/2016

Expected Similarity Estimation for Large-Scale Batch and Streaming Anomaly Detection

We present a novel algorithm for anomaly detection on very large dataset...
10/06/2022

Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Prediction

Conventional frequentist learning is known to yield poorly calibrated mo...

Code Repositories

exact-cp-optimization

Code accompaining the paper "Exact Optimization of Conformal Predictors via Incremental and Decremental Learning"


view repo