Optimally Efficient Sequential Calibration of Binary Classifiers to Minimize Classification Error

08/19/2021
by   Kaan Gokcesu, et al.
0

In this work, we aim to calibrate the score outputs of an estimator for the binary classification problem by finding an 'optimal' mapping to class probabilities, where the 'optimal' mapping is in the sense that minimizes the classification error (or equivalently, maximizes the accuracy). We show that for the given target variables and the score outputs of an estimator, an 'optimal' soft mapping, which monotonically maps the score values to probabilities, is a hard mapping that maps the score values to 0 and 1. We show that for class weighted (where the accuracy for one class is more important) and sample weighted (where the samples' accurate classifications are not equally important) errors, or even general linear losses; this hard mapping characteristic is preserved. We propose a sequential recursive merger approach, which produces an 'optimal' hard mapping (for the observed samples so far) sequentially with each incoming new sample. Our approach has a logarithmic in sample size time complexity, which is optimally efficient.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/04/2023

Sequential Linearithmic Time Optimal Unimodal Fitting When Minimizing Univariate Linear Losses

This paper focuses on optimal unimodal transformation of the score outpu...
research
06/01/2022

A Log-Linear Time Sequential Optimal Calibration Algorithm for Quantized Isotonic L2 Regression

We study the sequential calibration of estimations in a quantized isoton...
research
10/31/2021

Efficient, Anytime Algorithms for Calibration with Isotonic Regression under Strictly Convex Losses

We investigate the calibration of estimations to increase performance wi...
research
12/02/2013

The Law of Total Odds

The law of total probability may be deployed in binary classification ex...
research
11/10/2021

Non-Adaptive Stochastic Score Classification and Explainable Halfspace Evaluation

We consider the stochastic score classification problem. There are sever...
research
05/31/2019

Optimized Score Transformation for Fair Classification

This paper considers fair probabilistic classification where the outputs...
research
01/09/2023

The Optimal Input-Independent Baseline for Binary Classification: The Dutch Draw

Before any binary classification model is taken into practice, it is imp...

Please sign up or login with your details

Forgot password? Click here to reset