Classifier comparison using precision

09/29/2016
by   Lovedeep Gondara, et al.
0

New proposed models are often compared to state-of-the-art using statistical significance testing. Literature is scarce for classifier comparison using metrics other than accuracy. We present a survey of statistical methods that can be used for classifier comparison using precision, accounting for inter-precision correlation arising from use of same dataset. Comparisons are made using per-class precision and methods presented to test global null hypothesis of an overall model comparison. Comparisons are extended to multiple multi-class classifiers and to models using cross validation or its variants. Partial Bayesian update to precision is introduced when population prevalence of a class is known. Applications to compare deep architectures are studied.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/28/2016

Statistical comparison of classifiers through Bayesian hierarchical modelling

Usually one compares the accuracy of two competing classifiers via null ...
research
10/06/2020

Is the Best Better? Bayesian Statistical Model Comparison for Natural Language Processing

Recent work raises concerns about the use of standard splits to compare ...
research
12/12/2018

Towards Automating Precision Studies of Clone Detectors

Current research in clone detection suffers from poor ecosystems for eva...
research
03/27/2018

Cross-validation in high-dimensional spaces: a lifeline for least-squares models and multi-class LDA

Least-squares models such as linear regression and Linear Discriminant A...
research
05/21/2022

Visualizing CoAtNet Predictions for Aiding Melanoma Detection

Melanoma is considered to be the most aggressive form of skin cancer. Du...
research
03/10/2023

What is the state of the art? Accounting for multiplicity in machine learning benchmark performance

Machine learning methods are commonly evaluated and compared by their pe...

Please sign up or login with your details

Forgot password? Click here to reset