DeepAI AI Chat
Log In Sign Up

Confidence intervals for class prevalences under prior probability shift

by   Dirk Tasche, et al.

Point estimation of class prevalences in the presence of data set shift has been a popular research topic for more than two decades. Less attention has been paid to the construction of confidence and prediction intervals for estimates of class prevalences. One little considered question is whether or not it is necessary for practical purposes to distinguish confidence and prediction intervals. Another question so far not yet conclusively answered is whether or not the discriminatory power of the classifier or score at the basis of an estimation method matters for the accuracy of the estimates of the class prevalences. This paper presents a simulation study aimed at shedding some light on these and other related questions.


page 1

page 2

page 3

page 4


Extreme value theory based confidence intervals for the parameters of a symmetric Lévy-stable distribution

We exploit the asymptotic normality of the extreme value theory (EVT) ba...

Exceedance probability for parameter estimates

Many researchers and statisticians are conflicted over the practice of h...

How response designs and class proportions affect the accuracy of validation data

Reference data collected to validate land cover maps are generally consi...

Confidence regions for univariate and multivariate data using permutation tests

Confidence intervals are central to statistical inference. We devise a m...

Exploring the nuances of R0: Eight estimates and application to 2009 pandemic influenza

For nearly a century, the initial reproduction number (R0) has been used...

Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation

This work aims to assess how well a model performs under distribution sh...

Confidence intervals with maximal average power

In this paper, we propose a frequentist testing procedure that maintains...