Irrational Exuberance: Correcting Bias in Probability Estimates

10/29/2019
by   Gareth M. James, et al.
0

We consider the common setting where one observes probability estimates for a large number of events, such as default risks for numerous bonds. Unfortunately, even with unbiased estimates, selecting events corresponding to the most extreme probabilities can result in systematically underestimating the true level of uncertainty. We develop an empirical Bayes approach "Excess Certainty Adjusted Probabilities" (ECAP), using a variant of Tweedie's formula, which updates probability estimates to correct for selection bias. ECAP is a flexible non-parametric method, which directly estimates the score function associated with the probability estimates, so it does not need to make any restrictive assumptions about the prior on the true probabilities. ECAP also works well in settings where the probability estimates are biased. We demonstrate through theoretical results, simulations, and an analysis of two real world data sets, that ECAP can provide significant improvements over the original probability estimates.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

03/27/2013

Objective Probability

A distinction is sometimes made between "statistical" and "subjective" p...
12/14/2021

Biased Gradient Estimate with Drastic Variance Reduction for Meta Reinforcement Learning

Despite the empirical success of meta reinforcement learning (meta-RL), ...
03/05/2019

A Prediction Tournament Paradox

In a prediction tournament, contestants "forecast" by asserting a numeri...
11/21/2017

Constrained empirical Bayes priors on regression coefficients

In the context of model uncertainty and selection, empirical Bayes proce...
12/02/2013

The Law of Total Odds

The law of total probability may be deployed in binary classification ex...
11/15/2013

On Estimating Many Means, Selection Bias, and the Bootstrap

With recent advances in high throughput technology, researchers often fi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.