The false positive risk: a proposal concerning what to do about p-values

by   David Colquhoun, et al.

It is widely acknowledged that the biomedical literature suffers from a surfeit of false positive results. Part of the reason for this is the persistence of the myth that observation of p < 0.05 is sufficient justification to claim that you have made a discovery. Unfortunately there has been no unanimity about what should be done about this problem. It is hopeless to expect users to change their reliance on p-values unless they are offered an alternative way of judging the reliability of their conclusions. If the alternative method is to have a chance of being adopted widely, it will have to be easy to understand and to calculate. One such proposal is based on calculation of false positive risk. This is likely to be accepted by users because many of them already think, mistakenly, that the false positive risk is what the p- value tells them, and because it is based on the null hypothesis that the true effect size is zero, a form of reasoning with which most users are familiar. It is suggested that p-values and confidence intervals should continue to be given, but that they should be supplemented by a single additional number that conveys the strength of the evidence better than the p-value. This number could be the prior probability that it would be necessary to believe in order to achieve a false positive risk of, say. 0.05 (which is what many users think, mistakenly, is what the p-value achieves). Alternatively, the (minimum) false positive risk could be specified based on the assumption of a prior probability of 0.5 (the largest value that can be assumed in the absence of hard prior data).



There are no comments yet.


page 1

page 2

page 3

page 4


On agnostic post hoc approaches to false positive control

This document is a book chapter which gives a partial survey on post hoc...

Combining Biomarkers by Maximizing the True Positive Rate for a Fixed False Positive Rate

Biomarkers abound in many areas of clinical research, and often investig...

Lack of evidence for a substantial rate of templated mutagenesis in B cell diversification

B cell receptor sequences diversify through mutations introduced by purp...

Comparing Classification Models on Kepler Data

Even though the original Kepler mission ended due to mechanical failures...

Bicoherence analysis of nonstationary and nonlinear processes

Bicoherence analysis is a well established method for identifying the qu...

Distcomp: Comparing distributions

The distcomp command is introduced and illustrated. The command assesses...

Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation

Explicitly alerting users is not always an optimal intervention, especia...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.