Hypothesis testing and confidence sets: why Bayesian not frequentist, and how to set a prior with a regulatory authority

12/20/2021
by   Roger Sewell, et al.
0

We marshall the arguments for preferring Bayesian hypothesis testing and confidence sets to frequentist ones. We define admissible solutions to inference problems, noting that Bayesian solutions are admissible. We give six weaker common-sense criteria for solutions to inference problems, all failed by these frequentist methods but satisfied by any admissible method. We note that pseudo-Bayesian methods made by handicapping Bayesian methods to satisfy criteria on type I error rate makes them frequentist not Bayesian in nature. We give four examples showing the differences between Bayesian and frequentist methods; the first to be accessible to those with no calculus, the second to illustrate dramatically in abstract what is wrong with these frequentist methods, the third to show that the same problems arise, albeit to a lesser extent, in everyday statistical problems, and the fourth to illustrate how on some real-life inference problems Bayesian methods require less data than fixed sample-size (resp. pseudo-Bayesian) frequentist hypothesis testing by factors exceeding 3000 (resp 300) without recourse to informative priors. To address the issue of different parties with opposing interests reaching agreement on a prior, we illustrate the beneficial effects of a Bayesian "Let the data decide" policy both on results under a wide variety of conditions and on motivation to reach a common prior by consent. We show that in general the frequentist confidence level contains less relevant Shannon information than the Bayesian posterior, and give an example where no deterministic frequentist critical regions give any relevant information even though the Bayesian posterior contains up to the maximum possible amount. In contrast use of the Bayesian prior allows construction of non-deterministic critical regions for which the Bayesian posterior can be recovered from the frequentist confidence.

READ FULL TEXT

page 24

page 27

page 28

page 32

page 34

page 38

page 39

page 42

research
05/27/2020

How to choose between different Bayesian posterior indices for hypothesis testing in practice

Hypothesis testing is an essential statistical method in psychology and ...
research
02/24/2020

Demystify Lindley's Paradox by Interpreting P-value as Posterior Probability

In the hypothesis testing framework, p-value is often computed to determ...
research
02/28/2023

Finite sample inference for empirical Bayesian methods

In recent years, empirical Bayesian (EB) inference has become an attract...
research
06/14/2022

50 shades of Bayesian testing of hypotheses

Hypothesis testing and model choice are quintessential questions for sta...
research
05/04/2020

Uncertainty quantification in the stochastic block model with an unknown number of classes

We study the frequentist properties of Bayesian statistical inference fo...
research
02/10/2023

SCoPE Sets: A Versatile Framework for Simultaneous Inference

We study asymptotic statistical inference in the space of bounded functi...
research
11/19/2021

History and Nature of the Jeffreys-Lindley Paradox

The Jeffreys-Lindley paradox exposes a rift between Bayesian and frequen...

Please sign up or login with your details

Forgot password? Click here to reset