Practical Valid Inferences for the Two-Sample Binomial Problem

04/10/2019
by   Michael P. Fay, et al.
0

Consider comparing two independent binomial responses. Our interest is whether the two binomial parameters are different, and if different, which is larger, and if larger, by how much. This apparently simple problem was addressed by Fisher in the 1930's, and has been the subject of many review papers since then. Yet there continues to be new work on this issue and no consensus solution. Previous reviews have focused primarily on testing and the properties of validity and power, or primarily on confidence intervals, their coverage, and expected length. Here we evaluate both together. For example, we consider whether a p-value and its matching confidence interval are compatible, meaning that the p-value rejects at level α if and only if the 1-α confidence interval excludes all null parameter values. For focus, we only examine non-asymptotic inferences, so that most of the p-values and confidence intervals are valid (i.e., exact) by construction. Within this focus, we review different methods emphasizing many of the properties and interpretational aspects we desire from applied frequentist inference: validity, accuracy, good power, equivariance, compatibility, coherence, causal interpretation, and parameterization and direction of effect. We show that no one method can meet all the desirable properties and give recommendations based on which properties are given more importance.

READ FULL TEXT

page 14

page 24

research
05/16/2018

Valid and Approximately Valid Confidence Intervals for Current Status Data

We introduce a new framework for creating point-wise confidence interval...
research
03/04/2022

Coverage Properties of Empirical Bayes Intervals

This note is an invited discussion of the article "Confidence Intervals ...
research
06/01/2023

Confidence Intervals for Error Rates in Matching Tasks: Critical Review and Recommendations

Matching algorithms are commonly used to predict matches between items i...
research
09/20/2020

The Significance Filter, the Winner's Curse and the Need to Shrink

The "significance filter" refers to focusing exclusively on statisticall...
research
04/14/2021

Post-selection Problems for Causal Inference with Invalid Instruments: A Solution Using Searching and Sampling

Instrumental variable method is among the most commonly used causal infe...
research
09/26/2021

Mathematical properties and finite-population correction for the Wilson score interval

In this paper we examine the properties of the Wilson score interval, us...
research
11/09/2022

Conformal Frequency Estimation with Sketched Data under Relaxed Exchangeability

A flexible method is developed to construct a confidence interval for th...

Please sign up or login with your details

Forgot password? Click here to reset