p-value peeking and estimating extrema

11/02/2020
by   Akshay Balsubramani, et al.
0

A pervasive issue in statistical hypothesis testing is that the reported p-values are biased downward by data "peeking" – the practice of reporting only progressively extreme values of the test statistic as more data samples are collected. We develop principled mechanisms to estimate such running extrema of test statistics, which directly address the effect of peeking in some general scenarios.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/16/2019

Non-Stochastic Hypothesis Testing with Application to Privacy Against Hypothesis-Testing Adversary

In this paper, we consider privacy against hypothesis testing adversarie...
research
01/23/2020

On the Hauck-Donner Effect in Wald Tests: Detection, Tipping Points, and Parameter Space Characterization

The Wald test remains ubiquitous in statistical practice despite shortco...
research
10/11/2018

A Simple Way to Deal with Cherry-picking

Statistical hypothesis testing serves as statistical evidence for scient...
research
01/04/2022

The Most Difference in Means: A Statistic for Null and Near-Zero Results

Two-sample p-values test for statistical significance. Yet p-values cann...
research
05/06/2019

Computing a Data Dividend

Quality data is a fundamental contributor to success in statistics and m...
research
03/01/2019

Improved Differentially Private Analysis of Variance

Hypothesis testing is one of the most common types of data analysis and ...
research
12/25/2022

Optimal regularized hypothesis testing in statistical inverse problems

Testing of hypotheses is a well studied topic in mathematical statistics...

Please sign up or login with your details

Forgot password? Click here to reset