p-value peeking and estimating extrema

11/02/2020
by   Akshay Balsubramani, et al.
0

A pervasive issue in statistical hypothesis testing is that the reported p-values are biased downward by data "peeking" – the practice of reporting only progressively extreme values of the test statistic as more data samples are collected. We develop principled mechanisms to estimate such running extrema of test statistics, which directly address the effect of peeking in some general scenarios.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

04/16/2019

Non-Stochastic Hypothesis Testing with Application to Privacy Against Hypothesis-Testing Adversary

In this paper, we consider privacy against hypothesis testing adversarie...
01/04/2022

The Most Difference in Means: A Statistic for Null and Near-Zero Results

Two-sample p-values test for statistical significance. Yet p-values cann...
01/23/2020

On the Hauck-Donner Effect in Wald Tests: Detection, Tipping Points, and Parameter Space Characterization

The Wald test remains ubiquitous in statistical practice despite shortco...
10/11/2018

A Simple Way to Deal with Cherry-picking

Statistical hypothesis testing serves as statistical evidence for scient...
05/06/2019

Computing a Data Dividend

Quality data is a fundamental contributor to success in statistics and m...
03/01/2019

Improved Differentially Private Analysis of Variance

Hypothesis testing is one of the most common types of data analysis and ...
11/19/2019

Measuring spatiotemporal disease clustering with the tau statistic

Introduction: The tau statistic uses geolocation and, usually, symptom o...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.