Online Control of the False Discovery Rate under "Decision Deadlines"

by   Aaron Fisher, et al.

Online testing procedures aim to control the extent of false discoveries over a sequence of hypothesis tests, allowing for the possibility that early-stage test results influence the choice of hypotheses to be tested in later stages. Typically, online methods assume that a permanent decision regarding the current test (reject or not reject) must be made before advancing to the next test. We instead assume that each hypothesis requires an immediate preliminary decision, but also allows us to update that decision until a preset deadline. Roughly speaking, this lets us apply a Benjamini-Hochberg-type procedure over a moving window of hypotheses, where the threshold parameters for upcoming tests can be determined based on preliminary results. Our method controls the false discovery rate (FDR) at every stage of testing, as well as at adaptively chosen stopping times. These results apply even under arbitrary p-value dependency structures.


page 1

page 2

page 3

page 4


PAPRIKA: Private Online False Discovery Rate Control

In hypothesis testing, a false discovery occurs when a hypothesis is inc...

SAFFRON and LORD Ensure Online Control of the False Discovery Rate Under Positive Dependence

Online testing procedures assume that hypotheses are observed in sequenc...

Asynchronous Online Testing of Multiple Hypotheses

We consider the problem of asynchronous online testing, aimed at providi...

NeuralFDR: Learning Discovery Thresholds from Hypothesis Features

As datasets grow richer, an important challenge is to leverage the full ...

Dependence correction of multiple tests with applications to sparsity

The present paper establishes new multiple procedures for simultaneous t...

A framework for Multi-A(rmed)/B(andit) testing with online FDR control

We propose an alternative framework to existing setups for controlling f...

Please sign up or login with your details

Forgot password? Click here to reset