Log In Sign Up

Dynamic Algorithms for Online Multiple Testing

by   Ziyu Xu, et al.

We demonstrate new algorithms for online multiple testing that provably control false discovery exceedance (FDX) while achieving orders of magnitude more power than previous methods. This statistical advance is enabled by the development of new algorithmic ideas: earlier algorithms are more "static" while our new ones allow for the dynamical adjustment of testing levels based on the amount of wealth the algorithm has accumulated. We also prove relationships between controlling FDR, FDX, and other error metrics for our new algorithm, SupLORD, and how controlling one metric can simultaneously control all error metrics. We demonstrate that our algorithms achieve higher power in a variety of synthetic experiments.


page 1

page 2

page 3

page 4


The Power of Batching in Multiple Hypothesis Testing

One important partition of algorithms for controlling the false discover...

Only Closed Testing Procedures are Admissible for Controlling False Discovery Proportions

We consider the class of all multiple testing methods controlling tail p...

Contextual Online False Discovery Rate Control

Multiple hypothesis testing, a situation when we wish to consider many h...

A framework for Multi-A(rmed)/B(andit) testing with online FDR control

We propose an alternative framework to existing setups for controlling f...

Treatment Effect Detection with Controlled FDR under Dependence for Large-Scale Experiments

Online controlled experiments (also known as A/B Testing) have been view...

Learning Memory-Efficient Stable Linear Dynamical Systems for Prediction and Control

Learning a stable Linear Dynamical System (LDS) from data involves creat...

Drift Detection in Episodic Data: Detect When Your Agent Starts Faltering

Detection of deterioration of agent performance in dynamic environments ...