Game-theoretic Formulations of Sequential Nonparametric One- and Two-Sample Tests

12/16/2021
by   Shubhanshu Shekhar, et al.
0

We study the problem of designing consistent sequential one- and two-sample tests in a nonparametric setting. Guided by the principle of testing by betting, we reframe the task of constructing sequential tests into that of selecting payoff functions that maximize the wealth of a fictitious bettor, betting against the null in a repeated game. The resulting sequential test rejects the null when the bettor's wealth process exceeds an appropriate threshold. We propose a general strategy for selecting payoff functions as predictable estimates of the witness function associated with the variational representation of some statistical distance measures, such as integral probability metrics (IPMs) and φ-divergences. Overall, this approach ensures that (i) the wealth process is a non-negative martingale under the null, thus allowing tight control over the type-I error, and (ii) it grows to infinity almost surely under the alternative, thus implying consistency. We accomplish this by designing composite e-processes that remain bounded in expectation under the null, but grow to infinity under the alternative. We instantiate the general test for some common distance metrics to obtain sequential versions of Kolmogorov-Smirnov (KS) test, χ^2-test and kernel-MMD test, and empirically demonstrate their ability to adapt to the unknown hardness of the problem. The sequential testing framework constructed in this paper is versatile, and we end with a discussion on applying these ideas to two related problems: testing for higher-order stochastic dominance, and testing for symmetry.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/29/2023

Sequential Predictive Two-Sample and Independence Testing

We study the problems of sequential nonparametric two-sample and indepen...
research
06/10/2015

Sequential Nonparametric Testing with the Law of the Iterated Logarithm

We propose a new algorithmic framework for sequential hypothesis testing...
research
04/12/2022

Anytime-valid sequential testing for elicitable functionals via supermartingales

We design sequential tests for a large class of nonparametric null hypot...
research
05/23/2023

A Rank-Based Sequential Test of Independence

We consider the problem of independence testing for two univariate rando...
research
09/10/2021

On the eigenvalues associated with the limit null distribution of the Epps-Pulley test of normality

The Shapiro–Wilk test (SW) and the Anderson–Darling test (AD) turned out...
research
02/01/2021

How can one test if a binary sequence is exchangeable? Fork-convex hulls, supermartingales, and Snell envelopes

Suppose we observe an infinite series of coin flips X_1,X_2,…, and wish ...
research
03/26/2022

Design and performance evaluation in Kiefer-Weiss problems when sampling from discrete exponential families

In this paper, we deal with problems of testing hypotheses in the framew...

Please sign up or login with your details

Forgot password? Click here to reset