Log-Chisquared P-values under Rare and Weak Departures

by   Alon Kipnis, et al.

Consider a multiple hypothesis testing setting in which only a small proportion of the measured features contain non-null effects. Under typical conditions, the log of the P-value associated with each feature is approximately a sparse mixture of chi-squared distributions, one of which is scaled and non-central. We characterize the asymptotic performance of global tests combining these P-values in terms of the chisquared mixture parameters: the scaling parameters controlling heteroscedasticity, the non-centrality parameter describing the effect size, and the parameter controlling the rarity of the non-null features. Specifically, in a phase space involving the last two parameters, we derive a region where all tests are asymptotically powerless. Outside of this region, the Berk-Jones and the Higher Criticism tests of these P-values have maximal power. Inference techniques based on the minimal P-value, false-discovery rate controlling, and Fisher's combination test have sub-optimal asymptotic performance. We provide various examples for recently studied signal detection models that fall under our setting as well as several new ones. Our perturbed log-chisquared P-values formulation seamlessly generalizes these models to their two-sample variant and heteroscedastic situations. The log-chisquared approximation for P-values under the alternative hypothesis is different from Bahadur's classical log-normal approximation. The latter turns out to be unsuitable for analyzing optimal detection in rare/weak feature models.



There are no comments yet.


page 1

page 2

page 3

page 4


On the Inability of the Higher Criticism to Detect Rare/Weak Departures

Consider a multiple hypothesis testing setting involving rare/weak featu...

Two-sample Testing for Large, Sparse High-Dimensional Multinomials under Rare/Weak Perturbations

Given two samples from possibly different discrete distributions over a ...

Distributions associated with simultaneous multiple hypothesis testing

We develop the distribution of the number of hypotheses found to be stat...

On using empirical null distributions in Benjamini-Hochberg procedure

When performing multiple testing, adjusting the distribution of the null...

TFisher Tests: Optimal and Adaptive Thresholding for Combining p-Values

For testing a group of hypotheses, tremendous p-value combination method...

A two-stage Fisher exact test for multi-arm studies with binary outcome variables

In small sample studies with binary outcome data, use of a normal approx...

Valid Heteroskedasticity Robust Testing

Tests based on heteroskedasticity robust standard errors are an importan...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.