DeepAI AI Chat
Log In Sign Up

Log-Chisquared P-values under Rare and Weak Departures

by   Alon Kipnis, et al.

Consider a multiple hypothesis testing setting in which only a small proportion of the measured features contain non-null effects. Under typical conditions, the log of the P-value associated with each feature is approximately a sparse mixture of chi-squared distributions, one of which is scaled and non-central. We characterize the asymptotic performance of global tests combining these P-values in terms of the chisquared mixture parameters: the scaling parameters controlling heteroscedasticity, the non-centrality parameter describing the effect size, and the parameter controlling the rarity of the non-null features. Specifically, in a phase space involving the last two parameters, we derive a region where all tests are asymptotically powerless. Outside of this region, the Berk-Jones and the Higher Criticism tests of these P-values have maximal power. Inference techniques based on the minimal P-value, false-discovery rate controlling, and Fisher's combination test have sub-optimal asymptotic performance. We provide various examples for recently studied signal detection models that fall under our setting as well as several new ones. Our perturbed log-chisquared P-values formulation seamlessly generalizes these models to their two-sample variant and heteroscedastic situations. The log-chisquared approximation for P-values under the alternative hypothesis is different from Bahadur's classical log-normal approximation. The latter turns out to be unsuitable for analyzing optimal detection in rare/weak feature models.


page 1

page 2

page 3

page 4


On the Inability of the Higher Criticism to Detect Rare/Weak Departures

Consider a multiple hypothesis testing setting involving rare/weak featu...

Two-sample Testing for Large, Sparse High-Dimensional Multinomials under Rare/Weak Perturbations

Given two samples from possibly different discrete distributions over a ...

Distributions associated with simultaneous multiple hypothesis testing

We develop the distribution of the number of hypotheses found to be stat...

On using empirical null distributions in Benjamini-Hochberg procedure

When performing multiple testing, adjusting the distribution of the null...

TFisher Tests: Optimal and Adaptive Thresholding for Combining p-Values

For testing a group of hypotheses, tremendous p-value combination method...

General Behaviour of P-Values Under the Null and Alternative

Hypothesis testing results often rely on simple, yet important assumptio...

Valid Heteroskedasticity Robust Testing

Tests based on heteroskedasticity robust standard errors are an importan...