Log In Sign Up

On the Inability of the Higher Criticism to Detect Rare/Weak Departures

by   Alon Kipnis, et al.

Consider a multiple hypothesis testing setting involving rare/weak features: only few features, out of possibly many, deviate from their null hypothesis behavior. Summarizing the significance of each feature by a P-value, we construct a global test against the null using the Higher Criticism (HC) statistics of these P-values. We calibrate the rare/weak model using parameters controlling the asymptotic behavior of these P-values in their near-zero "tail". We derive a region in the parameter space where the HC test is asymptotically powerless. Our derivation involves very different tools than previously used to show the powerlessness of HC, relying on properties of the empirical processes underlying HC. In particular, our result applies to situations where HC is not asymptotically optimal, or when the asymptotically detectable region of the parameter space is unknown.


Log-Chisquared P-values under Rare and Weak Departures

Consider a multiple hypothesis testing setting in which only a small pro...

Two-sample Testing for Large, Sparse High-Dimensional Multinomials under Rare/Weak Perturbations

Given two samples from possibly different discrete distributions over a ...

Testing for Principal Component Directions under Weak Identifiability

We consider the problem of testing, on the basis of a p-variate Gaussian...

Testing One Hypothesis Multiple Times: The Multidimensional Case

The identification of new rare signals in data, the detection of a sudde...

Testing Homogeneity for Normal Mixture Models: Variational Bayes Approach

The test of homogeneity for normal mixtures has been conducted in divers...

On the Hauck-Donner Effect in Wald Tests: Detection, Tipping Points, and Parameter Space Characterization

The Wald test remains ubiquitous in statistical practice despite shortco...

Selective inference for the problem of regions via multiscale bootstrap

Selective inference procedures are considered for computing approximatel...