FARM-Test: Factor-Adjusted Robust Multiple Testing with False Discovery Control

11/15/2017
by   Jianqing Fan, et al.
0

Large-scale multiple testing with correlated and heavy-tailed data arises in a wide range of research areas from genomics, medical imaging to finance. Conventional methods for estimating the false discovery proportion (FDP) often ignore the effect of heavy-tailedness and the dependence structure among test statistics, and thus may lead to inefficient or even inconsistent estimation. Also, the assumption of joint normality is often imposed, which is too stringent for many applications. To address these challenges, in this paper we propose a factor-adjusted robust procedure for large-scale simultaneous inference with control of the false discovery proportion. We demonstrate that robust factor adjustments are extremely important in both improving the power of the tests and controlling FDP. We identify general conditions under which the proposed method produces consistent estimate of the FDP. As a byproduct that is of independent interest, we establish an exponential-type deviation inequality for a robust U-type covariance estimator under the spectral norm. Extensive numerical experiments demonstrate the advantage of the proposed method over several state-of-the-art methods especially when the data are generated from heavy-tailed distributions. Our proposed procedures are implemented in the R-package FarmTest.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/15/2017

A New Perspective on Robust M-Estimation: Finite Sample Theory and Applications to Dependence-Adjusted Multiple Testing

Heavy-tailed errors impair the accuracy of the least squares estimate, w...
research
03/14/2023

Robust Multiple Testing under High-dimensional Dynamic Factor Model

Large-scale multiple testing under static factor models is commonly used...
research
10/19/2020

A factor-adjusted multiple testing of general alternatives

Factor-adjusted multiple testing is used for handling strong correlated ...
research
01/25/2022

NAPA: Neighborhood-Assisted and Posterior-Adjusted Two-sample Inference

Two-sample multiple testing problems of sparse spatial data are frequent...
research
11/22/2022

Robust High-dimensional Tuning Free Multiple Testing

A stylized feature of high-dimensional data is that many variables have ...
research
08/12/2018

Robust high dimensional factor models with applications to statistical machine learning

Factor models are a class of powerful statistical models that have been ...
research
07/03/2022

Asymptotic Uncertainty of False Discovery Proportion

Multiple testing has been a popular topic in statistical research. Altho...

Please sign up or login with your details

Forgot password? Click here to reset