BET on Independence

10/17/2016
by   Kai Zhang, et al.
0

We study the problem of nonparametric dependence detection. Many existing methods suffer severe power loss due to non-uniform consistency, which we illustrate with a paradox. To avoid such power loss, we approach the nonparametric test of independence through the new framework of binary expansion statistics (BEStat) and binary expansion testing (BET), which examine dependence through a novel binary expansion filtration approximation of the copula. Through a Hadamard-Walsh transform, we find that the cross interactions of binary variables in the filtration are complete sufficient statistics for dependence. These interactions are also uncorrelated under the null. By utilizing these interactions, the BET avoids the problem of non-uniform consistency and improves upon a wide class of commonly used methods (a) by achieving the minimax rate in sample size requirement for specified power and (b) by providing clear interpretations of global and local relationships upon rejection of independence. The binary expansion approach also connects the test statistics with the current computing system to facilitate efficient bitwise implementation. We illustrate the BET by a study of the distribution of stars in the night sky and by an exploratory data analysis of the TCGA breast cancer data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/08/2019

Testing Independence with the Binary Expansion Randomized Ensemble Test

Recently, the binary expansion testing framework was introduced to test ...
research
03/01/2021

BEAUTY Powered BEAST

We study inference about the uniform distribution with the proposed bina...
research
12/18/2022

A Permutation-Free Kernel Independence Test

In nonparametric independence testing, we observe i.i.d. data {(X_i,Y_i)...
research
12/12/2019

Testing Independence under Biased Sampling

Testing for association or dependence between pairs of random variables ...
research
05/09/2015

An Empirical Study of Leading Measures of Dependence

In exploratory data analysis, we are often interested in identifying pro...
research
02/20/2022

Pairwise Nonlinear Dependence Analysis of Genomic Data

In The Cancer Genome Atlas (TCGA) dataset, there are many interesting no...
research
01/08/2018

The Expected Parameter Change (EPC) for Local Dependence Assessment in Binary Data Latent Class Models

Binary data latent class models crucially assume local independence, vio...

Please sign up or login with your details

Forgot password? Click here to reset