Test of Covariance and Correlation Matrices

12/04/2018
by   Longyang Wu, et al.
0

Based on a generalized cosine measure between two symmetric matrices, we propose a general framework for one-sample and two-sample tests of covariance and correlation matrices. We also develop a set of associated permutation algorithms for some common one-sample tests, such as the tests of sphericity, identity and compound symmetry, and the K-sample tests of multivariate equality of covariance or correlation matrices. The proposed method is very flexible in the sense that it does not assume any underlying distributions and data generation models. Moreover, it allows data to have different marginal distributions in both the one-sample identity and K-sample tests. Through real datasets and extensive simulations, we demonstrate that the proposed method performs well in terms of empirical type I error and power in a variety of hypothesis testing situations in which data of different sizes and dimensions are generated using different distributions and generation models.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

12/05/2021

Bayesian Optimal Two-sample Tests in High-dimension

We propose optimal Bayesian two-sample tests for testing equality of hig...
05/15/2018

Visualizing Tests for Equality of Covariance Matrices

This paper explores a variety of topics related to the question of testi...
09/10/2018

Maximum Pairwise Bayes Factors for Covariance Structure Testing

Hypothesis testing of structure in covariance matrices is of significant...
01/28/2020

WISDoM: a framework for the Analysis of Wishart distributed matrices

WISDoM (Wishart Distributed Matrices) is a new framework for the charact...
11/22/2021

Using prior information to boost power in correlation structure support recovery

Hypothesis testing of structure in correlation and covariance matrices i...
03/31/2020

Covariance-Robust Dynamic Watermarking

Attack detection and mitigation strategies for cyberphysical systems (CP...
05/04/2021

Resampling Methods for Detecting Anisotropic Correlation Structure

This paper proposes parametric and non-parametric hypothesis testing alg...

1 Introduction

Testing covariance and correlation matrices is an enduring topic in statistics. In addition to its apparent importance in classical multivariate analysis such as multivariate analysis of variance, Fisher’s linear discriminant analysis, principle component analysis, and repeated measure analysis

Johnson2007, the hypothesis testing of covariance or correlation matrices has a wide range of applications in many scientific fields such as psychometric analysis of behavior difference green1992testing, co-movement dynamics of credit risk annaert2006intertemporal, genetic analysis of gene expression singh2002gene, dynamic modeling of the correlation structure of the stock market djauhari2014dynamics.

Traditionally, the tests of covariance or correlation matrices are discussed separately in the literature.

bartR proposed a method to test if a sample correlation matrix is an identity matrix.

jennrich1970asymptotic proposed an asymptotic

test for the equality of two correlation matrices of multivariate normal distributions. Jennrich’s test has been adopted in several statistical softwares and applied in financial studies. Likelihood ratio based tests for covariance matrices of multivariate normal distributions are well documented in the classical multivariate analysis literature (e.g.

Muirhead2005, Chp. 8). Since the classical likelihood ratio based tests are generally not applicable in high-dimensional data due to the singularity of sample covariance matrices, the recent methodology developments focus on the tests of high-dimensional covariance and correlation matrices, where the dimension

is much bigger than the sample size . Among others, ledoit2002some studied the performance of two classical statistics for the one-sample test of sphericity and identity in a high-dimensional setting.

chen2010tests adopted the similar test statistics as in

ledoit2002some but used U-statistics to estimate the population covariance matrix.

li2012two proposed a two-sample equality test of covariance matrices. fujikoshi2011multivariate provided a summary of some classical and high-dimensional tests of covariance matrices of multivariate normal distributions.

yao2015sample discussed the recent developments on estimation and hypothesis tests of covariance matrices in high dimensions from the random matrix theory viewpoint.

cai2017global presented an up-to-date review of methodology developments of the equality test of high-dimensional covariance matrices with sparse signals.

There obviously lacks a unified and flexible hypothesis testing framework for practitioners to handle both covariance and correlation matrices without switching between different test statistics or procedures. In this study we propose such a unified and easy-to-use approach for many common tests of covariance and correlation matrices. We exploit a simple geometric property of symmetric matrices to construct test statistics and develop the associated permutation algorithms to compute p-values under a variety of null hypotheses. Our proposed method can handle a variety of one-sample tests including the tests of sphericity, identity and compound symmetry, and two- and -sample equality tests (). One advantage of our proposed sphericity and compound symmetry tests is that the proposed method does not involve either the unknown population variance or the intra-class correlation coefficient. Moreover, the proposed method is very flexible in the sense that it does not assume any underlying distributions or data generation models. Through both real data analyses and extensive simulations, we demonstrate that the proposed method can control the type I error at the pre-specified nominal level and provide good power under the alternative in both one-sample and -sample tests, even for high-dimensional data where the dimension is much bigger than the sample size .

This paper is organized as follows. Section 2 introduces a generalized cosine measure between two symmetric matrices and, based on this notion of generalized cosine, the test statistics for one-sample and two-sample tests are proposed. Section 3 studies the one-sample tests of sphericity, identity and compound symmetry. Section 4 discusses the two-sample equality test of covariance or correlation matrices and extends the method to handle the -sample problem. Section 5 contains extensive simulation studies on the performance of the proposed one-sample and two-sample tests. In addition, Section 5 probes into factors that affect the performance of the proposed methods. The paper concludes with discussions in Section 6.

2 Test statistic

Below is the definition of the generalized cosine measure between two symmetric matrices.

Definition.

Let

be a vector space of symmetric matrices of size

and : . The angle between two symmetric matrices and in with respect to is defined as and

(1)

where is an inner product and is the corresponding norm in .

We use two symmetric matrices and to illustrate some possible cosine values with different mappings, their associated inner products and norms. Matrix is

which is a modified Hilbert matrix defined as follows:

(2)

for . Matrix is

which is a realization of the random matrix , constructed from , as follows:

(3)

where

is generated from a uniform distribution over the interval (0, 0.5).

Case 1: Suppose is an identity mapping, i.e., . We consider the Frobenius inner product and its corresponding norm . By Equation (1), the cosine between and is calculated as

(4)

which is 0.92, corresponding to an angle of 0.41 radian or .

Case 2: In this case, we construct a two-step mapping : . In step one we apply the Cholesky decomposition to obtain and , where and are lower triangle matrices with positive diagonal elements. In step two we apply the half-vectorization operator (which vectorizes or stacks only the lower trianguler portion of a symmetric matrix) to and separately to obtain two vectors and , each with a length of . To apply Equation (1), we adopt the Euclidean inner product and norm to compute cosine as

(5)

which is 0.87, corresponding to an angle of 0.52 radian or . This mapping is feasible only if and are positive definite matrices.

Case 3: The mapping can also be constructed by first applying the eigen-decomposition to obtain for and for , respectively. and

are orthogonal matrices of eigenvectors, while

and

are diagonal matrices whose elements are the eigenvalues of

and , respectively. Provided that not all diagonal elements in and are zero, we can extract the diagonal elements from and to create vectors and , respectively. Thus, we construct a mapping : . To apply Equation (1), we adopt the Euclidean inner product and norm to compute the cosine as

(6)

which is 0.93, corresponding to a angle of 0.37 radian or approximately .

Case 4: Since a symmetric matrix is completely determined by its lower triangular elements together with the symmetry, we construct the mapping by applying the half-vectorization operator on and directly. The cosine between and can be obtained from Equation (1) as follows:

(7)

which is 0.94, corresponding to a angle of 0.35 radian or .

For correlation matrices, the computation can be simplified by introducing a modified half-vectorization operator from by excluding the diagonal elements of the matrix. Suppose and are two correlation matrices, the cosine can be computed by Equation (1) as

(8)

which is 0.95, corresponding to 0.31 radian or .

In summary, we have considered four different mappings to compute the generalized cosine between the two modified Hilbert matrices and . The mapping based on the Cholesky decomposition gives the lowest cosine value of 0.87. The mappings based on the Frobenius inner product and the eigen-decomposition give the similar cosine values of 0.92 and 0.93, respectively. When the redundant information is removed, the half-vectorization and the modified half-vectorization operators give the largest cosine values of 0.94 and 0.95, respectively. In this study we choose the half-vectorization operator, , for covariance matrices and the modified half-vectorization operator, , for correlation matrices, because these two operators completely remove the redundancy in symmetric matrices and are very easy to compute. Moreover, these two operators demonstrate better statistical power in our pilot simulation studies. The cosine value computed from Equation (1) measures the similarity between two symmetric matrices. When this value is one, the two matrices are identical. We exploit this property to propose new test statistics for one-sample and two-sample tests of covariance and correlation matrices.

One-sample test statistic.

Let be the sample covariance (correlation) matrix with the corresponding population covariance (correlation) matrix . The test statistic for the equality of and , which is either known or specified up to some unknown parameters, is

(9)

where the cosine is computed according to Equation (1) under a pre-determined mapping with a properly chosen inner product and norm.

Two-sample test statistic.

Let and be the sample covariance (correlation) matrices with the corresponding population covariance (correlation) matrices and , respectively. The test statistic for the equality of and is

(10)

where the cosine is computed according to Equation (1) under a pre-determined mapping with a properly chosen inner product and norm.

It is worth pointing out that the two newly proposed test statistics exploit a simple geometric property of symmetric matrices and require no distributional assumptions of data. We further adopt a permutation approach to obtain the null distributions of the two test statistics and compute p-values under the null hypotheses of different tests.

3 One-sample test

Suppose a matrix, , and , contains data collected on subjects and each with covariates. Its th row is a realization of random vector , which has an unknown distribution with a covariance matrix and a correlation matrix . In the one-sample problem one may use the data matrix to test if the covariance matrix is proportional to an identity matrix, known as the sphericity test, or to test if or equals to an identity matrix, known as the identity test.

3.1 Sphericity test

The hypotheses in the sphericity test for a covariance matrix are as follows:

(11)

where is the unknown population variance under the null. In this case, in the one-sample test statistic (9) is and is the sample covariance matrix of the data matrix . To test the sphericity of a covariance matrix, using , the Euclidean inner product and norm, the test statistic (9) can be shown to be

(12)

where denotes the trace. The derivation of (12) is provided in Appendix I. Note that the test statistic (12) does not depend on the unknown parameter .

The sphericity test (11) is a combination of two tests fujikoshi2011multivariate: 1) vs. and 2) vs. . To handle these two tests simultaneously, we propose Algorithm 1 to compute p-values under the null of sphericity. This algorithm first permutes the elements in each row of to generate a permuted data matrix , then the elements in each column of to generate . The permutation on each row of the data matrix is to test if the diagonal elements in are identical; the permutation on each column is to test if the off-diagonal elements of are zero. Algorithm 1 assumes that the elements in the random vector are exchangeable in the sense that is invariant to the permutations of the elements in

under the null hypothesis of (

11). This algorithm requires that the elements in have the same distribution so that the elements in each row of can be permuted.

the number of permutations
,
Compute the sample covariance matrix of
Compute the test statistic (12) using
 For to
   Permute the elements in each row of
   Permute the elements in each column of
   Compute the sample covariance matrix of
   Compute the test statistic (12) using
 End For
 Report -value
Algorithm 1 Test the sphericity of a covariance matrix.

We illustrate the proposed sphericity test and Algorithm 1 using the bfi data in the R package pysch psych. The bfi data contain twenty-five personality self reported items from 2800 subjects who participated in the Synthetic Aperture Personality Assessment (SAPA) web based personality assessment project. These twenty-five items are organized by five putative factors: Agreeableness, Conscientiousness, Extraversion, Neuroticism, and Openness. Each subject answered these personality questions using a six point response scale ranging from 1 (Very Inaccurate) to 6 (Very Accurate). After removing the missing data, we apply the proposed sphericity test and Algorithm 1 to test the sphericity of the covariance matrix of personality measurements of 2436 subjects. We obtain a p-value of 0.01 based on 100 permutations using Algorithm 1. Therefore, the null hypothesis of sphericity is rejected at the significance level of 0.05. We will further investigate the empirical type I error and power of our proposed sphericity test in Section 5.

3.2 Identity test

The identity test of a covariance matrix can be formulated as [Ledoit and WolfLedoit and Wolf2002]

(13)

which includes the more general null hypothesis , where is a known covariance matrix, since we can replace by and multiply data by .

Because in the hypothesis (13) and in the sphericity hypothesis (11) only differ by a constant and the test statistic (12) does not depend on , we can adopt the test statistic (12) to test the hypothesis (13).

For the identity test of a correlation matrix,

(14)

for which, the test statistic (12) can be simplified into

(15)

The derivation of (15) is provided in Appendix I.

Since the identity test mainly concerns the off-diagonal elements of a covariance or correlation matrix, we propose Algorithm 2, which permutes the elements in each column of the data matrix , to compute p-values under the null. Permuting the elements in columns can disrupt the covariance structure, but it does not change the sample variance of each column of the data matrix, i.e. the diagonal elements of the sample covariance matrix remain unchanged in different permutations. This implies that, for a given , remains a constant in (12) in different permutations. For a correlation matrix , we have =

. Since the column-wise permutations can sufficiently disassociate every pair of random variables, and hence, disrupt the covariance structure of the

random variables, the off-diagonal elements of the sample covariance matrix of the permuted data are much closer to zero than the original sample covariance (correlation) matrix under the alternative. Therefore, the sample covariance and correlation matrices with a good portion of non-zero off-diagonal elements have larger values of the statistics of (12) and (15), respectively, to reject the null hypothesis of identity. Moreover, Algorithm 2 does not require each column of a data matrix to have the same distribution.

the number of permutations
,
Compute the sample covariance or correlation matrix of
Compute the test statistic (12) or (15) using
 For to
   Permute the elements in each column of
   Compute the sample covariance or correlation matrix of
   Compute the test statistic (12) or (15) using
 End For
 Report -value
Algorithm 2 Test the identity of covariance or correlation matrix.

In addition to the Pearson’s correlation matrix, the test statistic defined in (15) and its associated Algorithm 2 can handle the Spearman’s and the Kendall’s rank correlation matrices, because this test statistic is based on the cosine measure between a sample correlation matrix and , and does not require any particular types of correlation matrices.

We use the bfi data as described in section 3.1 to illustrate the proposed identity test and its associated Algorithm 2 for correlation matrices. For a Pearson’s correlation matrix of twenty-five personality self reported items, Algorithm 2 produces a p-value of 0.01 based on 100 permutations. The Bartlett’s identity test implemented in the same R package gives a p-value of zero. We also apply the identity test to the Spearman’s rank and Kendell’s rank correlation matrices. Algorithm 2 produces a p-value of 0.01 for both types of correlation matrices based on 100 permutations . Therefore, we reject the identity of these three types of correlation matrices. We will further study the performance of our proposed identity test in Section 5.

3.3 Compound symmetry test

The hypothesis of compound symmetry is for a covariance matrix or for a correlation matrix, where is a square matrix of ones and is the intra-class correlation coefficient. In this case, in the one-sample test statistic (9) is for a covariance and for a correlation matrix, respectively. Using the modified half-vectorization operator, , a unified test statistic for testing the compound symmetry of both covariance and correlation matrices can be obtained from (9) as

(16)

where is either a sample covariance or correlation matrix. The derivation of (16) is provided in Appendix II. Note that this test statistic does not depend on the unknown parameters and .

Suppose a data matrix contains rows that are the realizations of a random vector , in which all ’s have the same distribution. Under the null hypothesis of compound symmetry, it is true that , for all ’s and for all pairs of , provided . This implies that the random permutations of the elements in do not alter its covariance matrix under the assumption of compound symmetry. Using this fact, we propose Algorithm 3 to compute p-values for the test statistic (16) by randomly permuting the elements in each row of the data matrix. For this algorithm to be applicable, each column of the data matrix must have the same distribution.

the number of permutations
,
Compute the sample covariance (correlation) matrix of
Compute the test statistic (16) using
 For to
   Permute the elements in each row of
   Compute the sample covariance (correlation) matrix of
   Compute the test statistic (16) using
 End For
 Report -value
Algorithm 3 Test the compound symmetry of covariance (correlation) matrix.

For an illustrative purpose, we consider the Cork data (rencher2012methods, Table 6.21). Four cork borings, one in each of the four cardinal directions, were taken from each of twenty-eight trees and their weights were measured in integers. The goal is to test the compound symmetry of the covariance matrix of these weight measurements taken in the four cardinal directions. Using the modified half-vectorization operator and the Euclidean inner product and norm, we obtain a cosine of 0.99 (0.998) between the sample covariance (correlation) matrix and the compound symmetry, indicating a strong similarity. We apply our proposed compound symmetry test (16) to this dataset and, using Algorithm 3, obtain a p-value of 0.099 for the compound symmetry test of the sample covariance matrix and 0.069 for the sample correlation matrix based on 100 permutations. Hence, the null hypothesis of compound symmetry can not be rejected at the significance level of 0.05. rencher2012methods, however, rejected the compound symmetry based on a -approximation to a likelihood ratio test using a multivariate normal distribution (rencher2012methods, Example 7.2.3). This discrepancy is due to their inappropriate adoption of the multivariate normal distribution, since three of the four marginal distributions failed to pass the normality test. The test statistic (16) and Algorithm 3, on the other hand, require no distributional assumptions and are robust to the underlying distribution of Cork borings weights.

4 -sample test ()

In this section we will first discuss the test statistics and their associated algorithms for the two-sample problem (=2), then extend the method to handle the -sample problem. Let and be two data matrices with sample covariance (correlation) matrices and , respectively. Suppose the rows of are the realizations of a random vector and the rows of are of random vector . Let () be the covariance (correlation) matrix of and () be the covariance (correlation) matrix of .

In the two-sample problem, the null hypotheses are for covariance matrix and for correlation matrix. We adopt , the Euclidean inner product and norm in the two-sample test statistic (10) to test the equality of covariance matrices, leading to a test statistic

(17)

and , the Euclidean inner product and norm to test the equality of correlation matrices, yielding a test statistic

(18)

where and are the sample covariance or correlation matrices.

We propose Algorithm 4 to compute p-values for these test statistics under the null hypothesis of equality. In this algorithm two data matrices and are stacked to form a new data matrix (). In each permutation the rows of are randomly permuted to generate a permuted data matrix , which is then split into two data matrices of and to compute the test statistic (17) or (18). Algorithm 4 assumes that in and in have the same distribution for all ’s. One advantage of our proposed two-sample test is that and , , need not to have the same distribution.

Under the null hypothesis of equality , the sample covariance matrices of and of can be considered as the realizations of matrix . The rationale behind Algorithm 4 is that the cosine value between and is similar to that of and under the null hypothesis and the permutations provide a good control of the type I error. Under the alternative, the repeated random-mixing rows of and produce and such that the cosine value between the two is bigger than that of and , therefore the test statistics (17) and (18) have good power to reject the null at a pre-determined significance level.

the number of permutations
,
Compute the sample covariance (correlation ) matrix from
Compute the sample covariance (correlation ) matrix from
Compute (17) for covariance or (18) for correlation matrix
stack and
For to
   randomly shuffle the rows of
   the first rows of
   the remaining rows of
   Compute the sample covariance (correlation ) matrix from
   Compute the sample covariance (correlation ) matrix from
   Compute (17) for covariance or (18) for correlation matrix
End For
Report -value =
Algorithm 4 Test for the equality of two covariance (correlation) matrices.

We use the flea beetles data (rencher2012methods, Table 5.5) to illustrate our proposed two-sample equality test. This dataset contains four measurements of two species of flea beetles, Haltica oleraces and Haltica cardorum. To test the equality of two covariance matrices of the four measurements of the two flea beetle species, we apply the proposed two-sample test (17) and obtain a p-value of 0.37 using Algorithm 4 with 100 permutations. Assuming normality and using - and - approximation, the - gives a similar scale p-value of 0.56 rencher2012methods. In this case our proposed equality test (17) agrees with the likelihood ratio based -, and the null hypothesis of the equality of two covariance matrices is not rejected at the significance level of 0.05.

To test the multivariate equality of several covariance or correlation matrices, we consider for the equality of multiple covariance matrices and for the equality of multiple correlation matrices. We propose the test statistic

(19)

where is the test statistic (17) or (18) for the pairwise two-sample comparison of () and () for all possible unique pairs of populations. Let be the data matrices from populations. Algorithm 5 provides a permutation approach to compute p-values under the null hypothesis of the multivariate equality of covariance or correlation matrices.

the number of permutations
,
Compute the test statistic (19) for , , ,
stack , , ,
For to
   randomly shuffle the rows of
   the first rows of
   the second rows of
              
   the remaining rows of
   Compute the test statistic (19) for , , ,
End For
Report -value =
Algorithm 5 Test for the equality of covariance (correlation) matrices.

We use the Rootstock data (rencher2012methods, Table 6.2) to illustrate our proposed -sample test. Eight apple trees from each of the six rootstocks were measured on four variables: 1) trunk girth at 4 years; 2) extension growth at 4 years; 3) trunk girth at 15 years; 4) weight of the tree above ground at 15 years. The null hypothesis is

(20)

for the equality of six covariance matrices of the six rootstocks. The likelihood ratio based - yields a p-value of 0.71 and 0.74 based on - and - approximation rencher2012methods, respectively. We apply the multivariate equality test (19) to this dataset and obtain a p-value of 0.99 using Algorithm 5 with 100 permutations. For this dataset, our method agrees with the -, and the null hypothesis of the multivariate equality of these six covariance matrices is not rejected at the significance level of 0.05.

5 Simulation studies

In this section we carry out extensive simulations to investigate the empirical type I error and power of our proposed one-sample and two-sample tests. In the one-sample setting we investigate the proposed sphericity test and identity test. In the two-sample setting, we investigate the proposed equality test of two covariance matrices. We design and conduct these simulation studies with the backdrop of high dimensions. We also probe into factors that may affect the performance of the proposed methods. All simulations are conducted in the R computing environment rsys.

5.1 One-sample tests

5.1.1 Sphericity test

In this section we evaluate the empirical type I error and power of our proposed sphericity test. We adapt the simulation design and model of chen2010tests for a -dimensional random vector as

(21)

where is a constant matrix with and is a -dimensional random vector with . We further let = . The elements ’s in are IID random variables with a pre-specified distribution. It is worth noting that Model (21) is more general than Model (2.4) in chen2010tests, where they additionally require that

and a uniform bound for the 8th moment for all

’s.

We consider two distributions for : 1) ; 2) Gamma(4,0.5). chen2010tests also considered these two distributions but forcing the mean of Gamma(4,0.5) to be zero in order to meet their requirement of data generation model for their sphericity test.

To evaluate the empirical type I error, we set to simulate data under the null with . To evaluate the power, following chen2010tests, we consider two forms of alternatives: 1) , where , and is the floor function. In this case and is a random vector with elements that are IID or Gamma(4,0.5) random variables. 2) with and . In this case and is a random vector with elements that are IID or Gamma(4,0.5) random variables. We choose the challenging cases in chen2010tests by setting in the first alternative and in the second, where the proposed tests of chen2010tests had lower power in their simulation study.

We apply the sphericity test (12) to a collection of simulated datasets with the varying samples sizes and dimensions as considered in chen2010tests, and compute p-values with 100 permutations using Algorithm 1. We replicate each simulation setting 2000 times and reject the null hypothesis at the significance level of 0.05. Table 1 shows that the proposed sphericity test produces p-values compatible with the nominal level of 0.05 under the null. Table 2 presents the empirical power of our proposed sphericity test under the first alternative, where for a fixed dimension, the empirical power increases quickly with the sample size for both distributions. For a fixed sample size, except for the case with the sample size of twenty, the empirical power increases with the dimension. Table 3 shows that our proposed sphericity test demonstrates superb empirical power, almost 100, under the second alternative.

Sample Size () 20 40 60 80 20 40 60 80
Dimension () Gamma(4, 0.5)
38 4.9 5.2 5.1 5.2 6.2 5.3 4.6 4.7
55 4.8 5.6 5.0 4.5 4.7 5.6 4.2 4.7
89 5.2 5.1 4.5 5.0 5.0 5.2 4.5 4.9
159 4.8 5.2 5.3 4.2 6.0 5.3 4.5 5.1
181 5.2 5.6 5.0 4.4 4.9 4.4 4.7 5.9
331 4.4 4.6 5.0 4.1 4.9 5.5 5.1 4.7
343 5.3 3.4 5.0 5.4 5.8 4.2 5.8 4.2
642 4.8 4.9 5.4 4.8 5.5 4.4 5.6 4.9
Table 1: Empirical type I error (%) of the sphericity test at the 5% significance level
Sample Size () 20 40 60 80 20 40 60 80
Dimension () Gamma(4, 0.5)
38 32 66 91 98 17 48 73 90
55 32 73 94 99 16 52 83 95
89 36 83 98 100 17 64 92 98
159 37 85 99 100 21 67 95 100
181 37 86 99 100 20 68 96 100
331 38 90 100 100 20 74 97 100
343 38 88 99 100 21 73 97 100
642 38 89 100 100 22 73 98 100
Table 2: Empirical power (%) of the sphericity test of vs
Sample Size () 20 40 60 80 20 40 60 80
Dimension () Gamma(4, 0.5)
38 97 100 100 100 93 100 100 100
55 99 100 100 100 97 100 100 100
89 100 100 100 100 99 100 100 100
159 100 100 100 100 100 100 100 100
181 100 100 100 100 100 100 100 100
331 100 100 100 100 100 100 100 100
343 100 100 100 100 100 100 100 100
642 100 100 100 100 100 100 100 100
Table 3: Empirical power (%) of the sphericity test vs

5.1.2 Identity test

In this section we evaluate the performance of the proposed identity test (12) for covariance or correlation matrices. We design a block-diagonal model for a random vector , where is a sub-vector, , such that , and for . The covariance matrix in this block-diagonal model takes the form of , where is a matrix for all ’s and has dimensions of with . We first simulate each , where is a random vector and is a lower triangle matrix from the Cholesky decomposition , where has a structure of . Thus , where is a diagonal matrix whose diagonal elements are the variances of the elements of . Then these four ’s are stacked to obtain the random vector .

We consider two configurations for . In the first configuration, the four ’s are IID and the elements in all ’s have one of the following four distributions: 1) ; 2) Log-normal(0,1); 3) Student ; 4) Gumbel(10, 2) to generate the random vector . In the second configuration, the four ’s are independently, but not identically, distributed and each has a different distribution. The second configuration is referred as a hybrid configuration. The reason to choose these four distributions is to investigate the impact of different types of distributions, such as asymmetric distributions, heavy-tail distributions, and extreme value distributions, on the performance of the proposed identity test (12) and its associated Algorithm 2.

We also consider two scenarios for the sample size: 1) a low-dimensional case with ; 2) a high-dimensional case with . Using the block-diagonal model, we generate data under the null with to evaluate the empirical type I error and under the alternative with to evaluate the power. Using Algorithm 2, we compute p-values based on 100 permutations and reject the null hypothesis at the significance level of 0.05. We replicate each simulation setting 2000 times. Table 4 summarizes the results of the empirical type I error and power of our proposed identity test. It is obvious that the p-values are compatible with the nominal level of 0.05 under the null. The empirical power is close to 100% when the sample size is bigger than the dimension of the data. When the sample size is much smaller than the dimension, the empirical power increases with the increase of the dimension for this block-diagonal model, though there are some variations across different distributions.

Sample Size ()
Dimension () 24 32 64 76 92 100 200 300 500 700 1000
Type I error Type I error
5.5 5.0 5.5 5.0 4.8 5.3 5.5 4.5 5.0 4.1 4.9
Log-normal(0,1) 5.0 4.2 4.4 4.9 5.4 5.8 4.9 4.4 5.2 5.1 5.0
Student 4.2 5.3 4.4 4.8 5.1 4.8 4.9 4.7 5.0 5.3 5.5
Gumbel(10,2) 5.0 4.5 5.2 5.0 4.6 4.9 5.2 4.6 5.2 5.2 5.1
Hybrid 4.6 4.9 4.2 4.4 5.4 4.9 4.5 5.1 5.2 5.1 4.6
Power Power
99 100 100 100 100 13 24 31 47 57 70
Log-normal(0,1) 97 100 100 100 100 39 69 82 93 97 99
Student 99 100 100 100 100 18 32 43 58 68 80
Gumbel(10,2) 99 100 100 100 100 14 28 40 59 68 79
Hybrid 99 100 100 100 100 22 43 57 74 84 90
Table 4: Empirical type I error (%) and power (%) of the identity test using block-diagonal model

5.2 Two-sample test

In this section we study the empirical type I error and power of our proposed two-sample test (17) for the equality of two covariance matrices. We consider two dimensionality configurations: 1) and 2) , where and are the two sample sizes and is the dimension. We adopt the block-diagonal model as in Section 5.1.2 to simulate data in these two dimensional configurations. Under the null hypothesis we set in both samples and under the alternative we set for sample one and for sample two. We apply Algorithm 4 with 100 permutations to compute p-values and reject the null hypothesis at the significance level of 0.05. We replicate each simulation setting 2000 times.

Table 5 shows that the proposed equality test maintains a good control of the empirical type I error at the nominal level of 0.05 under the null hypothesis regardless of the underlying distributions and data dimensions.

Note that 75% of the elements in the covariance matrix are zero under the block-diagonal model. This implies that at least 75% of the elements of and are the same regardless of the value of , which makes it rather difficult to test the equality of two covariance matrices. In fact, for the same data, the two-sample -, which approximates the two-sample likelihood ratio test based on a multivariate normal distribution, produces empirical power: 30%, 30%, 66%, 96% and 100% for the sample sizes of = =100 and the dimensions = 24, 32, 64, 76, 92, respectively. Clearly, the empirical power is not high even for the - based on a correct model. The results of our proposed two-sample tests are reported in Table 5. For the multivariate normal distribution, our proposed method, on average, outperforms the two-sample -. The model with log-normal(0,1) gives the lowest power in the first dimensionality configuration where ==100 (). The model with Gumbel(10, 2) gives the highest power among the four distributions in both configurations. Furthermore, it is interesting to notice that the model with hybrid distributions gives the second highest power in both dimensionality configurations. We discover that the empirical power of our proposed two-sample test is sensitive to the signal-to-noise ratio (SNR) of the underlying distribution, defined as the reciprocal of the coefficient of variation. The model with Gumbel(10,2) has the highest empirical power accompanied with the highest theoretical SNR of 1.70, followed by the model with log-normal(0,1) which has the second highest power with a theoretical SNR of 0.35. The models with and Student have similar low empirical power and both have the theoretical SNR of zero. We will further investigate the impact of SNR in Section 5.3.

Sample Size () ,   , 
Dimension () 24 32 64 76 92 100 200 300 500 700 1000
Type I error Type I error
5.5 4.3 4.2 4.8 4.5 4.6 5.5 4.6 5.1 4.8 4.8
Log-normal(0,1) 4.9 5.0 4.8 5.5 5.3 5.8 4.9 4.4 5.2 5.1 5.0
Student 5.2 4.4 4.3 4.3 5.3 5.0 5.0 4.4 4.4 5.3 3.9
Gumbel(10,2) 5.0 4.7 5.4 5.2 4.7 5.0 4.8 4.8 5.0 4.8 5.1
Hybrid 4.8 4.4 4.5 5.4 5.6 4.5 5.4 5.0 4.3 5.4 5.0
Power Power
46 65 90 92 95 23 30 30 33 34 35
Log-normal(0,1) 9.5 13 30 42 50 24 37 45 47 49 49
Student 29 43 81 85 90 18 28 27 30 33 31
Gumbel(10,2) 90 100 100 100 100 100 100 100 100 100 100
Hybrid 37 65 99 99 100 100 100 100 100 100 100
Table 5: Empirical type 1 error and power of two-sample test

5.3 Factors affecting the performance of the proposed two-sample test

In this section, we probe into some factors that may affect the performance of our proposed two-sample test (17) and its associated Algorithm 4. Define a random vector with for the population one and with for the population two. We recall the assumption required for the proposed two-sample test as

(22)

We first investigate the impact of violating assumption (22) on the empirical type I error. Secondly, we use distributions with different SNR’s to examine the sensitivity of the proposed two-sample test to the change of SNR. Finally, we study the power of the proposed two-sample test under a sparse alternative as in cai2013two, where the two covariance matrices only differ in a very few and fixed number of off-diagonal elements. In this section we focus on the scenarios where the sample size is smaller than the dimension .

5.3.1 Impact of assumption (22)

To investigate the impact of violating assumption (22) on our proposed two-sample test, we adapt the moving average models as in li2012two. To generate and under the null hypothesis, we use the model

(23)

where and for samples one and two, respectively. The two sequences and , , consist of IID random variables. Under the alternative, we generate the sample one using Model (23) and sample two using the following model

(24)

where the sequence consists of IID random variables.

To examine the impact of violating assumption (22) under the null hypothesis, we set to be IID Gamma(4, 0.5) random variables and to be IID Gamma(0.5, ) random variables in Model (23). We choose the smallest sample size of twenty as used in li2012two. We replicate the simulation 2000 times. In each replicate we compute p-values using 100 permutations in Algorithm 4 and reject the null hypothesis at the significance level of 0.05. Table 6 shows that the violation of assumption (22) can result in incorrect empirical type I errors in our proposed two-sample test. On the other hand, when the two samples are generated using the same distribution, the proposed two-sample test produces correct empirical type I errors compatible with the nominal level of 0.05.

5.3.2 Impact of SNR

To investigate the impact of SNR on the power of our proposed two-sample test, we generate using Model (23) in the sample one and using Model (24) in sample two. The sequences and have the same distribution of either Gamma(4, 0.5) or Gamma(0.5,

). The two Gamma distributions have the same variance but different means, which imply that they have different SNR’s. Table

6 shows that the model with Gamma(4, 0.5) has much higher empirical power than that of the model with Gamma(0.5, ). This is in line with our observation in Section 5.2 that higher SNR’s result in higher power in our proposed two-sample test, since the model with Gamma(4, 0.5) has a SNR that is twice bigger than that of the model with Gamma(0.5, ). To investigate the impact of SNR further, we consider an additional simulation study with two samples generated using the distributions of and , respectively, and the former has a SNR that is five times bigger than the latter. We use for the sample sizes and for the dimension. Simulation results in Table 6 show that the empirical power of the model with is about 100%, whereas the power reduces significantly to 27% using the sample generated by the model with