Power Comparison between High Dimensional t-Test, Sign, and Signed Rank Tests

12/27/2018 ∙ by Long Feng, et al. ∙ 0

In this paper, we propose a power comparison between high dimensional t-test, sign and signed rank test for the one sample mean test. We show that the high dimensional signed rank test is superior to a high dimensional t test, but inferior to a high dimensional sign test.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Testing the population mean vector is a fundamental problem in statistics. For univariate data, the classic

test is a very popular method. However, it is not robust and is sensitive to outliers and heavy tailed distributions, and the sign and Wilcoxon signed rank tests are preferable for heavy tailed distributions. For multivariate data, Hotelling’s

test is a natural extension of the test, but is also not robust for heavy tailed distributions. Randles (2000) extended the sign test to a multivariate sign test for elliptical symmetric distributions, and Mttnen and Oja (1995) proposed a multivariate signed rank test and showed it to be very efficient. With the rapid development of technology, various high dimensional datasets have been generated in many areas, such as hyperspectral imagery, internet portals, microarray analysis, and DNA mapping. This article considers high dimensional testing of location parameters where the dimensionality is potentially much larger than the sample size.

In multivariate hypothesis testing, a good estimator of the scatter matrix is a very important for the affine invariant test statistic. However, the sample covariance matrix is not invertible when the dimension,

, exceeds the sample size,

, and classic multivariate tests cannot be applied. Recent research has focused on large scale covariance matrix estimation under certain sparseness conditions (Bickel and Levina, 2008; Cai and Liu, 2011). While it seems reasonable to replace the scatter matrix with sparse matrix estimators, it is difficult to maintain significance levels for those modified test statistics (Feng, Zou, and Wang, 2016). An alternative method is to replace the sample scatter matrix by its diagonal matrix or the identity matrix, and many modified Hotelling’s

tests have been proposed based on this approach, such as Bai and Saranadasa (1996); Chen and Qin (2010); and Feng et al. (2016). However, these methods are all based on the diverging factor model or multivariate normal distributions, and are not efficient for heavy tailed distributions. Wang et al. (2015) proposed a high dimensional nonparametric multivariate test based on multivariate sign for the one sample location problem. Feng, Zou and Wang (2016) also proposed a scalar invariant high dimensional sign test for the two sample location problem. They demonstrated that the multivariate sign is a very efficient method to construct a robust test for high dimensionality.

Multivariate signed rank (Mttnen and Oja, 1995) is another efficient method to construct robust multivariate statistics, but also cannot be used in high dimensionality. We propose a high dimensional signed rank test (SR), replacing the sample scatter matrix in the multivariate signed rank test statistic with the identity matrix. The proposed SR test statistic is shown to be asymptotically normal under elliptical distributions, and simulation comparisons show the SR procedure performs reasonably well in terms of size and power for a wide range of dimensions, sample sizes, and distributions.

For the main contribution of our paper, we derive the explicit form of the asymptotic relative efficiency (ARE) for the SR test relative to the high dimensional test (Chen and Qin, 2010) and high dimensional sign test (Wang, Peng, and Li, 2015). For univariate or multivariate data, the sign and SR tests perform better than the test for heavy tailed distributions. For high dimensional data, the high dimensional sign test and proposed SR test are not worse than the high dimensional -test. For light tailed distributions, the three tests are equivalent, but for heavy tailed distributions, the high dimensional sign and proposed SR test will be more powerful than high dimensional test, which we have verified by simulation.

2 High dimensional signed rank test

Assume are i.i.d. random samples from a variate elliptically symmetric distribution with density function , where are the symmetry centers, and are the positive definite symmetric scatter matrices. Consider the one sample testing problem

For univariate data, the Wilcoxon signed rank test statistic is essentially the sign test statistic applied to the Walsh sums (or averages), for . Similarly, the multivariate one sample signed rank test statistic can be constructed using the signs of transformed Walsh sums (or averages),

where , is the estimator for the scatter matrix, and is a scalar parameter. However, is not available when the dimension is larger than the sample size, and is not well defined. A natural extension is excluding in ,

However, the expectation of is not zero and difficult to calculate. We also exclude the “same” terms, and , and propose the SR test statistic

where ; and denotes summation over distinct indexes, e.g. the summation in is over the set , for all .

This appears to be an calculation for , but is sufficient, since

where

There are calculations for and for . Thus, we need only calculations for .

The following conditions are required for asymptotic analysis.

  1. .

  2. .

(C1) is similar to condition (3.8) in Chen and Qin (2010), and will hold if all the eigenvalues of

are bounded. Define . Similar to Condition (C4) in Feng, Zou and Wang (2016), (C2) is used to reduce the difference between modules and

. Thus, we may obtain an explicit relationship between the variance of

and . Consider a simple setting, , (C2) becomes .

Theorem 1

Under conditions (C1), (C2), and , as ,

where .

We propose a ratio consistent estimator for (Feng, 2015),

which requires calculations. However, one can simply fix , so that requires only calculations. Simulation studies show this modified estimator is similar to the native estimator.

Consider the asymptotic distribution of under the alternative hypothesis

  1. , , where .

(C3) requires that the difference between and is not large such that the variance of is still asymptotic to . This can be viewed as a high dimensional version of the local alternative hypotheses.

Theorem 2

Under conditions (C1)–(C3), as ,

Consequently, the SR asymptotic power becomes

3 Power comparison

Chen and Qin (2010) proposed a high dimensional two sample location (CQ) test by modifying the classic Hotelling’s test. We simplify this to the one sample problem, i.e.,

They showed that the asymptotic power the CQ test was

The SR (ARE) with respect to CQ is

Since ,

and . So,

Thus, the proposed SR test can be no worse than the CQ test. From the Cauchy inequality, is one if and only if , or . Therefore, if converges to , the SR test is asymptotically equivalent to CQ. For example, if , , and SR has the same efficiency as CQ. Otherwise, SR is preferable in terms of asymptotic power under local alternatives, since increases with increasing variance.

Wang et al. (2015) proposed a high dimensional one sample location (SS) test based on spatial sign,

They showed the ARE of their SS test with respect to CQ was

Similarly, if converges to , SS is asymptotically equivalent to CQ. Otherwise, SS will be more powerful than CQ, since increases with increasing variance.

Therefore, the SR ARE with respect to SS is

Define , where , , then ,. So and .

The relation holds only with . Otherwise, SS is more efficient than SR, since increases with increasing variance.

Generally, if , these three tests are equivalent. Otherwise, SS is superior, then SR and finally CQ. Table 1 shows the ARE between the tests under multivariate

distributions with different degrees of freedom and mixed normal distributions.

MN(0.2,3) MN(0.05,10)
ARE(SS,CQ) 2.54 1.76 1.51 1.38 1.18 1.00 1.95 5.79
ARE(SR,CQ) 1.98 1.48 1.31 1.22 1.10 1.00 1.64 5.26
ARE(SR,SS) 0.78 0.84 0.87 0.88 0.93 1.00 0.84 0.91

Note: These results were calculated by simulation with and 10,000 replications.

Table 1: Asymptotic relative efficiencies for different distributions. ( denotes multivariate distributions with degree of freedom ; denotes mixed normal distributions with density function .)

4 Simulation

A simulation study was undertaken to evaluate the performance of the proposed SR test, incorporating 2,500 replications. Three scenarios were considered.

  1. Multivariate normal distribution. .

  2. Multivariate distribution . ’s were generated from with .

  3. Multivariate mixed normal distribution . ’s were generated from , denoted by , where is the density function of -variate multivariate normal distribution. was chosen to be 0.2.

The scatter matrix was . First, consider the low dimensional case . Two sample sizes were evaluated with dimension . Under the alternative hypothesis, two allocation patterns were considered: dense and sparse. These assumed the first and , respectively, of components of to be zero. We set , where is the covariance matrix. Since the empirical sizes of the classic spatial signed rank test (TSR) deviate significantly from the nominal, we propose a size corrected power comparison. The critical values of TSR and SR tests were found through simulation, so both tests have accurate sizes for each case. Table 2 shows the size corrected power of the tests. The proposed SR test is more powerful than TSR when dimension is comparable to sample size. However, classical Mahalanobis distance may not work well because the contamination bias in estimating the covariance matrix grows rapidly with (Bai and Saranadasa, 1996). When and are comparable, using the inverse of the estimate of the scatter matrix in constructing tests is no longer beneficial.

Dense Case Sparse Case
(30,24) (40,32) (30,24) (40,32)
Scenario TSR SR TSR SR TSR SR TSR SR
(I) 9.6 51.8 11.4 65.4 11.8 86.0 16.2 86.8
(II) 11.7 68.2 16.6 84.8 16.0 97.0 21.5 96.8
(III) 15.2 74.0 19.6 88.4 17.2 97.8 25.5 97.9
Table 2: Size corrected power comparison at 5% significance for low dimensionality ().

Consider the high dimensional case, . Two sample sizes and three dimensions were evaluated with . The other settings were the same as the low dimensional case. Table 3

shows the comparison between SR, CQ, and SS tests for empirical size and power. All tests maintain significance well in all cases. For scenario I, SS and SR tests perform similarly to CQ. For multivariate normal distributions, the difference between direction and moment based tests disappears with increasing dimensionality. However, when the underlying distribution is not normal, direction based tests are superior to moment based tests. For scenarios II and III, both SS and SR tests are significantly superior to CQ, and asymptotic relative efficiencies are close to the theoretical results from Table

1. As expected, the proposed SR test outperforms CQ, but is slightly inferior to SS for multivariate and mixed normal distributions.

Size Dense Sparse
CQ SS SR CQ SS SR CQ SS SR
Scenario I
30 100 4.7 5.6 5.7 26.9 30.0 29.6 33.2 37.4 36.4
30 200 5.0 6.8 6.3 26.4 29.9 29.3 29.8 31.9 32.8
30 400 4.8 6.0 5.8 26.3 30.0 29.3 29.0 32.5 31.9
40 100 4.5 5.8 4.8 38.0 40.2 39.6 46.4 49.1 48.0
40 200 4.9 6.2 5.8 38.5 41.2 41.1 42.6 45.1 45.4
40 400 5.3 6.2 6.2 37.1 40.5 40.4 41.9 44.4 43.7
Scenario II
30 100 5.5 5.6 5.2 31.2 50.5 42.7 39.6 63.0 55.0
30 200 4.5 6.8 5.9 31.1 55.4 44.9 34.8 61.4 50.2
30 400 4.5 6.0 5.6 29.0 56.2 43.8 32.1 59.5 48.9
40 100 4.3 5.8 5.2 41.3 67.7 58.4 49.1 80.1 67.8
40 200 5.0 6.2 5.5 42.7 69.6 59.6 48.8 77.3 67.0
40 400 5.8 6.2 6.5 41.7 72.2 62.7 45.7 75.7 65.5
Scenario III
30 100 5.1 5.6 5.6 29.7 56.0 47.5 36.8 68.6 59.8
30 200 4.9 6.8 5.8 29.3 60.3 48.8 32.8 67.9 56.6
30 400 4.8 6.0 5.3 29.6 62.3 53.8 31.2 64.5 55.4
40 100 5.0 5.8 5.0 39.4 72.2 62.9 45.7 85.1 74.9
40 200 4.7 6.2 6.4 42.8 75.8 66.1 45.7 82.2 71.9
40 400 4.0 6.2 5.5 39.8 77.9 68.8 44.3 81.4 72.2
Table 3: Empirical sizes and power ()at 5% significance for scenarios I–III

To give a broader picture of the robustness and efficiency of the proposed method, we also considered the diverging factor models (Bai and Saranadasa, 1996), where . Two distributions were considered,

  1. .

  2. .

Random vectors from scenarios IV and V are not elliptically distributed, and Table 4 shows the simulated results for these scenarios with the same settings as Table 3. SR controls empirical size very well in these cases. SR test power is also lager than CQ and a little smaller than SS. Thus, the proposed SR test performs very well for non-elliptical distributions.

The outcomes verify the proposed SR test is efficient and robust across a wide range of distributions. When the distributions are heavy tailed, SR is significantly more efficient than moment based tests. This may be due to the proposed test using only the observation direction from the origin, rather than its distance from the origin, which would tend to be more for heavy tailed distributions.

Size Dense Sparse
CQ SS SR CQ SS SR CQ SS SR
Scenario IV
30 100 5.6 7.4 6.1 26.6 30.4 29.1 31.5 37.2 36.4
30 200 3.6 5.8 5.7 27.5 31.0 30.5 30.8 33.4 33.3
30 400 4.5 5.5 5.7 24.9 30.0 29.2 26.7 31.3 29.8
40 100 5.7 7.3 6.3 39.5 42.7 41.6 46.9 51.9 50.1
40 200 5.6 6.9 6.7 35.5 38.6 38.0 38.3 42.1 41.5
40 400 6.0 6.9 6.9 39.3 43.1 42.8 41.2 44.1 44.1
Scenario V
30 100 4.5 7.1 6.0 29.5 55.4 46.6 33.9 66.1 56.7
30 200 5.7 6.1 6.0 30.1 58.6 48.5 34.3 63.5 53.3
30 400 3.7 6.4 5.3 30.9 58.8 49.8 30.8 62.6 51.7
40 100 6.5 7.1 6.0 42.2 73.4 65.4 50.4 84.0 74.5
40 200 6.3 7.0 6.2 43.2 76.4 66.3 45.4 80.8 70.7
40 400 4.6 4.9 4.5 39.0 75.6 65.7 42.4 79.4 69.7
Table 4: Empirical sizes and power ()at 5% significance for scenarios IV–V

5 Discussion

Many high dimensional scalar invariant tests have been proposed (Park and Ayyala, 2013; Srivastava, 2009; Feng, et al., 2015; Feng, 2015). The concept is to replace by its diagonal matrix in , so that all variables have the same scale. How to construct an SR scalar invariant test will be the topic of further study.

Acknowledgement

This research was supported by the National Natural Science Foundation of China (grant No. 11501092) and the Fundamental Research Funds for the Central Universities.

6 Appendix: Theorem Proofs

Define and , where the constant depends on the background distribution, ;, ; and . Recall Zou, et al., 2014, Lemma 4:

Lemma 1

Suppose are i.i.d. uniform on the unit sphere. For any symmetric matrix ,

Recall, also, Feng, 2015, Lemma 3.

Lemma 2

as .

6.1 Proof of Theorem 1

From the above definitions and lemmas,

We need to show that .

Under , , and from the definition of , , . Therefore,

We need to show that and .

Define , then . From symmetry of , and Lemma 1, . Similarly, . Therefore,

Since ,

and . Therefore,

From Lemma 2, as , so that. Thus, .

Similarly, .

Finally, we need to show that

Define , where , and let be the field generated by . Since , it follows that

is a zero mean martingale. The central limit theorem will hold if

(1)

and, for any ,

(2)

It can be shown that

and since , then . From Lemma 1, . Thus, from C1, . Thus, .

Similarly, . Thus, (1) holds.

To prove (2), by Chebyshev’s inequality, we need only show that

Note that

which can be decomposed as , where

Since, from Lemma 1 and (C1), then . Similarly, . This completes the proof.

6.2 Proof of Theorem 2

From Taylor’s expansion,

Then,

Using the same procedure as Theorem 1,

Since