Testing for Stochastic Order in Interval-Valued Data

11/17/2019 ∙ by Hyejeong Choi, et al. ∙ Seoul National University 0

We construct a procedure to test the stochastic order of two samples of interval-valued data. We propose a test statistic which belongs to U-statistic and derive its asymptotic distribution under the null hypothesis. We compare the performance of the newly proposed method with the existing one-sided bivariate Kolmogorov-Smirnov test using real data and simulated data.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

We discuss the two-sample tests for stochastic order of two interval-valued samples. In the interval-valued data, the variable of interest is not observed as a single point but is displayed in the form of an interval, with lower and upper bounds. For example, interval-valued data is observed when stock price is reported monthly by lower and upper limit prices. In addition, blood pressure data, which motivates our research, has diastolic blood pressure (DBP) and systolic blood pressure (SBP) as lower and upper bounds. It is a fundamental problem in statistics to test the stochastic order of two populations as well as to verify the equality of the two distributions. However, little research has been done for the interval-valued data; even definition of stochastic order for interval-valued data is not clearly established. Thus, this paper introduces its definition and proposes a method to test the stochastic order of two samples of interval-valued data.

The remainder of the paper is organized as follows. In section 2, we define the stochastic order of interval-valued data. In section 3, we propose a test statistic for testing the order of interval-valued data and derive its asymptotic null distribution using the general theory on U-statistic. In section 4, we examine the performance of the modified two-dimensional Komogorov-Smirov(K-S) statistic and the proposed through a numerical study. In section 5, we apply the methods to the blood pressure data from female students in the US. In section 6, we conclude the paper with a summary.

2 Simple stochastic order

Before we introduce the notion of the stochastic order for interval-valued data, we look at the stochastic order for the usual univariate case. Let and

be two univariate random variables such that

Then, is said to be stochastically greater than (denoted by ). If additionally for some , then is said to be stochastically strictly greater than (Shaked and Shanthikumar, 2006).

The stochastic order for interval-valued data can be defined similarly. Let and be two intervals. Then we denote and say is greater than if and . Now, let and be two random intervals such that

(1)

Then, is said to be stochastically greater than and denoted by . Let and be the survival functions of the random intervals and , respectively. Then, (1) is equivalent to

Figure 1: A graphical illustration of the order of interval-valued data. Region A, B, C, and D are respectively defined by intersection of the half-plane with the first, second, third, and fourth quadrant when is set as the origin.

We can illustrate the order of the intervals as follows (see Figure 1). Let the interval denoted by the point in the plane. Note that in the plane, interval-valued data is displayed at the top of the line due to the constraint . Any interval-valued data of the half-plane belongs to any of three cases according to the order relation with the interval .

  1. region A: intervals are greater than .

  2. region C: intervals are less than .

  3. region B or D: intervals do not have an order relation with .

For the last case, an interval in region B satisfies , while an interval in region D satisfies .

3 Test statistic

Let us consider two independent samples of random intervals. Suppose that a first sample , , has a survival function and the second sample , , has a survival function . We want to verify the null hypothesis that both samples come from an identical distribution, “: for all ” against to the alternative hypothesis that is stochastically strictly greater than , i.e., “ for all and for some interval ”.

The statistic we propose to test the stochastic order is

(2)

where

Note that under the null , and thus .

The statistic belongs to a class of U-statistics, which allows one to derive its asymptotic null distribution based on the asymptotic theory of the U-statistic. We introduce below a general asymptotic theory of U-statistics reported in Chapter 6 of Lehmann (1999). Let be a symmetric kernel of () arguments. Here, the symmetric kernel denotes a function whose value does not change by changing the order of arguments or . Let defined below be a parameter of interest;

and define its U-statistic by

(3)

where is the collection of all subsets of with size and dummy indices running over summations are and , respectively.

is an unbiased estimator of

and its variance is

where is given by

and and are independent copies of and . The theorem below from Chapter 6 of Lehmann (1999) explains the asymptotic distribution of the U-statistic (3) above.

Theorem 1 (Lehmann(1999), Theorem 6.1.3 (ii)).

As and ,

converges in distribution to the normal distribution with mean

and variance . Here, and are computed by

Applying the general theory above for U-statistics to our case, we can derive the asymptotic null distribution of our statistic.

Theorem 2.

Under the null hypothesis that , if as , then

where , , and .

Parameters used to compute the asymptotic variance can be approximated by permuting observations within each sample. To understand it, we observe the followings.

(4)

where are independent random intervals from the first population. Consequently, can be approximated by

Equation (4) has an implication that the above approximation would be a valid estimate of even under the alternative hypothesis.

Proof.

For , let us define . Then, can be presented by a two sample U-statistic when ;

Therefore, by applying Theorem 1, we have

where , , , and .

Now, let us denote interval random variables by , , and . Under the null hypothesis , we have . The variance component (= ) is evaluated as

Now, we write , , and . Thus, under , we get

Hence, the asymptotic variance of is

4 Numerical study

In this section, we compare the power of our proposed test (denoted as “U-test”) to one-sided bivariate K-S test (denoted as “K-S test”). U-test can be classified by how its null distribution is approximated. “U-perm” designates U-test where we approximate the null distribution by a permutation method, while “U-asym” is the one depending on the approximation given in Theorem

2. K-S test for the alternative hypothesis is given by (Feller, 1948)

where and . The null distribution of is approximated using a permutation method (Gail and Green, 1976).

In the study, to generate interval-valued data , we consider a transformation to obtain and half-range . We consider two underlying distributions for ; bivariate normal distribution and bivariate

distribution with the degrees of freedom

.

For two populations, we consider and parameterized as follows;

Figure 2: A graphical illustration of two populations in the simulation study

For , the following four values are used : where indicates the alternative hypothesis. Figure 2 shows the graphical illustration of the simulation setting. To examine the effect of correlation between the center and range, we use three values for . The significance level is set as . The size and power are evaluated as the rejection rate among replicates. The number of permutations to generate a null distribution is set as . For the sample size , we consider following 4 cases: (30, 30), (30, 120), (50, 50), (50, 200).

max width= case U-perm U-asym B-KS U-perm U-asym B-KS U-perm U-asym B-KS (N) 0.0 0.045 0.044 0.042 0.043 0.042 0.041 0.045 0.045 0.040 0.3 0.301 0.293 0.158 0.307 0.306 0.178 0.425 0.425 0.289 0.5 0.573 0.568 0.321 0.599 0.598 0.366 0.789 0.788 0.630 1.0 0.980 0.979 0.829 0.988 0.988 0.900 0.999 0.999 0.995 0.0 0.049 0.047 0.052 0.051 0.050 0.058 0.050 0.050 0.046 0.3 0.396 0.393 0.267 0.422 0.420 0.312 0.578 0.578 0.489 0.5 0.745 0.744 0.551 0.781 0.780 0.619 0.929 0.928 0.876 1.0 0.999 0.999 0.979 1.000 1.000 0.991 1.000 1.000 1.000 0.0 0.055 0.055 0.042 0.054 0.054 0.040 0.049 0.049 0.040 0.3 0.411 0.412 0.252 0.436 0.439 0.287 0.589 0.590 0.476 0.5 0.756 0.757 0.525 0.790 0.792 0.605 0.936 0.937 0.873 1.0 0.999 0.999 0.973 1.000 1.000 0.992 1.000 1.000 1.000 0.0 0.052 0.051 0.040 0.052 0.050 0.047 0.057 0.056 0.048 0.3 0.557 0.556 0.378 0.602 0.590 0.462 0.775 0.776 0.709 0.5 0.904 0.903 0.733 0.925 0.922 0.831 0.987 0.987 0.975 1.0 1.000 1.000 0.999 1.000 1.000 1.000 1.000 1.000 1.000 (T) 0.0 0.055 0.052 0.042 0.050 0.050 0.045 0.050 0.042 0.047 0.3 0.239 0.238 0.171 0.253 0.265 0.188 0.334 0.354 0.271 0.5 0.467 0.488 0.302 0.491 0.518 0.346 0.663 0.664 0.542 1.0 0.934 0.936 0.752 0.949 0.952 0.810 0.991 0.991 0.919 0.0 0.052 0.048 0.053 0.051 0.048 0.048 0.053 0.044 0.046 0.3 0.349 0.324 0.225 0.370 0.351 0.246 0.479 0.476 0.344 0.5 0.650 0.634 0.419 0.685 0.676 0.446 0.843 0.844 0.632 1.0 0.987 0.988 0.817 0.993 0.992 0.849 1.000 1.000 0.867 0.0 0.053 0.052 0.044 0.049 0.050 0.044 0.046 0.044 0.040 0.3 0.350 0.333 0.215 0.367 0.362 0.246 0.490 0.486 0.361 0.5 0.661 0.650 0.426 0.686 0.691 0.490 0.852 0.849 0.687 1.0 0.993 0.993 0.852 0.996 0.998 0.881 1.000 1.000 0.893 0.0 0.050 0.054 0.052 0.049 0.048 0.051 0.051 0.046 0.056 0.3 0.482 0.494 0.278 0.499 0.494 0.306 0.690 0.644 0.453 0.5 0.845 0.860 0.517 0.863 0.860 0.563 0.965 0.958 0.708 1.0 1.000 1.000 0.825 1.000 1.000 0.834 1.000 1.000 0.843

Table 1: Simulation results for the stochastic order tests. The power of each test is displayed. The first column denotes the distribution of : “N” indicates the normal case and “T” indicates distribution with df . “U-perm” and “U-asym” represent two types of U-tests, where the former approximates the null distribution by a permutation method and the latter by the asymptotic result in Theorem 4.2. B-KS denotes the bivariate K-S test.

Table 1 shows some interesting findings with regard to the proposed U-test. First, the power of our U-test is higher than the one-sided K-S test in all cases under consideration regardless of the magnitude of

. Also, it is noted that when it comes to U-test, the powers based on a permutation method and asymptotic results are almost same in all cases, which proves the asymptotic result and its accuracy. Third, the greater the correlation between center and range, the higher power of each test we can get. This phenomenon can be explained using the Mahalanobis distance between two mean vectors from the null and the alternative. The distance is

, which is increasing in terms of . Specifically, when is , , and , the corresponding distance is , and , respectively.

5 Data example

In this section, we apply the stochastic order tests to a real dataset. The data we use is obtained from National Heart, Lung, and Blood Institute Growth and Health Study (NGHS), which is a -year cohort study to evaluate the temporal trends of cardiovascular risk factors, such as systolic and diastolic blood pressures (SBP, DBP) based on annual visits of 2,379 African-American and Caucasian girls. The blood pressure (BP) data, which is measured at two levels, can be an example of the MM-type interval-valued data. In this analysis, we only use BP measurements at the first visit and remove subjects with missing values. After all, the total number of subjects is , where Caucasians and African-American girls are and , respectively. The goal of this application is to test a hypothesis “BP of African-American is stochastically greater than that of Caucasian girls”.

Caucasian African-American p-value
mid-BP 78.67 (9.09) 80.13 (8.03)
DBP 56.72 (12.19) 58.03 (11.72)
SBP 100.62 (9.28) 102.23 (8.65)
half-range 21.95(5.89) 22.10 (6.44)
Table 2: Descriptive statistics of the BP data by race. Mean and standard deviation (in parenthesis) are given. “p-value” at the last column is the p-value of t-test on the alternative hypothesis “the BP of African-American is higher than that of Caucasian”.

Table 2 shows that SBP, DBP, and their center are significantly higher in African-American than in Caucasian. Meanwhile, it is confirmed that there is no difference in the range between two groups. These results are very similar to the setting of the numerical study, where centers of two groups are similar, but ranges are different.

Now, we verify whether the BP of African-American is stochastically greater than that of Caucasian based on interval-valued data, instead of marginal distributions. Table 3 presents test results of previously compared methods. In all tests, the p-values are smaller than 0.001, which ensures that the BP of African-American is stochastically greater than that of Caucasians.

U-perm U-asym B-KS
p-value
Table 3: Two-sample order tests for the BP data. “U-perm” and “U-asym” are U-tests depending, respectively, on a permutation method and the asymptotic result in Theorem 4.2. B-KS denotes the bivariate K-S test.

6 Conclusion

In this paper, we introduce the notion of stochastic order between two samples of interval-valued data and propose a test statistic based on U-statistic. We compute the asymptotic null distribution of the proposed statistic. The numerical study shows that the asymptotic distribution approximates the null distribution with accuracy, even with small size of samples. Also, the proposed test has higher power than the one-sided bivariate KS test in all cases we consider. Therefore, it can be said that the procedure proposed in this paper is of great use for testing the order of interval-valued data.

Notes

Authors want to inform that this manuscript is an English version of the article written in Korean and accepted at The Korean Journal of Applied Statistics.

Acknowledgements

We would like to show our gratitude to two anonymous reviewers and the editor of The Korean Journal of Applied Statistics for their detailed and instructive comments.

References

  • Blanco-Fernandez and Winker (2016) Blanco-Fernández, A. and Winker, P. (2016). Data generation processes and statistical management of interval data. AStA Advances in Statistical Analysis, 100(4), 475-494.
  • Feller (1948) Feller, W. (1948). On the Kolmogorov-Smirnov limit theorems for empirical distributions. The Annals of Mathematical Statistics, 19(2), 177-189.
  • Lehmann (1999) Lehmann, E.L. (1999). Elements of Large Sample Theory. Springer.
  • Gail and Green (1976) Gail, M., and Green, S. (1976). Critical values for the one-sided two-sample Kolmogorov-Smirnov Statistic. Journal of the American Statistical Association, 71(355), 757-760.
  • Shaked and Shanthikumar (2006) Shaked, M. and Shanthikumar, J.G. (2006). Stochastic Orders. Springer.