Nonparametric tests for transition probabilities in nonhomogeneous Markov processes

04/06/2019 ∙ by Giorgos Bakoyannis, et al. ∙ 0

This paper proposes nonparametric two-sample tests for the direct comparison of the probabilities of a particular transition between states of a continuous time nonhomogeneous Markov process with a finite state space. The proposed tests are a linear nonparametric test, an L2-norm-based test and a Kolmogorov-Smirnov-type test. Significance level assessment is based on rigorous procedures, which are justified through the use of modern empirical process theory. Moreover, the L2-norm and the Kolmogorov-Smirnov-type tests are shown to be consistent for every fixed alternative hypothesis. The proposed tests are also extended to more complex situations such as cases with incompletely observed absorbing states and non-Markov processes. Simulation studies show that the test statistics perform well even with small sample sizes. Finally, the proposed tests are applied to data on the treatment of early breast cancer from the European Organization for Research and Treatment of Cancer (EORTC) trial 10854, under an illness-death model.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Continuous time nonhomogeneous Markov processes with a finite state space are important in many areas of science and particularly in medicine and public health (Tattar and Vaman, 2014; Bakoyannis et al., 2019). Consideration of specific transitions between two states of a multi-state process can provide a deeper and more detailed insight about the treatment effect in clincal trials compared to the analysis of standard survival outcomes, such as event-free survival (Le-Rademacher et al., 2018). Important special cases of a Markov process are the univariate survival model, the competing risks model, and the Markov illness-death model (Andersen et al., 2012).

The stochastic behaviour of a Markov process can be described by either the transition intensities, which represent the instantaneous rates of transition between two states, or the transition probabilities. The transition probabilities are also known as survival functions in the framework of the univariate survival model, and as cumulative incidence functions in the competing risks model. It is important to note that, in general, a difference in the transition intensities between two groups does not necessarily imply a difference in the corresponding transition probabilities and vice versa. This phenomenon has been well documented for the special case of the competing risks model (Gray, 1988; Pepe, 1991; Putter et al., 2007; Bakoyannis and Touloumi, 2012). Nonparametric tests for comparing transition intesities between groups in general Markov multi-state processes have been well developed (Andersen et al., 2012). However, the issue of nonparametric comparison of transition probabilities in general Markov multi-state processes has not received much attention. Nevertheless, transition probabilities, unlike transition intensities, directly quantify clinical prognosis (Bakoyannis et al., 2019), which is the target of scientific interest in many applications.

Nonparametric estimation of the transition probabilities of a general Markov process can be performed using the Aalen–Johansen estimator

(Aalen and Johansen, 1978). The issue of nonparametric comparison of transition probabilities under the univariate survival model has be extensively studied in the literature. For a review of these methods see Kalbfleisch and Prentice (2011) and Andersen et al. (2012). A number of researchers have proposed nonparametric tests for the comparison of transition probabilities for the special case of the competing risks model (Gray, 1988; Pepe and Mori, 1993; Lin, 1997). Dabrowska and Ho (2000) proposed a graphical procedure based on simultaneous confidence bands to test for differences between transition probabilities in a general Markov process. However, their method imposes proportional hazards assumptions for the transition intensities and, thus, it is not fully nonparametric. Also, this approach does not provide the actual level of statistical significance. Tattar and Vaman (2014) proposed two nonparametric tests for the comparison of the whole transition probability matrices between groups, by comparing all the possible transition intensities. The first test only compares the transition probability matrices at a specific time point , while the second test is a Kolmogorov–Smirnov-type test based on the supremum norm. However, the tests proposed by Tattar and Vaman (2014) do not provide a direct comparison of the transition probability of a particular transition, which is frequently of scientific interest (Le-Rademacher et al., 2018). A statistically significant result with these tests only indicates a difference in any transition between groups. Recently, Bluhmki et al. (2018) proposed a wild bootstrap approach for the Aalen–Johansen estimator, which can be used to construct a simultaneous confidence band for the difference between the transition probabilities of two independent groups. This approach, which is related to a Kolmogorov–Smirnov-type test, can be used as a graphical two-sample comparison procedure at a predetermined level. However, this approach does not provide the actual level of statistical significance and, also, a Komogorov–Smirnov-type test may not be the most powerful nonparametric test for every alterantive hypotheses. Additionally, there is no rigorous justification about the consistency of this graphical hypothesis testing procedure against any fixed alternative hypothesis (Van der Vaart, 2000). Last but not least, the proposed approach is not readily adaptable to more complex situations such as cases with missing data.

This paper addresses the issue of direct nonparametric two-sample comparison of the transition probabilities of a particular transition in a general continuous-time nonhomogeneous Markov process with a finite state space. For this, we propose a linear nonparametric test, an -norm-based test and a Kolmogorov–Smirnov-type test. The asymptotic null distributions of the tests are derived. The evaluation of the actual level of statistical significance is based on rigorous procedures justified through the use of modern empirical process theory. Moreover, the -norm-based and Kolmogorov–Smirnov-type tests are shown to be consistent against any fixed alternative hypothesis (Van der Vaart, 2000). We also propose extensions related to interesting partical problems such as cases with missing absorbing states (Bakoyannis et al., 2019) and non-Markov processes (Putter and Spitoni, 2018). The proposed tests exhibit good small sample properties as illustrated in our simulation experiments. Finally, the tests are applied to data on the treatment of early breast cancer from the European Organization for Research and Treatment of Cancer (EORTC) trial 10854.

Compared to the previous work by Bluhmki et al. (2018), which used counting process theory arguments in their derivations, we justify the properties of the proposed tests through the use of modern empirical process theory (Van Der Vaart and Wellner, 1996; Kosorok, 2008). As it will be argued later in the text, the practical advantage of our derivations lies on the fact that our proposed tests can be straightforwardly adapted to more complex settings such as cases with incompletely observed absorbing states (Bakoyannis et al., 2019). This can be done by replacing the influence function of the standard Aalen–Johansen estimator with the influence function of any other well-behaved and asymptotically linear estimator of the transition probabilities in our proposed testing procedures. Such adaptations are not trivial within the framework of the graphical testing procedure proposed by Bluhmki et al. (2018). An important reason for this is that with more complex estimators, certain predictability conditions assumed by counting process and martingale theory techniques are violated. For such situations, empirical process theory provides a powerful alternative tool. Moreover, we provide two additional tests, a linear test and an -norm-based test, which may be more powerful compared to a Kolmogorov–Smirnov-type test in certain settings. Additionally, we argue about the consistency of our -norm-based and Kolmogorov–Smirnov-type tests against any fixed alternative hypothesis. Finally, our tests provide the actual level of statistical significance which is useful in pactical applications.

The structure of this paper is as follows. In Section 2 we introduce some notation about Markov processes, provide the proposed nonparametric tests, and consider extensions to more complex situations that are frequently met in practice. Section 3 presents a simulation study to evaluate the small sample performance of the proposed tests. Section 4 illustrates the use of the proposed tets using data from the EORTC trial 10854. Finally, Section 5 conlcudes the article with some key remarks. Outlines of the asymptotic theory proofs are provided in the Appendix.

2 Two-sample nonparametric tests

2.1 Nonparametric estimation of transition probabilities

The stochastic behaviour of a Markov process with a finite state space can be described by the transition probability matrix whose elements are the transition probabilities

where is the event history prior to time , with being the number of direct transitions from state to state , , in , , and , , is the transition intensity at time . The conditional independence between the probability of and the prior history , conditionally on , is the so-called Markov assumption. Because

is a stochastic matrix we have that

.

The observed data from a sample of i.i.d. observations of a Markov process are the counting processes , , which represent the number of direct transitions of the th observation from the state to the state by time , and the at-risk processes which are the indicator functions of whether the th observation is at the state just before time . Based on such a sample, the transition probability matrix of a nonhomogeneous Markov process can be estimated using the Aalen–Johansen estimator (Aalen and Johansen, 1978):

where is the product integral and a matrix whose elements are the Nelson–Aalen estimates of the cumulative transition intensities

2.2 Linear nonparametric tests

First consider the two-sample problem of comparing the transition probabilities and , , of two populations of interest, for a particular transition , with . For simplicity of presentation we will set the starting point for the remainder of the paper. Based on two independent random samples of and observations from the two populations, define the pointwise weighted difference

where is a weight function and and are the nonparametric Aalen–Johansen estimates of the transition probabilities of the two populations under comparison. Example of weight function choices are and

where ,

. The latter choice assigns more weight to times with more observations at risk. A natural linear test for the null hypothesis

is the area under the weighted difference curve

where is the Lebesgue measure on the Borel -algebra . To establish the asymptotic distribution of the test statistic we assume the following conditions.

  • The potential right censoring and left truncation are independent of the counting processes and noninformative about .

  • as .

  • The counting processes are bounded in the sense that for some constant .

  • for all the transient states .

  • The cumulative transition intensities are continuous functions of bounded variation on .

  • The weight converges uniformly to a nonnegative uniformly bounded function on .

Remark 1.

In some applications condition C4 may not be satisfied for some timepoints for one or more states . In such cases one can restrict the comparison interval to with , such that for those . In such cases the test statistic becomes

Before stating the theorem about the asymptotic distribution of test statistic we define the functions

where and are the counting and at-risk processes of the th observation in the th sample at time . Also, define to be the subset of which contains the potential absorbing states. The set will be null for non-absorbing Markov processes.

Theorem 1 provides the asymptotic distribution of under the null hypothesis.

Theorem 1.

Suppose that conditions C1-C6 hold. Then under the null hypothesis

where and

with

for .

Remark 2.

The functions , p=1,2, in Theorem 1 are the influence functions of the Aalen–Johansen estimator.

A consistent (in probability) estimator of the variance

is

where , , are estimated by replacing the expectations with sample averages and the unknown parameters with their uniform consistent estimates. Now, Theorem 1 and can be used to constuct a -test for the null hypothesis as:

The actual significance level can then be avaluated under the standard normal distribution as usual.

2.3 -norm-based and Kolmogorov–Smirnov-type tests

A linear test is not the optimal choice when the two transition probability curves under comparison cross at one or more timepoints. In this section, we propose alternative tests for such situations. The first test is a test based on an norm

while the second test is a Kolmogorov–Smirnov-type test

The Kolmogorov–Smirnov-type test is related to the graphical hypothesis testing procedure proposed by Bluhmki et al. (2018). The asymptotic null distributions of these tests are complicated. However, significance level can be easily calculated numerically by proper simulation realizations from the null distribution of these test statistics. Theorem 2 provides the basis for an approach to properly simulate realizations from the null distributions of and . Before stating Theorem 2 define the estimated functions

where , , are independent draws from .

Theorem 2.

Suppose that conditions C1-C6 hold. Then under the null hypothesis

and, conditionally on the observed data,

where and are two independent tight zero-mean Gaussian processes with covariance functions

Corollary 1.

By Theorem 2 and the continuous mapping theorem it follows that under the null hypothesis

and

The asymptotic null distributions of the omnibus tests are quite complicated and, thus, they are of limited use in terms of evaluating the significance level. However, Theorem 2 provides justification about a way to numerically calculate -values through a simple simulation technique. This can be performed as follows. In light of Theorem 2, one can simulate from the asymptotic null asymptotic distributions of the tests and by simulating multiple versions of and independently from for , and then calculating a sample for the above null distributions as

and , , respectively, where

Now, the significance level for each test can be calculated as the proportion of realizations from the corresponding null distribution that is greater than or equal to the calculated tests statistic value from the observed data.

The tests and are consistent for every fixed alternative hypothesis with . This follows from Theorem 2, the uniform consistency of the Aalen–Johansen estimator of the transition probabilities (Aalen and Johansen, 1978), condition C6, the continuity of these tests in , and Lemma 14.15 in Van der Vaart (2000).

2.4 Extensions to more complex settings

Many complications that frequently occur in practice make the application of the proposed tests improper. An important example is the problem of incompletely observed absorbing states, where missingness occurs either due to the usual nonresponse or the study design (Bakoyannis et al., 2019). A special case of this is the issue of missing causes of death in biomedical applications. In such cases, a complete case analysis, which discards cases with a missing cause of death, is well known to lead to biased estimates (Gao and Tsiatis, 2005; Lu and Liang, 2008; Bakoyannis et al., 2019). In general, more complicated cases require extensions of the standard Aalen–Johansen estimator, denoted by , to consistently estimate the transition probabilities of interest over a compact interval . In such cases, one can replace the standard Aalen–Johansen estimator with another approriate estimator in the testing procedures. Then, the linear test becomes

where

while the -norm based and Kolmogorov–Smirnov-type tests become

and

The following conditions ensure the validity of the proposed testing procedures in more complex settings.

  • The estimator is consistent in the sense

    for some , where is a compact subset of .

  • The estimator is an asymptotically linear estimator with

    where the influence functions belong to a Donsker class indexed by .

  • The empirical versions of the influence functions satisfy

    where are independent random draws from .

Remark 3.

Condition D2 is sufficient for establishing the weak convergence of the estimator

to a tight mean-zero Gaussian process. Condition D3 along with the conditional multiplier central limit theorem

(Van Der Vaart and Wellner, 1996; Kosorok, 2008) and condition D2, provide a simulation approach for the construction of simultaneous confidence bands (Kosorok, 2008). Therefore, conditions D1-D3 are expected to have been established in works extending the standard Aalen–Johansen estimator to more complex settings. This is the case, for example, for the nonparametric estimator of the transition probability matrix with incompletely observed absorbing states (Bakoyannis et al., 2019).

Hypothesis testing in more complex settings can be simply performed by replacing the influence functions , , of the standard Aalen–Johansen estimator with the influence functions of the estimator . The theorems stated below justify the direct use of the proposed tests in more complex situations. Before stating those theorems define the functions

where , , are independent draws from .

Theorem 3.

Suppose that conditions C2, C6, D1 and D2 hold. Then under the null hypothesis

where and

The proof of Theorem 3 involves the same arguments to those used in the proof of Theorem 1 given in the Appendix.

Theorem 4.

Assume that conditions C2, C6, and D1–D3 are satisfied. Then, under the null hypothesis

and, conditionally on the observed data,

where and are two independent tight zero-mean Gaussian processes with covariance functions

The proof of Theorem 4 follows from similar arguments to those used in the proof of Theorem 2 given in the Appendix.

2.4.1 Missing absorbing states

In many settings one can observe that a process has arrived at some absorbing state, but the actual absorbing state is unobserved for some study participants, such as in cases with missing causes of death. For such situations, Bakoyannis et al. (2019) proposed a nonparametric maximum pseudolikelihood estimator (NPMPLE) under a missing at random assumption. To review this estimator, let be an indicator variable with if the th observation arrived at the absorbing state , and otherwise. Also, let be another indicator variable with indicating that the absorbing state of the th observation has been successfully ascertained. Finally, let be the probability that given the fully observed data

, under a parametric model indexed by an unknown Euclidean parameter

. In this setting, the cumulative transition intensities can be estimated using the NPMPLE:

where

with being a consistent estimator of . The transition probability matrix can then be estimated as

where the components of the matrix are . By Theorems 1 and 2 in Bakoyannis et al. (2019) and calculations provided in the proof of Theorem 2 in the same source, the NPMPLE estimator satisfies the conditions D1-D3 above. Therefore, if the conditions in Bakoyannis et al. (2019) and the conditions C2 and C6 above are satisfied, two-sample comparison can be performed by utilizing the NPMPLE of the transition probabilities along with the corresponding influence functions in the proposed tests. This is justified by Theorems 3 and 4 above.

2.4.2 Non-Markov processes

Trivially, the Aalen–Johansen estimator is uniformly consistent for the transition probability even under a non-Markov process (Datta and Satten, 2001; Titman, 2015). When the interest lies on the marginal , i.e. unconditionally on the prior history , for some , under a non-Markov process, then the landmark Aalen–Johansen estimator is consistent for (Putter and Spitoni, 2018) under the conditions of Datta and Satten (2001) and, also, the assumption that . The landmark Aalen–Johansen estimator is essentially equivalent to the standard Aalen–Johansen estimator, except for the fact that only observations with are considered. This is achieved by considering the modified counting and at-risk processes and , for . Therefore, the influence functions of the landmark Aalen–Johansen estimator are the same to that of the standard Aalen–Johansen estimator, with the only exception that the former involves the modified and instead of the standard counting and at-risk processes and . Consequently, it is clear that conditions D1–D3 are satified if and, also, if the conditions in Datta and Satten (2001) hold. Thus, in light of Theorems 3 and 4, the proposed nonparametric tests can be used with non-Markov processes by utilizing the landmark Aalen–Johansen estimator.

3 Simulation studies

To evaluate the finite sample performance of the proposed test statistics, we conducted a simulation study. We considered a nonhomogeneous Markov process with 2 transient states and 1 absorbing state , under the illness-death model without recovery (Andersen et al., 2012). This model is illustrated in Figure 1. In this simulation study, we focused on the null hypothesis . Initially, we independently generated the times from state 1 to states 2 and 3 by assuming the cumulative transition intensities and . For observations that first arrived at the transient state 2, we generated the time from state 2 to the absorbing state 3, assuming a cumulative transition intensity . The parameter values considered were and . Then, the right censoring times were independently simulated from Exp(0.25). Under this set-up the transition probability of interest was

Different simulation scenarios were considered according to the sample sizes , and the parameter values of the two groups. 1,000 datasets were simulated for each scenario, and the distance test and Kolmogorov–Smirnov-type test were calculated using 1,000 independent simulations of and from . Finally, the weight function was considered in all cases.

Figure 1: Illness-death model without recovery assumed in the simulation study.

Simulation results regarding the empirical type I error rates are presented in Tables 

1 and 2, respectively. Under these scenaria, the empirical type I errors rates for all tests were close to the nominal levels, even in situations with small sample sizes. Thus, these results provide numerical evidence for the validity of the proposed hypothesis testing procedures under . Simulation results regarding the empirical power levels under alternative hypotheses with non-crossing transition probabilities are presented in Table 3. Under these scenaria, the empirical power levels increased with sample size and, also, with a more pronounced difference between the two groups, as expected. The power levels for the three tests were in general similar. However, under a less pronounced difference between the two groups, the linear test exhibited a somewhat larger empirical power with larger sample sizes. These results provide numerical evidence for the consistency of the proposed tests with non-crossing transition probabilities. Simulation results regarding the empirical power levels under alternative hypotheses with crossing transition probabilities are presented in Table 4. These scenaria illustrate numerically the inconsistency of the linear tests with crossing transition probabilities, as the empirical power levels did not systematically increase with sample size. On the contrary, the empirical power of the -norm-based and Kolmogorov–Smirnov-type tests increased with sample size and with a more pronounced difference between the two groups. These results indicate numerically the consistency of the omnibus tests against alternatives with crossing transition probabilities.

scenario Linear KS Linear KS
50 50 0.009 0.007 0.015 0.051 0.054 0.063
100 50 0.010 0.009 0.018 0.053 0.051 0.047
100 100 0.012 0.011 0.012 0.060 0.060 0.054
200 100 0.013 0.014 0.008 0.049 0.047 0.045
200 200 0.010 0.012 0.010 0.051 0.047 0.052
50 50 0.014 0.013 0.015 0.067 0.061 0.066
100 50 0.014 0.014 0.017 0.057 0.053 0.065
100 100 0.016 0.015 0.013 0.058 0.057 0.056
200 100 0.011 0.014 0.009 0.058 0.060 0.057
200 200 0.008 0.014 0.016 0.060 0.055 0.062
Table 1: Simulation results about empirical type I error rates for the linear test (Linear), the -norm-based test (), and the Kolmogorov–Smirnov-type test (KS) under simulation scenaria 1 and 2.
scenario Linear KS Linear KS
50 50 0.016 0.014 0.011 0.075 0.069 0.066
100 50 0.014 0.018 0.013 0.052 0.048 0.060
100 100 0.011 0.012 0.012 0.048 0.047 0.046
200 100 0.009 0.009 0.012 0.051 0.050 0.054
200 200 0.013 0.015 0.013 0.053 0.051 0.058
50 50 0.012 0.016 0.016 0.064 0.062 0.076
100 50 0.012 0.012 0.013 0.060 0.059 0.067
100 100 0.010 0.013 0.016 0.058 0.056 0.051
200 100 0.007 0.008 0.010 0.044 0.054 0.058
200 200 0.010 0.011 0.018 0.049 0.051 0.054
Table 2: Simulation results about empirical type I error rates for the linear test (Linear), the -norm-based test (), and the Kolmogorov–Smirnov-type test (KS) under simulation scenaria 3 and 4.
scenario Linear KS Linear KS
50 50 0.074 0.069 0.055 0.170 0.162 0.166
100 50 0.085 0.084 0.075 0.210 0.200 0.205
100 100 0.106 0.101 0.101 0.251 0.240 0.240
200 100 0.143 0.142 0.127 0.307 0.296 0.307
200 200 0.233 0.206 0.207 0.448 0.417 0.397
50 50 0.183 0.177 0.178 0.361 0.348 0.370
100 50 0.246 0.230 0.228 0.442 0.438 0.445
100 100 0.341 0.331 0.328 0.556 0.549 0.564
200 100 0.458 0.464 0.480 0.703 0.696 0.708
200 200 0.665 0.677 0.665 0.847 0.861 0.875
Table 3: Simulation results about empirical power levels for the linear test (Linear), the -norm-based test (), and the Kolmogorov–Smirnov-type test (KS) under simulation scenaria 5 and 6.
scenario Linear KS Linear KS
50 50 0.015 0.033 0.033 0.064 0.099 0.123
100 50 0.016 0.028 0.033 0.064 0.111 0.135
100 100 0.017 0.056 0.060 0.074 0.164 0.169
200 100 0.018 0.066 0.058 0.094 0.204 0.203
200 200 0.026 0.102 0.105 0.095 0.304 0.288
50 50 0.016 0.055 0.138 0.055 0.247 0.356
100 50 0.018 0.091 0.189 0.064 0.326 0.441
100 100 0.010 0.205 0.325 0.049 0.554 0.598
200 100 0.016 0.351 0.490 0.062 0.712 0.795
200 200 0.007 0.658 0.743 0.057 0.919 0.927
Table 4: Simulation results about empirical rejection rates for the linear test (Linear), the distance test (), and the Kolmogorov–Smirnov-type test (KS) under simulation scenaria 7 and 8

4 Data analysis

In this section we analyze the data on treatment of early breast cancer from the European Organization for Research and Treatment of Cancer (EORTC) trial 10854. This randomized clinical trial was conducted to evaluate whether the combination of surgery with polychemotherapy is benefical to early breast cancer patients compared to surgery alone. The original analysis of this clinical trial was presented in Van der Hage et al. (2001).

In this trial, 1619 patients where randomly assinged to the surgery group and 1559 to the surgery plus polychemotherapy group. The data set contains information about the time to cancer relapse or death. Therefore, an illness-death model is a natural choice for this data set. It is important to note that the transition probability to relapse, which was not analyzed in the original analysis of this trial, is a non-monotonic function of time as patients can move to the “death” state after relapse. Thus, standard survival and competing risks analysis methods are not applicable for this transition probability. Here, we focus on this probability which can be interpreted as the probability of being alive and in relapse. The estimated transition probabilities of relapse in the two intervention groups are presented in Figure 2. Based on Figure 2, the probability of being alive and in relapse was lower in the group that received polychemotherapy during surgery. To perform hypothesis testing here we considered the weight function . For the -norm-based and Kolmogorov–Smirnov-type tests we considered 1,000 standard normal simulation realizations. The -value from the linear test was 0.001, while the -values from the -norm-based and Kolmogorov–Smirnov-type tests were 0.001 and 0.004, respectively. These results provide evidence for the superiority of the surgery plus polychemotherapy combination with respect to the transition probability of relapse, in early breast cancer patients.

Figure 2: Transition probabilities of being alive in relapse by intervention group in the EORTC Trial 10854.

5 Concluding remarks

This paper addressed the issue of direct nonparametric two-sample comparison of transition probabilities , , for a particular transition in a continuous time nonhomogeneous Markov process with a finite state space. The proposed tests were a linear nonparametric test, an -norm-based test and a Kolmogorov–Smirnov-type test. Rigorous approaches to evaluate the significance level grounded on modern empirical process theory were provided. Moreover, the -norm-based and Kolmogorov–Smirnov-type tests were argued to be consistent against any fixed alternative hypothesis. We also considered extensions of the tests to more complex situations such as cases with missing absorbing states (Bakoyannis et al., 2019) and non-Markov processes (Putter and Spitoni, 2018). The simulation study provided numerical evidence for the validity of the proposed testing procedures, which exhibited good performance even with small sample sizes. Finally, a data analysis of a clinical trial on early breast cancer illustrated the utility of the proposed tests in practice.

The issue of nonparametric comparison of transition probabilities in general nonhomogeneous Markov processes has received little attention in the literature. To the best of our knowledge, the only fully nonparametric approach for comparing the transitions probabilitis for a particular transtion in general non-homogeneous Markov processes is a graphical procedure proposed by Bluhmki et al. (2018). This proposal is based on the construction of a simultaneous confidence band for the difference between the transition probabilities of two groups. However, this approach does not provide the exact level of statistical significance. Also, the justification of this approach was based on counting process theory arguments and not on modern empirical process theory. A concequence of that is that this approach cannot be directly adapted to more complex settings that are frequently occur in practice, such as cases with missing absorbing states. An important reason for this is that with more complex estimators, certain predictability conditions assumed by counting process and martingale theory techniques are violated. On the contrary, our proposed methods can be trivially adapted to many other complex settings, provided that appropriate estimators, in the sense of conditions D1–D3, of the transition probabilities exist. Such adaptations can be theoretically justified using the Theorems 3 and 4 provided in our manuscript.

The proposed tests can be easily adapted for the comparison of state occupation probabilities , as these are simple linear combinations of the transition probabilities. The state occupation probabilities describe the marginal behavior, i.e. unconditional on the prior history, of the processes and are of interest in many applications, such as in HIV studies focusing on the event history of patients in HIV care (Lee et al., 2018). It is important to note that these probabilities can be consistently estimated using the Aalen–Johansen estimator even in non-Markov processes (Datta and Satten, 2001). It is not hard to justify conditions D1–D3 for the state occupation probabiltities under a set of weak regularity conditions. Thus, Theorems 3 and 4 provide a rigorous justification about the use of the proposed tests for comparing state occupation probabilities.

Acknowledgement

This project was supported, in part, by the Indiana Clinical and Translational Sciences Institute funded, in part by Grant Number UL1TR002529 from the National Institutes of Health, National Center for Advancing Translational Sciences, Clinical and Translational Sciences Award. We would like to thank the European Organisation for Research and Treatment of Cancer (EORTC) for sharing with us the data from the EORTC trial 10854. The content of this manuscript is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health and the EORTC.

References

  • Aalen and Johansen (1978) Aalen, O. O. and S. Johansen (1978).

    An empirical transition matrix for non-homogeneous markov chains based on censored observations.

    Scandinavian Journal of Statistics 5(3), 141–150.
  • Andersen et al. (2012) Andersen, P. K., O. Borgan, R. D. Gill, and N. Keiding (2012). Statistical models based on counting processes. Springer Science & Business Media.
  • Bakoyannis and Touloumi (2012) Bakoyannis, G. and G. Touloumi (2012). Practical methods for competing risks data: a review. Statistical Methods in Medical Research 21(3), 257–272.
  • Bakoyannis et al. (2019) Bakoyannis, G., Y. Zhang, and C. T. Yiannoutsos (2019). Nonparametric inference for Markov processes with missing absorbing state. Statistica Sinica In press.
  • Bluhmki et al. (2018) Bluhmki, T., C. Schmoor, D. Dobler, M. Pauly, J. Finke, M. Schumacher, and J. Beyersmann (2018). A wild bootstrap approach for the Aalen–Johansen estimator. Biometrics 74(3), 977–985.
  • Dabrowska and Ho (2000) Dabrowska, D. M. and W.-t. Ho (2000). Confidence bands for comparison of transition probabilities in a markov chain model. Lifetime Data Analysis 6(1), 5–21.
  • Datta and Satten (2001) Datta, S. and G. A. Satten (2001).

    Validity of the Aalen–Johansen estimators of stage occupation probabilities and Nelson–Aalen estimators of integrated transition hazards for non-Markov models.

    Statistics & Probability Letters 55(4), 403–411.
  • Gao and Tsiatis (2005) Gao, G. and A. A. Tsiatis (2005).

    Semiparametric estimators for the regression coefficients in the linear transformation competing risks model with missing cause of failure.

    Biometrika 92(4), 875–891.
  • Gray (1988) Gray, R. J. (1988). A class of -sample tests for comparing the cumulative incidence of a competing risk. The Annals of Statistics 16(3), 1141–1154.
  • Kalbfleisch and Prentice (2011) Kalbfleisch, J. D. and R. L. Prentice (2011). The statistical analysis of failure time data, Volume 360. John Wiley & Sons.
  • Kosorok (2008) Kosorok, M. R. (2008). Introduction to empirical processes and semiparametric inference. Springer.
  • Le-Rademacher et al. (2018) Le-Rademacher, J. G., R. A. Peterson, T. M. Therneau, B. L. Sanford, R. M. Stone, and S. J. Mandrekar (2018). Application of multi-state models in cancer clinical trials. Clinical Trials 15(5), 489–498.
  • Lee et al. (2018) Lee, H., J. W. Hogan, B. L. Genberg, X. K. Wu, B. S. Musick, A. Mwangi, and P. Braitstein (2018). A state transition framework for patient-level modeling of engagement and retention in hiv care using longitudinal cohort data. Statistics in Medicine 37(2), 302–319.
  • Lin (1997) Lin, D. (1997). Non-parametric inference for cumulative incidence functions in competing risks studies. Statistics in Medicine 16(8), 901–910.
  • Lu and Liang (2008) Lu, W. and Y. Liang (2008). Analysis of competing risks data with missing cause of failure under additive hazards model. Statistica Sinica 18(1), 219–234.
  • Pepe (1991) Pepe, M. S. (1991). Inference for events with dependent risks in multiple endpoint studies. Journal of the American Statistical Association 86(415), 770–778.
  • Pepe and Mori (1993) Pepe, M. S. and M. Mori (1993). Kaplan–meier, marginal or conditional probability curves in summarizing competing risks failure time data? Statistics in Medicine 12(8), 737–751.
  • Putter et al. (2007) Putter, H., M. Fiocco, and R. B. Geskus (2007). Tutorial in biostatistics: competing risks and multi-state models. Statistics in Medicine 26(11), 2389–2430.
  • Putter and Spitoni (2018) Putter, H. and C. Spitoni (2018). Non-parametric estimation of transition probabilities in non-Markov multi-state models: The landmark Aalen–Johansen estimator. Statistical Methods in Medical Research 27(7), 2081–2092.
  • Tattar and Vaman (2014) Tattar, P. N. and H. Vaman (2014). The -sample problem in a multi-state model and testing transition probability matrices. Lifetime Data Analysis 20(3), 387–403.
  • Titman (2015) Titman, A. C. (2015). Transition probability estimates for non-Markov multi-state models. Biometrics 71(4), 1034–1041.
  • Van der Hage et al. (2001) Van der Hage, J., C. van De Velde, J.-P. Julien, J.-L. Floiras, T. Delozier, C. Vandervelden, L. Duchateau, et al. (2001). Improved survival after one course of perioperative chemotherapy in early breast cancer patients: long-term results from the european organization for research and treatment of cancer (eortc) trial 10854. European Journal of Cancer 37(17), 2184–2193.
  • Van der Vaart (2000) Van der Vaart, A. W. (2000). Asymptotic statistics. Cambridge University Press.
  • Van Der Vaart and Wellner (1996) Van Der Vaart, A. W. and J. A. Wellner (1996). Weak convergence and empirical processes with applications to Statistics. Springer.

Appendix A Outlines of proofs

Outlines of the proofs of Theorems 1 and 2 are provided below. The proofs of Theorems 3 and 4 follow from similar arguments and, therefore, are omitted. The proofs rely on empirical process theory techniques (Van Der Vaart and Wellner, 1996; Kosorok, 2008). Before providing the proofs it is useful to introduce some notation. First, let be the sample space, and an arbitraty sample point in . Now, define , for some measurable function . Also, define to be the expectation of under the probability measure on the measurable space , where is a -algebra on . For simplicity, but without loss of generality, we set the starting point in the following proofs. It has to be noted that conditions C1 and C3–C5 imply the uniform consistency of the standard Aalen–Johansen estimator. This can be shown using similar arguments to those used in the proof of Theorem 1 in Bakoyannis et al. (2019).

a.1 Proof of theorem 1

Clearly, Theorem 1 relies on the asymptotic linearity of the estimators , . This can be established by first showing the asymptotic linearity of the Nelson–Aalen estimators of the cumulative transition intensities and then by utilizing the functional delta method (Van der Vaart, 2000). The steps to achieve this utilize conditions C1 and C3–C5 and arguments similar to those used in the proof of Theorem 2 of Bakoyannis et al. (2019). After this analysis it can be shown that

with

By Lemma 1 in the supplementary material of Bakoyannis et al. (2019) and arguments similar to those use in the proof of Theorem 2 of the same source, it follows that the influence functions , , belong to -Donsker classes of functions. Now, it is not hard to see that under the null hypothesis and by conditions C2 and C6

Finally, the statement of Theorem 1 follows as a result of the usual central limit theorem and the independence between the two terms, as a consequence of the fact that the two samples are independent.

a.2 Proof of theorem 2

Due to the asymptotic linearity of the transition probability estimators , for , as argued in the proof of Theorem 1, along with conditions C2 and C6, it follows that

Now, by the Donsker property of the class of functions and the uniform boundedness of the class of fixed functions , it follows that is also a -Donsker class. Therefore, by the independence between the two samples, it follows that

where and are two independent tight zero-mean Gaussian processes with covariance functions

Now, define

where , , are independent random draws from . By the Donsker property of the class