# On the Theory of Covariate-Adaptive Designs

Pocock and Simon's marginal procedure (Pocock and Simon, 1975) is often implemented forbalancing treatment allocation over influential covariates in clinical trials. However, the theoretical properties of Pocock and Simion's procedure have remained largely elusive for decades. In this paper, we propose a general framework for covariate-adaptive designs and establish the corresponding theory under widely satisfied conditions. As a special case, we obtain the theoretical properties of Pocock and Simon's marginal procedure: the marginal imbalances and overall imbalance are bounded in probability, but the within-stratum imbalances increase with the rate of √(n) as the sample size increases. The theoretical results provide new insights about balance properties of covariate-adaptive randomization procedures and open a door to study the theoretical properties of statistical inference for clinical trials based on covariate-adaptive randomization procedures.

## Authors

• 4 publications
• 2 publications
07/25/2018

### Statistical Inference of Covariate-Adjusted Randomized Experiments

Covariate-adjusted randomization procedure is frequently used in compara...
12/08/2019

### Theory on Covariate-Adaptive Randomized Clinical Trials: Efficiency, Selection bias and Randomization Methods

The theocratical properties of the power of the conventional testing hyp...
09/24/2020

### Principles for Covariate Adjustment in Analyzing Randomized Clinical Trials

In randomized clinical trials, adjustments for baseline covariates at bo...
02/08/2022

### Validity of tests for time-to-event endpoints in studies with the Pocock and Simon covariate-adaptive randomization

In the presence of prognostic covariates, inference about the treatment ...
06/20/2021

### Constrained randomization and statistical inference for multi-arm parallel cluster randomized controlled trials

Cluster randomized controlled trials (cRCTs) are designed to evaluate in...
08/13/2020

### Improving the Power of the Randomization Test

We consider the problem of evaluating designs for a two-arm randomized e...
03/25/2022

### Sequential matched randomization and a case for covariate-adaptive randomization

Background: Sequential Matched Randomization (SMR) is one of multiple re...
##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

It is well known that covariates play an important role in clinical trials. Clinical trialists are often concerned about unbalanced treatment arms with respect to key covariates of interest. In the literature, covariate-adaptive randomization procedures (Rosenberger and Lachin, 2002) are sometimes employed to balance on important covariates. Pocock and Simon’s marginal procedure (Pocock and Simon, 1975) is one popularly used method in the literature. As pointed out in Taves (2010), there are over 500 clinical trials which implemented Pocock and Simon’s marginal procedure to balance important categorical covariates from 1989 to 2008. Simulation studies (Weir and Lees, 2003; Toorawa, Adena et al., 2009; Kundt, 2009) found that Pocock and Simon’s marginal procedure indeed reduces marginal imbalances as well as the overall imbalance. But the performance within strata is not as satisfactory (Signorini, Leung et al., 1993; Kundt, 2009). However, all studies of Pocock and Simon’s procedure are merely carried out by simulations. There is “no theoretical justification that the procedure even works as intended” (Rosenberger and Sverdlov, 2008).

Over the past several decades, scientists have identified many new biomarkers (Ashley, Butte, Matthew et al., 2010; Lipkin, Chao et al. , 2010; McIlroy, McCartan et al., 2010; etc.) that may link with certain diseases in the fields of translational research (genomics, proteomics, and metabolomics). Based on these biomarkers, we would like develop personalized medicine algorithms that help patients to receive better treatment regimens based on their individual characteristics (which could be biomarkers or other covariates). To design a superior and efficient clinical study for personalized medicine, one should incorporate information on important biomarkers (Hu, 2012). Therefore, balancing treatment allocation for influential covariates has become increasingly important in today’s clinical trials. As pointed out in Hu (2012), classical covariate-adaptive designs have several drawbacks to incorporate many important biomarkers. Recently, Hu and Hu (2012) developed a class of covariate-adaptive biased coin randomization procedure and studied its theoretical properties. However, condition (C) of their Theorem 3.2 is very strict and hard to verify in practice. More importantly, Hu and Hu’s theoretical results do not apply to Pocock and Simon’s marginal procedure.

In this paper, we establish a general theoretical foundation for covariate-adaptive randomization procedures under widely satisfied conditions. In particular, we have theoretically proved that under Pocock and Simon’s marginal procedure, the marginal imbalances and overall imbalance are bounded in probability, but all the within-stratum imbalances increase with the rate of

as the sample size increases. The theory provides some new insights about theoretical properties of covariate-adaptive randomization procedures. In particular, the theory provides critical conditions for general covariate-adaptive randomization procedures to achieve good within-stratum balance and good marginal balance. As discussed in the concluding remarks (Section 5), our theoretical results also open a door to study the theoretical behavior of inferential methods (estimation, hypothesis testing, etc.) of clinical trials based on covariate-adaptive randomization procedures.

We first propose a general family of covariate-adaptive randomization procedures which includes some new designs and many existing designs as special cases: stratified randomization, Pocock and Simon’s (1975) marginal procedure, Hu and Hu’s (2012) procedures, Efron’s (1971) biased coin design and Baldi Antognini and Giovagnoli’s (2004) adjustable biased coin design, etc. To study the theoretical properties under this general framework, the main difficulties include (i) the correlation structure of within-stratum imbalances; (ii) the relationship among within-stratum and marginal imbalances under Pocock and Simon’s type procedures; and (iii) the discreteness of the allocation function. In the literature, Taylor’s expansion and martingale approximation are two common techniques to study the theoretical properties of adaptive designs with a continuous allocation function (Hu and Zhang, 2004; Hu and Rosenberger, 2006; Zhang, Hu and Cheung, 2006). Under Pocock and Simon’s type procedures, to overcome the complex relationship among within-stratum and marginal imbalances, we have to approximate these imbalances using martingales by solving Poisson’s equations. To deal with the discreteness of the allocation function, we use the technique of “drift conditions” (Meyn and Tweedie 1993), which was developed for Markov chains on general state spaces.

The paper is organized as following. The general framework of the randomization procedure is described in Section 2 and the theoretical results are given in Section 3. We consider multi-arm clinical trials in Section 4. Some concluding remarks are in Section 5. The proofs of the theorems can be found in Section 6.

## 2 The Framework for Covariate-Adaptive Designs

We consider the same setting as that of Pocock and Simon (1975) and only focus on two treatment groups and . Consider covariates and levels for the th covariate, resulting in strata. Let be the assignment of the th patient, , i.e., for treatment 1 and for treatment 2. Let indicate the covariate profile of that patient, i.e., if his or her th covariate is at level , and . For convenience, we use to denote the stratum formed by patients who possess the same covariate profile , and use to denote the margin formed by patients whose th covariate is at level .

The procedure is defined as follows:

1. The first patient is assigned to treatment 1 with probability 1/2.

2. Suppose patients have been assigned to treatments () and the th patient falls within stratum .

3. For the first patients,

• let be the difference between the numbers of patients in treatment group 1 and 2, i.e., the number in group 1 minus the number in group 2;

• similarly, let and be the differences between the numbers of patients in the two treatment groups on the margin , and within the stratum , respectively;

• these differences can be positive, negative or zero, and each one is used to measure the imbalance at the corresponding level (overall, marginal, or within-stratum).

4. If the th patient were assigned to treatment , then would be the “potential” overall difference in the two groups; similarly,

 D(1)n(i;k∗i)=Dn−1(i;k∗i)+1

and

 D(1)n(k∗1,…,k∗I)=Dn−1(k∗1,…,k∗I)+1

would be the potential differences on margin and within stratum , respectively.

5. Define an imbalance measure by

 Imb(1)n=wo[D(1)n]2+I∑i=1wm,i[D(1)n(i;k∗i)]2+ws[D(1)n(k∗1,…,k∗I)]2,

which is the weighted imbalance that would be caused if the th patient were assigned to treatment 1. , and are nonnegative weights placed on overall, within a covariate margin and within a stratum cell, respectively. Without loss of generality we can assume

 wo+ws+I∑i=1wm,i=1.
6. In the same manner we can define , the weighted imbalance that would be caused if the th patient were assigned to treatment 2. In this case, the three types of potential differences are the existing ones minus 1, instead of plus 1.

7. Conditional on the assignments of the first patients as well as the covariates’ profiles of the first patients, assign the th patient to treatment 1 with probability

 P(Tn=1|Zn−1,Zn=(k∗1,…,k∗I),Tn−1) =g(Imb(1)n−Imb(2)n) (2.1)

where , , , is a real function with , ,

 g(x)≤0.5 when x≥0, and limsupx→+∞g(x)<0.5.

Using the basic equation , the critical quantity in Step can be simplified as

 Imb(1)n−Imb(2)n = 4{woDn−1+I∑i=1wm,iDn−1(i;k∗i)+wsDn−1(k∗1,…,k∗I)} := 4⋅Λn−1(k∗1,…,k∗I) (2.2)

Therefore, the allocation probability is determined by the value of , which is a weighted average of current imbalances at different levels. In the literature different views have been given as to the selection of the allocation probability function . Efron (1971), Pocock and Simon (1975), Hu and Hu (2012) suggested

 g(x)=⎧⎪⎨⎪⎩q, if x>0,12, if x=0,p, if x<0, (2.3)

where and . In general, we can define to be either a continuous function or a discrete function.

###### Remark 2.1

Instead of using the biased coin function (2.3) (Efron, 1971; Pocock and Simon, 1975; etc.), we use a general allocation function which is defined as a decreasing function of the weighted average imbalances. When the covariates are not considered, Baldi Antognini and Giovagnoli (2004) suggested a heavy tail function which can reduce both the allocation bias and selection bias of Efron’s biased coin design. We hope that the general framework is flexible enough to define applicable randomization procedures with good properties. The theoretical results will be established under widely satisfied conditions so that they can apply for all cases. In practice, one may use Efron’s biased coin function (2.3) with as discussed and suggested in the literature (Hu and Hu, 2012). However, selection bias could be a concern with a large in Efron’s biased coin function (2.3). To reduce selection bias and allocation bias, one may use the heavy tail function suggested by Baldi Antognini and Giovagnoli (2004).

## 3 Theoretical Properties

We now investigate the asymptotic properties of the design. For the first patients, we know that is the true difference between the two treatment arms within stratum . Let

 Dn=[Dn(k1,…,kI)]1≤k1≤m1,…,1≤kI≤mI

be an array of dimension which stores the current assignment differences in all strata and therefore stores the current imbalances. Also, assume that the covariates are independently and identically distributed. Since can take different values, it in fact follows an -dimension multinomial distribution with parameter , each element being the probability that a patient falls within the corresponding stratum. Obviously, and . Notice

 Dn(k1,…,kI)=Dn−1(k1,…,kI)+2(Tn−12)I{Zn=(k1,…,kI)}.

It is easily seen that

 P(Dn(k1,…,kI)=D1(k1,…,kI)∀n)=1 if p(k1,…,kI)=0.

We can ignore those strata with . Hence without loss of generality, we assume for all .

Our purpose is to the study the properties of . Besides , we will also consider the weighted average of the imbalances as in (2.1). Let

 Λn(k1,…,kI)= woDn+I∑i=1wm,iDn(i;ki)+wsDn(k1,…,kI), Λn= [Λn(k1,…,kI)]1≤k1≤m1,…,1≤kI≤mI.

The allocation probability (2) of the -th patient is a function of . We will find later that plays a very important role for investigating the properties of . It is obvious that

is a linear transform of

. The following proposition gives the relation between and and tells us that both and are Markov chains.

###### Proposition 3.1

(i) If , then is a one to one linear map; If , then each is a linear transform of ; For any case, is a linear transform of ;

(ii) is an irreducible Markov chain on the space with period 2 and with the property that and have the same transition probabilities;

(iii) is an irreducible Markov chain on the space with period 2 and with the property that and have the same transition probabilities.

Now we give our main results.

###### Theorem 3.1

Consider covariates and levels for the th covariate, where , , and . , , and , , are nonnegative with . Then is a positive recurrent Markov chain with period 2 on and for any . In particular,

(i)

If , then is a positive recurrent Markov chain with period 2 on , and for any ;

(ii)

If , then in probability and for any ; Further, if , then the collection of all marginal imbalances is a positive recurrent Markov chain;

(iii)

For any case in probability and for any ; Further, if , then a positive recurrent Markov chain.

###### Remark 3.1

Recently Hu and Hu (2012) obtained theoretical result (i) under very strict condition of the weights and when is defined in (2.3). The Condition (C) in their Theorem 3.2 is very restrictive and usually not satisfied in practice. When the number of strata is large, their Condition (C) can be satisfied only when is very close to and the design reduces closely to stratified randomization. Their results do not apply to Pocock and Simon’s (1975) design (where ) and the design with equal weights . Our Theorem 3.1 eliminates Hu and Hu (2012)’s condition (C) so that it applies to most covariate-adaptive randomization procedures.

The next theorem tell us that the within-stratum imbalances either are bounded in probability or increase with rate as the sample increases.

###### Theorem 3.2

Under the conditions in Theorem 3.1,

(iv)

There exist non-negative constants such that

 ED2n(k1,…,kI)=nσ2(k1,…,kI)+O(√nσ(k1,…,kI)), (3.1)
 Dn(k1,…,kI)√nD→N(0,σ2(k1,…,kI)) (3.2)

and

 limn→∞E∣∣∣Dn(k1,…,kI)√n∣∣∣r=σr(k1,…,kI)E|N(0,1)|r (3.3)

for all stratum s and , where

is a standard normal random variable;

(v)

For any fixed stratum , if in probability, then in probability;

(vi)

If in probability for one stratum , then . In other words, if , then for all stratum

 limn→∞ED2n(k1,…,kI)n=σ2(k1,…,kI)>0.

The main conclusion of Theorems 3.1 and 3.2 can be summarized in the following corollary which indicates that the condition is critical to ensure that is positive recurrent.

###### Corollary 3.1

The following statements are equivalent:

(1)

is a positive recurrent Markov chain;

(2)

in probability;

(3)

for all ;

(4)

in probability for at least one stratum ;

(5)

.

The next theorem tells us that the marginal procedures will not provide good balance with respect to the margin if the margin is not considered in the imbalance measure for defining the allocation probability (2).

###### Theorem 3.3

Suppose the conditions in Theorem 3.1 are satisfied. If , then

 limn→∞E[D2n(i;ki)]n>0 for all % ki=1,…,mi.
###### Remark 3.2

By Theorem 3.1, 3.2 and Theorem 3.3, the conditions and () are critical to ensure that the within stratum and the marginal imbalances in probability respectively. However, we have not discussed the selection of these weights in practice. Here are some suggestions based on the results of this paper: (i) Always choose . (ii) When the sample size is relatively large and the total number of strata is relatively small, there are enough patients in each strata. In these cases, balance within each strata is important and plays an important role, we may choice a relatively large . For example, we may use under these situations. (iii) When the number of covariates () are increasing and the number of strata is relatively large, we may select weights according to the number of covariates () and the important of each covariate. For example, we may select or and according the important of th covariate (). Some simulation studies can be found in Hu and Hu (2012).

It is an interesting observation from Theorem 3.2

that if one of the asymptotic variances

is positive, then all of them are positive, while, if they are zeros, then is bounded in probability. The later will happen only in the case of . When , the design reduces to the marginal procedure which includes Pocock and Simon’s (1975) design as a special case. Based on Theorem 3.1 (ii), Theorem 3.2 (iv) and (vi), and Theorem 3.3, we have the following asymptotic properties of Pocock and Simon’s procedure.

###### Corollary 3.2

For Pocock and Simon’s marginal procedure (), we have the following results:

(a)

All within-stratum imbalances increase with the rate as the sample size increases. Also

is asymptotically normal distributed with a positive variance

.

(b)

When , then the corresponding marginal imbalance (the -th covariate) and the overall imbalance are bounded in probability, that is, and in probability; Further, the collection of all marginal imbalances is a positive recurrent Markov chain with period 2.

(c)

When , then the corresponding marginal imbalance increase with the rate , that is, in probability.

As in Hu and Hu (2012), to prove the Theorem 3.1, we will use the technique of “drift conditions” (Meyn and Tweedie 1993), which was developed for Markov chains on general state spaces. In stead of considering directly as in Hu and Hu (2012), we have to consider in this paper. In order to prove the positive recurrence of we need to find a test function , a bounded test set on , and a positive constants such that

 △λV(Λ):=∑Λ′∈L(Zm)Pλ(Λ,Λ′)V(Λ′)−V(Λ) (3.4)

satisfies the following condition:

 △λV(Λ)≤−1+bIΛ∈C, (3.5)

where is the transition probability from to on the state space of the chain , and is a function with value if is in , and zero if not. is often a norm-like function on

. For considering the convergence of moments of the Markov chain, we will also find the drift condition of

. The test function is the key component in the proofs. We have to choose a good such that it is norm-like function and the drift is also very close to the norm of , so that the drift condition is satisfied without any additional condition on the weights , , and s.

When (Pocock and Simon’s marginal procedure), the within-stratum imbalance is not considered in the allocation procedure. We need to introduce a new technique (Poisson’s equation) to deal with the complicated structure of the within-stratum imbalances and marginal imbalances. In fact, we will approximate as a martingale plus a function of by solving Poisson’s equation in the proof of Theorem 3.2. We will prove that this martingale is a constant when the asymptotic variance is zero and so that is a function of , which is a contradiction when . All the proofs will stated in the Section 6.

## 4 An extension to the multi-arm clinical trials

In some clinical trials, one would like to compare three or more treatments (Pocock and Simon, 1975; Tymofyeyev, Rosenberger and Hu, 2007; Hu, 2012; etc.). In this section we consider clinical trials with () treatments. Let be the assignment of the th patient, , i.e., for treatment . Under the same covariate structure of Section 2, the allocation procedure is defined as follows:

1. The first patient is assigned to treatment with probability .

2. Suppose patients have been assigned to treatments () and the -th patient falls within stratum .

3. For the first patients, let be the number of patients in treatment group t. Further let and be the numbers in treatment group on the margin and within the stratum , respectively. We denote

 NAven−1=1TT∑t=1Nn−1,t, NAven−1(i;k∗i)=1TT∑t=1Nn−1,t(i;k∗i),
 and NAven−1(k∗1,…,k∗I)=1TT∑t=1Nn−1,t(k∗1,…,k∗I)

be the corresponding average numbers over treatments. Define the differences

 Dn−1,t=Nn−1,t−NAven−1, Dn−1,t(i,k∗i)=Nn−1,t(i,k∗i)−NAven−1(i,k∗i), and Dn−1,t(k∗1,…,k∗I)=Nn−1,t(k∗1,…,k∗I)−NAven−1(k∗1,…,k∗I).

These differences are used to measure the overall imbalance, the imbalance on the margin and the imbalance within stratum , respectively, for each treatment .

4. If the -th patient is assigned to treatment , then , and will increase , and others remain unchanged. So the “potential” imbalance at the corresponding level (overall, marginal, and within-stratum) is

 D(t)n,h=Dn−1,h+I{h=t}−1T, D(t)n,h(i;k∗i)=Dn−1,h(i;k∗i)+I{h=t}−1T, and D(t)n,h(k∗1,…,k∗I)=Dn−1,h(k∗1,…,k∗I)+I{h=t}−1T,

for .

5. Define an imbalance measure by

 Imbn,t=T∑h=1{ wo[D(t)n,h]2+I∑i=1wm,i[D(t)n,h(i;k∗i)]2 +ws[D(t)n,h(k∗1,…,k∗I)]2},

which is the weighted imbalance that would be caused if the -th patient were assigned to treatment . Here , () and are nonnegative weights with

6. Having the imbalance measure, we define the allocation probabilities in the same way as Pocock and Simon (1975). One can rank the treatments according to the values of , , in a non-decreasing order so that

 Imbn,(1)≤Imbn,(2)≤…≤Imbn,(T).

In the case of ties a random ordering can be determined. The assigned treatment of the -th patient can be determined from the following set of probabilities

 P(Tn=(t)|Zn−1,Zn=(k∗1,…,k∗I),Tn−1)=pt, (4.1)

where , , , and are ordered nonnegative fixed constants with and .

When , i.e., only the marginal imbalances are considered, the proposed design reduces to Pocock and Simon’s (1975) marginal method.

Let

 Dn=[Dn,t(k1,…,kI)]1≤t≤T,1≤k1≤m1,…,1≤kI≤mI,
 Λn=[Λn,t(k1,…,kI)]1≤t≤T,1≤k1≤m1,…,1≤kI≤mI,

where . The following theorem is the main result for multi-treatment case.

###### Theorem 4.1

Consider covariates and levels for the th covariate, where , , and . , , and , , are nonnegative weights with . Assume that are nonnegative constants with and .

Then is a positive recurrent Markov chain with period and for any . In particular,

(i)

If , then is a positive recurrent Markov chain with period , and for any ;

(ii)

If , then in probability and for any and ; Further, if , then the collection of all marginal imbalances is a positive recurrent Markov with period ;

(iii)

For any case in probability and for any and ; Further, if , then a positive recurrent Markov chain with period .

Further,

(iv)

if , then for any stratum and treatment ,

 limn→∞ED2n,t(k1,…,kI)n>0;
(v)

if , then

 limn→∞E[D2n,t(i;ki)]n>0 for % all ki=1,…,mi,t=1,…,T.

When , the design reduces to the marginal procedure which includes Pocock and Simon’s (1975) design as a special case. Based on Theorem 4.1, we have the following asymptotical properties of Pocock and Simon’s procedure.

###### Corollary 4.1

For Pocock and Simon’s marginal procedure ( and ), we have the following results:

(a)

All within-stratum imbalances increase with the rate as the sample size increases.

(b)

When , then the corresponding marginal imbalance (the -th covariate) and the overall imbalance are bounded in probability, that is, and in probability; Further, the collection of all marginal imbalances is a positive recurrent Markov chain with period .

(c)

When , then the corresponding marginal imbalance increase with the rate , that is, in probability.

## 5 Concluding Remarks

In this paper we study the theoretical properties of a general family of covariate-adaptive designs. These results provide a unified and fundamental theory about balance properties of covariate-adaptive randomization procedures. In the literature, it is well known that the imbalance is a positive recurrent Markov chain for Efron’s (1971) biased coin design (without involving covariates). Markaryan and Rosenberger (2010) studied some exact properties of Efron’s biased coin design. Recently, Hu and Hu (2012) showed that the imbalances are positive recurrent Markov chains for a very limited family of covariate-adaptive designs with Efron’s bias coin allocation function. The condition (C) in their paper is too restrictive and it is almost impossible to check this condition in real applications. The results in this paper also provide new insights about imbalances on covariate-adaptive randomization procedures: (i) when (the within-stratum weight is positive), the imbalances are positive recurrent Markov chains and therefore, all three types of imbalances (within-stratum, marginal and overall) are bounded in probability; (ii) when and , then the marginal (the -th covariate) and overall imbalances are bounded in probability, but the within-stratum imbalance is not; (iii) when for all and , only the overall imbalance is bounded in probability.

It is very important to understand statistical inference under covariate-adaptive randomization. In the literature, several authors (Birkett, 1985, Forsythe, 1987, etc.) have raised concerns about the conservativeness of the unadjusted analysis (such as two-sample t-test) under covariate-adaptive randomization based on simulation studies. Recently, Shao, Yu and Zhong (2010) studied this problem theoretically under a very special covariate-adaptive biased coin randomization procedure, which is a stratified randomization procedure and only applies to a single covariate case. Also they focused on a simple homogeneous linear model and only considered a two-sample t-test. This is because the theoretical properties of covariate-adaptive randomization procedures are usually not available in the literature. The results in this paper open a door to study the theoretical behavior of classical statistical inference under covariate-adaptive randomization. For example, based on Corollary 3.2, we can study the behavior of testing hypotheses and other methods under the Pocock and Simon’s procedure. We leave these as future research projects.

In this paper, we only consider balancing discrete (categorical) covariates. In the literature, continuous covariates are typically discretized in order to be included in the randomization scheme (Taves, 2010). We may apply the proposed designs to balance continuous covariates after discretized. However, as discussed in Scott et. al. (2002), the breakdown of a continuous covariate into subcategories means increased effort and loss of information. Ciolino et. al. (2011) also pointed out the lack of publicity for practical methods for continuous covariate balancing and lack of knowledge on the cost of failing to balance continuous covariates. We may consider balancing continuous covariates under similar framework of this paper. However, it is usually difficult to obtain the corresponding theoretical properties. There is not much study in the literature.

The proposed procedures and their properties may be generalized in several ways. First, we may apply the same idea to problems of unequal ratios (Hu and Rosenberger, 2006). Sometimes, if one treatment is superior (or less costly) than the other, then assigning more patients to the treatment would be more ethical (economical). Second, we may combine the idea in this paper with the ERADE of Hu, Zhang and He (2009) to get a new family of “covariate-adjusted response-adaptive randomization” (CARA) procedure (Zhang, Hu, Cheung and Chan, 2007); it could be a real challenge to study the corresponding theoretical properties. We leave all these as future research topics.

## 6 Appendix: Proofs

Our proofs are based on the properties of Markov chains on a countable state space. For general notations and theory for Markov chains we refer to Meyn and Tweedie (1993). For simplification, we write . Let be the state space of , i.e., each has only one non-zero element which is or . Also let be the history -field generated by the covariates and results of allocation .

Proof of Proposition 3.1. For (i), taking the summation of over all yields

 ∑kΛn(k)=(ws+I∑i=1wm,i∏j≠imj+wom)Dn.

So is a linear transform of . Taking the summation of over all except yields

 ∑k1,…,ki−1,ki+1,…,kIΛn(k1,…,kI) = (ws+wm,i∏j≠imj)Dn(i;ki)+(∑l≠iwm,l∏j≠i,lmj+wo∏j≠imj)Dn.

Hence, when , each is a linear transform of and , and so it is a linear transform of . Finally, when , it is obvious that each is a linear transform of , and , and so it is a linear transform of . Hence, when , is a one to one linear map.

For (ii) and (iii), it is sufficient to show the Markov property. Notice

 Dn(k)=Dn−1(k)+2(Tn−12)I{Zn=k}.

Then

 P(ΔDn(k)=1|Fn−1)= g(4Λn−1(k))p(k), P(ΔDn(k)=−1|Fn−1)= [1−g(4Λn−1(k))]p(k)