First, we recall certain basic definitions
and fix our notations.
For a random variable and evolving on a common probability space, where
First, we recall certain basic definitions and fix our notations. For a random variable, we denote its distribution as . Let be a finite Markov chain on the state space , and with the transition matrix . Following [No97], we abbreviate the Markov chain , with and for , as Markov. A coupling of the Markov chain is a process , with each taking values in
and evolving on a common probability space, wheresatisfies the property that, for , and similarly, satisfies, for , .
Remark A: The definition of Markov chain coupling, as given in [LPW09], says that both and are Markov chains with transition matrix . The reason we prefer the above weaker definition is because in literature we find processes considered as couplings that do not satisfy the stronger notion of [LPW09], but do satisfy our above definition. A Markov chain with transition matrix , by definition, satisfies the property () that for , conditioned on has the distribution and is indepedent of . As we shall see in the next section, there is a Markovian coupling , where does not satisfy the property (), although ’s do evolve (under certain conditions) as per the transition matrix .
Let be and be . In the above, since each pair, and , , is defined on the same probability space, the two random variables, , for each , are coupled, and so we have, from the above definition of Markov chain coupling, and Proposition 4.7 of [LPW09]:
(Here, denotes the total variational distance between two distributions.)
Let us suppose that a coupling of has the ‘now-equals-forever’ property ([Ro97]), namely, if for some , then for all . Then, if is defined as the random time
we get from the relation (1)
as the two events and imply each other, making use of the ‘now-equals-forever’propery.
As such, the applicability of the inequality (3) is limited as most couplings of Markov chains will not have the ‘now-equals-forever’ property. The way that is often resorted to to tackle this problem is to define a new process as follows: ([Ro97])
where is as in (2) above. If it so happens that then, since is a coupling of the Markov chain satisfying ‘now-equals-forever’ property, we have
The construction of has been evocatively termed as sticking and the resultant new coupling as a sticky coupling [HM17].
Definition 1 (Markovian coupling of a Markov chain)
A coupling of a Markov chain on state space , and with its transition
matrix as , where and
, for given and , is said to be a Markovian coupling
if , with is itself a Markov chain on state space ,
with a specified joint distribution of
, with a specified joint distribution ofand as the distribution . In other words, there is a transition matrix, say , such that for some distribution on , satisfying that its marginals are and , we will have, for , with , the two marginals of will be , that is, and , that is .
Remark B: We note that the defintion above is specific to a joint distribution of . The reason for the specificity is that in the example given by Rosenthal ([Ro97]) of a Markovian coupling for which sticking fails, as we note in the next section, the ability to evolve the two copies as per the transition matrix crucially depends on the initial coupling of and .
As noted in [HM17], it has been stated at times that sticking will work for all Markovian couplings. However, this is not correct, Rosenthal [Ro97] provides a counter example. He then provides a stronger version of Markovian couplings, termed as faithful couplings, for which it is proved that sticking will provably result in a coupling of where , as defined in (4) above, will indeed be Markov. We discuss Rosenthal’s counterexample in Section 2 and identify what we consider to be the reason why the sticking operation fails there. This reason then motivates us in defining a stronger version of Markovian coupling (Section 3) for which it is easy to see that the sticking will indeed work. We then prove that the new notion of coupling that we define is actually equivalent to the notion of faithful coupling.
2 Rosenthal’s counterexample
In his example, Rosenthal [Ro97] considers the Markov chain with state space , and with the transition matrix defined as:
are defined as follows:
is the uniform distribution on the state space,
Each is also the uniform distribution, whereas for , being the exclusive-or operation.
Thus, the joint evolution of the two copies of the chain, on the state space , that is, is governed by the following transition matrix :
Convention: The distribtion of a random variable on is specified as the vector
is specified as the vector
and similarly, the distribution of a random variable on as the vector
We note that for any distribution , , will be the uniform distribution. We also note that for the trivial coupling , of and , which is , will again be . Therefore, with the joint distribution of and , indeed defines a Markovian coupling of the Markov chain .
Remark C: We also note that there are other joint distributions of the same and , for which will not be a Markovian coupling: consider , . The marginals of the latter distributions are and respectively. Thus, if we use the joint distribution of the same two initial distributions, while evolves ’s (understandably) as per , it does not evolve ’s as per . Hence with the initial joint distribution , fails to be a Markovian coupling of . This is why we felt that it is necessary to make explicit the initial joint distribution in the definition of Markovian coupling of a Markov chain.
Rosenthal proves that although is a Markovian coupling of with as the initial joint distribution, the result of the sticking operation will fail to evolve correctly: with defined as in (2) and ’s as defined in (4),
However, had ’s been evolving as per with the same initial distribution as the uniform distribution, the probability above would have been , and not . Hence, sticking fails.
We see that sticking failed with . As it happens, sticking will fail here for every value of (except, of course, for ). The reason is provided by the Proposition below. We need the following definition:
For a state space with an element in it, denotes the distribution on in which the probability of is , and (therefore,) all the other states have probability .
Let be a Markov chain with as its state space and as its transition matrix, and let be a transition matrix for the state space that defines a Markovian coupling of where and , where the initial joint distribution used is . This Markovian coupling of can be turned into a sticky coupling using the sticking operation if the following condition holds: for every , defining as the unique joint distribution on with and as its two marginals, and further defining as , for each , we have that for each , will be a joint distribution that has and as its marginals. On the other hand, if does not satisfy the condition, then the -defined Markovian coupling, in general, cannot be turned into a sticky coupling.
Proof Sketch. The condition ensures that both and of the Markovian coupling of satisfy the strong Markov property (Theorem 1.4.2, [No97]) with respect to the stopping time , as defined in (2). Therefore, conditioned on and , each of and will be Markov and the former will be independent of and the latter will be independent of . Thus, and can replace each other. Because of this, sticking is guaranteed to work as and will be indistinguishable.
We now show that the counter example of Rosenthal does not satify the condition as stated in the statement of the proposition. As we have seen that the Markovian coupling there cannot be turned into a sticky coupling, the counter example proves the second part of the Proposition. Let us suppose that and . Conditioned on these events, . The only joint distribution with these two distributions as marginals is . The joint distribution of and will be given by , which is . This gives as , which is not . Thus, fails to ensure that, conditioned as above, . On the other hand, , and so and are different and cannot replace each other, as required for the sticking operation to work.
3 Strong Markovian coupling
We saw in Section 2 that sticking fails for because it does not evolve as per for certain joint distributions. Therefore, the following definition of a type of coupling of Markov chains immediately suggests itself to correct the defect:
Definition 4 (Strong Markovian coupling)
A coupling of a finite Markov chain with state space and transition matrix is a strong Markovian coupling if there is a transition matrix from to such that for every pair of distributions and , each on , and for every joint distribution of and , will be a joint distribution of and .
From this definition it follows that:
Let be a strong Markovian coupling of a finite Markov chain with state space and transition matrix , and for two random variables and on let be and respectively, and let be any joint distribution of and . If we define as , for , and and as two random variables whose distributions are respectively the two marginals of then
will be Markov and will be Markov. Consequently,
and , for ,
Both and will satisfy the strong Markov property, and therefore, the coupling of can be turned into a valid sticky coupling through sticking.
We see next that the strong Markovian coupling is actually equivalent to the familiar notion of faithful coupling which is defined as:
Definition 6 (Faithful coupling [Ro97])
A Markov coupling of a finite Markov chain , with as its state space and as its transition matrix, is a faithful coupling given by a Markov chain , on state space , with transition matrix , if satisfies, for all , the following
Equivalently, for all and all
Remark D: This kind of coupling has been termed in [LPW09], as well as in [MU05], as Markovian coupling; however, we follow the terminology of [HM17] for reasons given therein. It is to be noted that most coupling constructions of Markov chains turn out to be faithful couplings, as the various coupling examples in [LPW09] demonstrate.
A Markov chain coupling is faithful if and only if it is a strong Markovian coupling.
The two lemmas below prove the two directions of the above proposition.
Every faithful coupling of a Markov chain is a strong Markovian coupling.
Proof. Let be a Markov chain with state space and transition matrix . Let the transition matrix , giving transition probabilities from to , define a faithful coupling of the Markov chain . Let and be two distributions on , and let be a joint distribution of and . We need to prove that and are the two marginals of . In particular, we need to show:
We prove (5) as follows:
In a similar manner we can prove (6).
For the other direction, we prove
Every strong Markovian coupling of a Markov chain is a faithful coupling of .
Proof. Let be a Markov chain with state space and transition matrix . Let the transition matrix , giving transition probabilities from to , define a strong Markovian coupling of the Markov of . For any , consider the probabilty distribution . This distribution is the joint distribution and , both defined on . (The definition of as in Definition 2.) is the th row of , namely, . Similarly, will be the th row of , namely, , and will be the th row of , that is, . As defines a strong Markovian coupling of , we have that will be the joint distribution of and . From this condition, we get for any :
Similarly, for any , we get
4 Concluding remarks
We have seen that it is sufficient for a Markovian coupling to have the faithfulness property in order to be turned to a sticky coupling. Is the faithfulness property also a necessary property? It may be that there is a Markovian coupling of a Markov chain which evolves two copies of the chain correctly for a pair of initial distributions of , using a joint distribution of the pair, and satisfies the condition in the statement of Proposition 3, but it either does not work for some other joint distribution of the same pair and , or, does not satisfy the condition for some other pair of initial distributions of . Such a Markovian coupling can be turned into a sticky coupling for the pair, but will not be strongly Markovian, and hence will not be a faithful coupling of . One feels that even if such an example exists, it is unlikely to be a natural example. Faithfulness appears to be the only natural strengthening of the Markovian coupling notion that ensures that the sticking operation will work.
- [HM17] Hirscher, Timo and Anders Martinsson, Segregating Markov chains, manuscript, arXiv:1510.036661v2 [math.PR], 2017.
- [MU05] Mitzenmacher, Michael and Eli Upfal, Probability and Computing: Randomized Algorithms and Probabilistic Analysis, Cambridge University Press, 2005.
- [No97] Norris, J.R., Markov Chains, Cambridge University Press, 1997. Paperbackk edition 1998, 15th printing 2009.
- [LPW09] Levin, David A., Yuval Peres and Elizabeth L. Wilmer, Markov Chains and Mixing Times, American Mathematical Society, 2009.
- [Ro97] Rosenthal, Jeffrey S., Faithful Couplings of Markov Chains: Now Equals Forever, Advances in Applied Mathematics, Vol. 18, pp. 372–381, 1997.