In parameterized probabilistic concurrent systems, a population of agents, each typically modeled as a finite-state probabilistic program, run concurrently in discrete time and update their states based on probabilistic transition rules. The interaction is governed by an underlying topology, which determines which agents can interact in one step, and a scheduler, which picks the specific agents involved in the interaction. Concurrent probabilistic systems arise as models of distributed algorithms [1, 2, 3, 4, 5], where each agent is a processor, the interaction between processors is determined by a communication topology, and the processor can update its internal state based on the communication as well as randomization. In each step, the scheduler adversarially chooses a processor to run. Concurrent probabilistic populations also arise in agent-based population models in biology , wherein an agent can represent an allele, a cell, or a species, and the interaction between agents describes how these entities evolve over time. For a population of a fixed size, there is a rich theory of probabilistic verification [7, 8, 9, 10]
based on finite-state Markov decision processes (MDPs). Verification questions for population models, however, ask if a property holds for populations ofall sizes: even if each agent is finite-state, the family of all processes (for each population size) is an infinite-state MDP. Indeed, for many simple population models, one can show that the verification question is undecidable, even for reachability or safety properties in the non-probabilistic setting [11, 12, 13]. Consequently, the verification question for populations requires techniques beyond finite-state probabilistic verification, and requires symbolic techniques to represent potentially infinite sets of states.
One well-known symbolic framework for verifying parameterized non-probabilistic concurrent systems is regular model checking [14, 15, 16, 17, 18, 19], where states of a population are modeled using words over a suitable alphabet, sets of states are represented as regular languages, and the transition relation is defined as a regular transducer. From parameterized verification of non-probabilistic processes, it is known that regular languages provide a robust symbolic representation of infinite sets, and automata-theoretic algorithms provide the basis of checking safety or termination properties.
In this paper, we consider the problem of verifying that a given parameterized family of probabilistic concurrent systems almost surely terminates, i.e., reaches certain final states with probability 1 from each initial state regardless of the behaviour of the schedulers. Termination is a fundamental property when verifying parameterized probabilistic systems. Since termination typically, however, fails without imposing certain fairness conditions on the scheduler, it is crucial to be able to incorporate fairness assumptions into a termination analysis. Therefore, although the framework of regular model checking has recently been extended for proving termination (without fairness) over parameterized probabilistic concurrent systems , it still cannot be used to prove termination for many interesting parameterized probabilistic concurrent systems.
What notion of fairness should we consider for proving termination for parameterized probabilistic concurrent systems? To answer this question, one would naturally start by looking at standard notions of fairness in probabilistic model checking , which asserts that every process must be chosen infinitely often. However, this notion seems to be too weak to prove termination for many of our examples, notably Herman’s self-stabilizing protocol  in an asynchronous setting, and population models from biology (e.g. Moran’s process ). The standard notion of fairness gives rise to a rather unintuitive and unrealistic strategy for the scheduler, which could delay an enabled process for as long as it desires while still being fair (see [21, Example 8] and the Herman’s protocol example in Section 3). For this reason, we propose to consider Alur & Henzinger’s  finitary fairness — a stronger notion of fairness that allows the scheduler to delaying executing an enabled process in an infinite run for at most steps, for some unknown but fixed bound . Alur & Henzinger argued that this fairness notion is more realistic in practice, but it is not as restrictive as the notion of -fairness, which fixes the bound a priori. In addition, it should be noted that finitary fairness is strictly weaker than probabilistic fairness (scheduler chooses processes randomly) for almost-sure termination over finite MDPs and parameterized probabilistic systems (an infinite family of finite MDPs). We will show in this paper that there are many interesting examples of parameterized probabilistic concurrent systems for which termination is satisfied under finitary fairness, but not under the most general notion of fairness.
Contributions. Our main contribution is a systematic, regularity-preserving, encoding of finitary fairness in the framework of regular model checking for parameterized probabilistic concurrent systems. More precisely, our encoding reduces the problem of verifying almost sure termination under finitary fairness to almost sure termination without fairness in regular model checking, for which a verification framework exists .
In general, the difficulty with finding an encoding of fairness is how to deal with an infinite number of fairness requirements (one for each process) in a systematic and regularity-preserving manner. There are known encodings of general notions of fairness in regular model checking, e.g., by using a token that is passed to the next process (with respect to some ordering of the processes) when the current process is executed, and ensuring that the first process holds the token and passes it to the right infinitely many times (e.g. see [16, 15]). However, these encodings do not work in in our case for several reasons. Firstly, they do not take into account the unknown upper bound (from finitary fairness) within which time a process has to be executed. Adapting these encodings to finitary fairness would require the use of unbounded counters, which do not preserve regularity. Secondly, such encodings would yield the problem of verifying an almost-sure Rabin property (of the form in LTL notation, where and are regular sets). Although we could reduce this to an almost-sure termination property by means of product automata construction (i.e. by first converting the formula to deterministic Rabin automaton), the target set in the resulting termination property (consisting of configurations in strongly connected components satisfying some properties) is not necessarily regular.
Instead, we revisit the well-known abstract program transformation in the setting of non-probabilistic concurrent systems  encoding fairness into the program by associating to each process an unbounded counter that acts as an “alarm clock”, which will “set off” if an enabled process has not been chosen by the scheduler for “too long.” This abstract program transformation has been adapted by Alur & Henzinger  in the case of finitary fairness by additionally incorporating an extra counter that stores the unknown upper bound and resetting the value of a counter belonging to a chosen process to the “default value” . Our contributions are as follows:
We show how Alur & Henzinger’s program transformation could be adapted to the setting of probabilistic parameterized concurrent systems (infinite family of finite MDPs). This involves constructing a new parameterization of the system (using the idea of weakly finite systems) and a proof that the transformation preserves reachability probabilities.
We show how the resulting abstract program transformation could be made concrete in the setting of regular model checking without using automata models beyond finite automata.
We have implemented this transformation in FairyTail. Combined with the existing algorithm  for verifying almost sure termination (without fairness) in regular model checking, we have successfully verified a number of models obtained from distributed algorithms and biological systems including Herman’s protocol , Moran processes in a linear array [24, 6], and the cell cycle switch model  on ring and line topologies. To the best of our knowledge, our algorithm is the first fully-automatic method that can prove termination for these examples.
Related work. There are few techniques for automatic verification of liveness properties of parameterized probabilistic programs. Almost sure verification of probabilistic finite-state programs goes back to Pnueli and co-workers [26, 27]. Esparza et al. 
generalize the reasoning to weakly finite programs, and describe a heuristic to guess aterminating pattern by constructing a nondeterministic program from a given probabilistic program and a terminating pattern candidate. This allows them to exploit model checkers and termination provers for nondeterministic programs. More recently, Lin and Rümmer  consider unconditional termination for parameterized probabilistic programs. While our work builds on these techniques, our main contribution is the incorporation of fairness in regular model checking of probabilistic programs, which was not considered before.
Fairness for concurrent probabilistic systems was considered by Vardi  and by Hart, Sharir, and Pnueli , and generalized later [27, 29, 30]. The focus was, however, on a fixed number of processes. The notion of fairness through explicit scheduling was developed by Olderog and Apt . More recently, notions of fairness for infinitary control (i.e., where an infinite number of processes can be created) was considered by Hoenicke, Olderog, and Podelski [32, 33].
General notations: For any two given real numbers , we use a standard notation (with an extra subscript) to denote real intervals, e.g., and . We will denote intervals over integers by removing the subscript, i.e., . Given a set , we use to denote the set of all finite sequences of elements from . The set always includes the empty sequence, which we denote by . We use to denote the set . Given two sets of words , we use to denote the set of words formed by concatenating words from with words from . Given two relations , we define their composition as .
Transition systems: We fix the (countably infinite) set AP of atomic propositions. Let ACT be a finite set of action symbols. A transition system over ACT is a tuple , where is a set of configurations, is a binary relation over , and maps atomic propositions to sets of configurations (we omit if it is not important). We use to denote the relation . The notation (resp. ) is used to denote the transitive (resp. transitive-reflexive) closure of . We say that a sequence is a path (or run) in (or in ). Given two paths and in , we may concatenate them to obtain (by gluing together ). We call a prefix of . For each , we use the notations and to denote the pre/post image of under . That is, and .
Words and automata: We assume basic familiarity with finite word automata. Fix a finite alphabet . For each finite word , we write , where , to denote the segment . Given an automaton , a run of on is a function with that obeys the transition relation . We may also denote the run by the word over the alphabet . The run is said to be accepting if , in which case we say that is accepted by . The language of is the set of words in accepted by .
Reachability games: We recall some basic concepts on 2-player reachability games (see e.g. [39, Chapter 2] on games with 1-accepting conditions). An arena is a transition system , where (i.e. the set of “game configurations”) is partitioned into two disjoint sets and such that for each . The transition relation denotes the actions of Player . Similarly, for each , the configurations are controlled by Player . In the following, Player 1 will also be called “Scheduler,” and Player 2 “Process”. Given a set of initial configurations and a set of final (a.k.a. target) configurations, the goal of Player 2 is to reach from , while the goal of Player 1 is to avoid it. More formally, a strategy for Player is a partial function such that, for each and , if is a path in and is not a dead end (i.e., for some ), then is defined in such a way that . Given a strategy for Player and an initial configuration , we can define a unique (finite or infinite) path in such that where for is the (unique) configuration s.t. . Player 2 wins iff some configuration in appears in , or if the path is finite and the last configuration belongs to Player 1. Player 1 wins iff Player 2 does not win; we say Player 2 loses. A strategy for Player is winning from if for each strategy of Player , the unique path in from each witnesses a win for Player . Such games (a.k.a. reachability games) are determined (see e.g. [39, Proposition 2.21]): either Player 1 has a winning strategy or Player 2 has a winning strategy.
For notational simplicity, w.l.o.g., we make the following assumptions on our reachability games. They suffice for the purpose of proving liveness for parameterised systems.
Arenas are strictly alternating, i.e., a move made by a player does not take the game back to her configuration (, for each ).
Initial and final configurations belong to Player 1, i.e.,
Non-final configurations are not dead ends: .
Markov chains: A (discrete-time) Markov chain (a.k.a. DTMC) is a structure of the form where is a set of configurations, is a function that associates a configuration
with a probability distribution over a sample space(i.e. the probability of going to a certain configuration from ), and maps atomic propositions to subsets of . In what follows, we will assume that each is a discrete probability distribution with a finite sample space. This assumption allows us to simplify our notation: a DTMC can be seen as a transition system with a transition probability function mapping a transition to a value such that . That is, transitions with zero probabilities are removed from . We will write to denote and that . The underlying transition graph of a DTMC is the transition system with omitted. Given a finite path from the initial configuration , let be the set of all finite/infinite paths with as a prefix, i.e., of the form for some finite/infinite path . Given a set of target configurations, the probability (the subscript may be omitted when understood) of reaching from in can be defined using a standard cylinder construction (see e.g ) as follows. For each finite path in from , we set to be a basic cylinder, to which we associate the probability . This gives rise to a unique probability measure for the -algebra over the set of all runs from . The probability is then the probability of the event containing all paths in with some “accepting” finite path as a prefix, i.e., a finite path from ending in some configuration in . In general, given an LTL formula over AP, the event containing all paths from in satisfying is measurable  and its probability value is well-defined.
Notation: Whenever understood, we will omit mention of from .
3 Abstract Models of Probabilistic Concurrent Programs
In this section, we recall the notion of Markov Decision Processes (MDPs) and fair MDPs . These serve as our abstract models of probabilistic concurrent programs. We then define the notion of finitary fairness  and discuss its basic properties in the setting of MDPs.
3.1 Markov Decision Processes
A Markov decision process (MDP) is a strictly alternating arena such that is a DTMC, i.e., is associated with some transition probability function, and that the atomic propositions are not important. Intuitively, the transition relation is nondeterministic (controlled by a “demonic” scheduler), whereas the transition relation is probabilistic. By definition of arenas, the configurations of the MDPs are partitioned into the set of nondeterministic states (controlled by Scheduler) and the set of probabilistic states. Formally, . Each Scheduler’s strategy111Also called “scheduler” or “adversary” for short. gives rise to an infinite-state DTMC with the underlying transition system and the transition probability function defined as follows. Here, is the set of all finite/infinite paths from . For each state and each path from ending in some state , we define iff: (1) if is a nondeterministic state, then , and (2) if is a probabilistic state, then . Intuitively, is an unfolding of the game arena (i.e. a disjoint union of trees) where branching only occurs on probabilistic states. Transitions satisfying Case (1) have the probability ; otherwise, its probability is . We let be a function mapping each subset (used as an atomic proposition) to the set of all finite paths in from to . Since is a DTMC, given an LTL formula over subsets of as atomic propositions, the probability of satisfying in from under the scheduler is well-defined. In particular, is the probability of reaching from in under the scheduler . The probability of satisfying from in the MDP under a class of schedulers is defined to be the infimum of the set of all probabilities over all . We will omit mention of when it denotes the class of all schedulers.
An MDP is weakly-finite  if from each configuration, the set of all configurations that are reachable from it (in the underlying transition system of the MDP) is finite. Note that the state space of weakly-finite MDPs can be infinite. The restriction of weak finiteness is another way of defining the notion of parameterized systems, which are an infinite family of finite-state systems. Weakly-finite MDPs capture many interesting probabilistic concurrent systems in which each process is finite-state; this is the case for many probabilistic distributed protocols.
3.2 Fair Markov Decision Processes
A fair Markov decision process (FMDP) is a structure of the form , where is an MDP, is a weak fairness (a.k.a. justice) requirement, and is a strong fairness (a.k.a. compassion) requirement. More precisely, a weak fairness requirement is a set (at most countably infinite) of atomic weak fairness requirements of the form , for some . Here, the and modalities are the standard “always” and “eventually” LTL operators. The set (resp. ) will be called the premise (resp. consequence). Intuitively, if is interpreted as “Process 1 is waiting to move” and as “Process 1 is chosen”, then this fairness requirement may be read as: at no point can Process 1 be continuously waiting to move without being chosen. In addition, a strong fairness requirement is a set (again, at most countably infinite) of atomic strong fairness requirements of the form , for some . Using the above example, a strong fairness requirement reads: if Process 1 is waiting to move infinitely often, then it is chosen infinitely often. As before, the set (resp. ) will be called the premise (resp. consequence). In the following, when it is clear whether a fairness requirement is a justice or a compassion, we will denote it by the pair of premise and consequence.
Given an FMDP , a configuration , and a scheduler , since each atomic fairness requirement is an LTL formula and there are at most countably many atomic fairness requirements, the set of paths from in the DTMC induced by satisfying and is measurable. We say that a scheduler is -fair if for every initial configuration . The fairness conditions are realizable in if there exists at least one -fair scheduler.
A natural fairness notion we consider in this paper is process fairness, which asserts that each process is chosen infinitely often. For this notion of fairness, we can assume that the consequence of each atomic fairness requirement asserts that a particular process is chosen. We make one simplifying assumption: each process is always enabled (i.e., can always be chosen by the scheduler). This assumption is reasonable since we can always introduce an idle transition for each process. Under this assumption, we have that from each , there exists a transition for some . This implies that our fairness conditions are always realizable and that the probability of event over the set of all -fair schedulers is well-defined.
3.3 Finitary Fairness
Given an FMDP , a configuration , and a number , we say that a scheduler is --fair (or -fair whenever is understood) if for each atomic fairness requirement :
if is justice, then (the underlying graph of) contains no path of length satisfying the LTL formula .
if is compassion, then contains no path satisfying the LTL formula , where and for each .
In other words, a premise in a justice requirement cannot be satisfied for consecutive steps without satisfying a consequence, while a premise in a compassion requirement cannot be satisfied for (not necessarily consecutive) steps without satisfying a consequence. A scheduler is said to be finitary fair (fin-fair) if it is -fair for some . The fairness conditions are said to be finitary-realizable (fin-realizable) in if there exists at least one fin-fair scheduler. Under this assumption, the probability of an event over the set of all fin-fair schedulers is well-defined. In what follows, for an FMDP , we will simply denote as . In this paper, we propose to study termination of probabilistic concurrent programs under finitary fairness, i.e., to determine whether , where is the class of all fin-fair schedulers.
The following proposition states one special property of weakly-finite MDPs.
Let and be two weakly-finite fair MDPs with identical underlying transition systems (but possibly different probability values). For each set of final states, and each initial configuration , it is the case that iff .
This proposition can be proved using basic machineries from probabilistic model checking . Consider the finite MDPs and that are obtained from and by removing configurations in and their fairness requirements . It suffices to prove the following: for all schedulers iff for all schedulers . This follows from standard results from probabilistic model checking [8, Theorem 10.122] since is a limit linear-time property.
By Proposition 1, when dealing with almost-sure finitary-fair termination of weakly-finite MDPs, we only care whether a transition has a zero or a non-zero probability, i.e., if it is non-zero, then the exact value is irrelevant. Incidentally, the same also holds for other properties including almost-sure termination without fairness and qualitative temporal specifications [26, 27, 20]. For this reason, we may simply omit these probability values from our symbolic representation of weakly-finite MDPs, which we will do from the next section onwards.
3.4 Herman’s Protocol
Herman’s protocol  is a distributed self-stabilization algorithm for a population of processes organized in a ring. The correct configurations are those where exactly one process holds a token. If, through some error, the ring enters an erroneous configuration (in which multiple processes hold tokens), Herman’s protocol ensures that the system will self-stabilize: it will almost surely go back to a configuration with only one token.
Let us discuss how the protocol works in more detail. Fix processors organized in a ring. If a chosen process does not hold a token, then it can perform an idle transition (i.e. do nothing). If a chosen process holds a token, then it can keep holding the token with probability or pass it on to its clockwise neighbor (the process , for processes numbered ) with probability . If a process currently holds a token and receives another token from its (counter-clockwise) neighbor, then the two tokens are merged222Herman  describes a more general protocol in which tokens can be merged/destroyed with some probability. We consider this restriction for simplicity of presentation. into one, leaving the process with one token.
Formally, Hermann’s protocol can be modeled as a weakly-finite Markov decision process whose states are vectors in. For each , the state of the protocol is described by a vector of bits, with the -th bit being 1 iff the -th process holds a token. From a state , the scheduler picks a process . Given a chosen process , the new state remains if the chosen process did not hold a token (). If , the new state is with probability and with probability . Here, denotes a vector with in the -th position and everywhere else, and is the XOR operation. We want to ensure that, starting from an arbitrary initial assignment of tokens, any population self-stabilizes with probability 1.
Process fairness for Herman’s protocol is a set of atomic fairness requirements, each asserting that the process is executed infinitely often, for each . Unfortunately, Herman’s protocol does not terminate with probability 1 against some fair schedulers. To see this, consider the start state . Let us call the token held by Process 0 “the first token”, and the token held by Process 2 “the second token”. Define a round as the following sequence of moves by the scheduler: keep choosing the process that holds the first token until it passes the token to the right, and do the same to the second token. For example, the two configurations obtained after completing the first and second rounds from are, respectively, and . To see that the scheduler is fair, for each integer , the probability that the -th round is not completed is 0 since the probability that one of the tokens will be kept at the same process for an infinite amount of time is 0. Therefore, the probability that some round is not completed is also 0. Completing two rounds ensure that all the processes are picked. Therefore, every process will be chosen with probability 1. On the other hand, observe that correct configurations are not seen in the induced DTMC, showing that self-stabilization holds with probability 0 under this scheduler.
Herman’s protocol can be shown to self-stabilize with probability 1 under all fin-fair schedulers, which can be proved by our fully-automatic verification algorithm (presented later in the paper).
4 Regular Model Checking: A Symbolic Framework
In this section, we recall regular model checking (see e.g. [14, 15, 41]), a symbolic framework for specifying infinite-state systems based on finite automata and regular transducers and developing automatic verification (semi-)algorithms.
A transition system is specified in the framework as a regular language (e.g. as a regular expression over some alphabet ), and two “regular relations” . For simplicity, in the following we will assume that . How do we specify regular relations? One standard way is to restrict to length-preserving relations (i.e. the relation may only contain a pair of words of the same length) and specify such relations as regular languages over the alphabet . There is, then, a simple one-to-one correspondence between the set of words over and the set of all pairs of words over of the same length. This can be achieved by mapping a pair of words with to a word , defined as whenever and .
Proving that a property holds over a transition system is done “in a regular way,”, by finding a “regular proof” for the property. For example, if asserts that the set of bad states can never be reached, then a regular proof amounts to finding an inductive invariant in the form of a regular language [14, 15] that does not intersect with , i.e., , ( is a regular set of initial states), and , where . Since regular languages are effectively closed under boolean operations and taking pre/post images w.r.t. regular transducers, an algorithm for verifying the correctness of a given regular proof can be obtained by using language inclusion algorithms for regular automata, e.g., [42, 43]. The framework of regular proofs is incomplete in general since it could happen that there is a proof, but no regular proof. The pathological cases when only non-regular proofs exist do not, however, seem to frequently occur in practice, e.g., see [44, 45, 18, 14, 19, 46, 47, 15, 48].
The framework of regular proofs has been extended to deal with almost-sure termination for weakly-finite probabilistic concurrent programs in . We briefly summarise the main idea, since we reduce the fair termination problem to their setting. By Proposition 1, the actual probability values do not matter in proving almost-sure termination. For this reason, we may specify a weakly-finite MDP as a regular specification in the same way as we specify a non-probabilistic transition system in our regular specification language. Given an MDP , a set of initial configurations, and a set of final configurations, a regular proof for for each is a pair consisting of a regular inductive invariant and a regular relation such that:
is a strict preorder on , i.e., it is irreflexive () and transitive ().
irrespective of the nondeterministic transitions from any configuration in , there is a probabilistic transition to a configuration in that decreases its rank with respect to :
An automata-theoretic algorithm can then be devised for checking the above verification conditions with respect to a given regular proof .
[Herman’s protocol, continued] We provide a regular encoding of Herman’s protocol. The configurations are words over the alphabet , where (resp. ) signifies that a process holds (resp. does not hold) a token, while overlining the character signifies that the process is chosen by the scheduler. We set . The set of initial configurations is , i.e., at least one process holds a token. The set of final configurations is , i.e., there is only a single token in the system. The actions of the scheduler is to choose a process; this can be expressed as the regular expression , where denotes the regular language . The probabilistic actions can be expressed as a union of the following three regular expressions:
|(pass token right)|
5 Handling Fairness Requirements
We now describe the main result of the paper: a general method for embedding finitary fairness into regular model checking for probabilistic concurrent systems.
5.1 Regular Specifications of Fairness
When a complex system or a distributed protocol is being modelled in regular model checking, it is often necessary to add an infinite number of fairness requirements. This is because such a system admits a finite but arbitrary number of agents or processes, each with its own fairness requirement (e.g. that the process should be executed infinitely often). For this reason, it is not enough to simply express the fairness requirements as a finite set of pairs of regular languages (one for the premise, and one for the consequence). We describe a regular way of specifying infinitely many fairness constraints. Our presentation is a generalisation of the regular specification of fairness from [16, 15].
The general idea is to define a “regular function” that maps a configuration to a word such that contains: (1) a bit indicating whether is in the premise of the -th fairness requirement, (2) a bit indicating whether is in the consequence of the -th fairness requirement, and (3) a bit indicating whether the -th fairness requirement is justice or compassion. Such a regular specification of fairness allows an infinite number of fairness constraints since is potentially infinite (i.e., containing words of unbounded lengths), though only the first fairness requirements matter for a word . This is sufficient for weakly-finite MDPs since the set of reachable configurations from any given configuration is finite and so, among the infinite number of fairness constraints, only finitely many are distinguishable. The regular function can be defined by a letter-to-letter transducer with input alphabet and output alphabet . Without loss of generality, we assume that the -th letter in the output of every input word of agree on the third bit (i.e., whether the fairness requirement is justice or compassion is well-defined): for every and , if and , then . Observe this condition on can be algorithmically checked by using a simple automata-theoretic method: find two accepted words in which in some position their third bits differ.
In this case, gives rise to compassion requirements and justice requirements by associating the -th position in all output words by a unique fairness constraint. More precisely, let
Therefore, by Proposition 1, our regular fairness specification allows us to define weakly-finite fair MDPs . In the following, we shall call such fair MDPs regular.
Our main theorem is a regularity-preserving reduction from proving almost sure termination for regular FMDPs (under finitary fairness) to proving almost sure termination for regular MDPs (without fairness).
Let be a regular representation of an FMDP, be a regular set of initial configurations, and be a regular set of final configurations. Then one can compute a regular representation of MDP and two regular sets such that it holds that if and are realizable, then iff .
5.2 Abstract Program Transformation
Before proving Theorem 5.1, let us first recall an abstract program transformation à la Alur & Henzinger , which encodes finitary fairness into a program using integer counter variables. Intuitively, we reserve one variable for each atomic fairness condition as an “alarm clock” that will set off if its corresponding process has not been executed for a long time, and one global variable that acts as a default value to reset a clock to as soon as the corresponding process is executed. Although Alur & Henzinger  did not discuss about probabilistic programs, their transformation can be easily adapted to the setting of MDPs, though correctness still has to be proven.
We now elaborate on the details of the transformation. Given an FMDP with a probability distribution , the transformation will produce an MDP with a probability distribution as follows. Introduce a set of “counter” variables that range over natural numbers: (for each ), (for each ), and . Let be the set of all valuations mapping each variable in to a natural number such that for each and . We define and . We now define the transition relation such that if and
for each , ,
(for ) and (for ) change as follows:
( denotes the set-complement of ). Finally, we define the probability distribution underlying as whenever .
Intuitively, the variables ’s and ’s keep track of how long the scheduler has delayed choosing an enabled process, while the variable (unchanged once the initial configuration of the MDP is fixed) aims to ensure that the scheduler is -fair. Since is a variable (not a constant), the resulting MDP captures precisely the behaviour of under fin-fair schedulers.
If is a weakly-finite FMDP, then is weakly-finite.
Since is weakly-finite, once a configuration of is chosen, there are only finitely many different valuation such that (for some ) is reachable from . In the following, we show that there are also only finitely many different valuations such that (for some ) is reachable from . Let be the (finite) set . Define two equivalence relations and on the set of pairs of subsets of as follows:
iff (a) and , and (b) iff .
iff (a) and , and (b) iff .
Observe that, since is finite, both equivalence relations are of finite index (i.e. have only finitely many equivalence classes). This implies that we need not distinguish two variables in if they are both for the justice or both for the compassion requirements, in the same equivalence class in the appropriate relation or , and they both have the same initial -values. To see this, let . Observe that, for each and , it is the case that iff . Similarly, for each and , it is the case that iff . In other words, identical counter values across similar fairness constraints remain identical under an application of . Since all counter values in all reachable configurations are in , it immediately follows that is finite.
We next state a correctness lemma for the transformation. To this end, given a set of initial configurations in , we define:
, where contains functions such that for each and .
, where contains all such that for some or for some (i.e. one of the alarms has been triggered), and .
Lemma 2 (Correctness).
If is a weakly-finite FMDP, it is the case that
In this proof, we make use of the following notation. For a sequence of pairs , we use the notation (resp. ) to denote the sequence projected to the first (resp. second) arguments, i.e., (resp. ). Moreover, for each , we define to be the set of all functions such that .
We first prove that , i.e., the transformation does not increase the probability of reaching final states. For each , consider a -fair scheduler for . It suffices to prove that given any , , where and (note that contains exactly one compatible with ). In turn, to prove this, it suffices to prove that the DTMC restricted to configurations reachable from is isomorphic to restricted to configurations reachable from . This can be seen from the fact that configurations of the form are not reachable from ; if they were reachable, since the counter encoding precisely emulates the definition of finitary fairness , the witnessing path would give rise to a path that would witness that is not -fair, contradicting our original assumption.
We next prove that , i.e., our transformation does not decrease the probability of reaching final states. Consider any and any scheduler on . Consider the scheduler on that simulates the behaviour of , but as soon as one of the alarm clocks has set off the scheduler goes through all consequence sets (say, for some ; the sequence is finite since and are weakly finite) in some order and chooses actions that satisfy them in a round robin manner (which can be done since we consider process constraints). More precisely, for each path in , define . For each path ending in a configuration in , the action of the scheduler on any path with as a prefix is to loop through and pick actions that satisfy them. Therefore, the scheduler is -fair for . Furthermore, consider the two tree-shaped DTMCs and , where is obtained from by restricting the sets of configurations to those that are reachable from , and is obtained from by restricting the sets of configurations to those that are reachable from . and are isomorphic except for subtrees where is a path in from ending in without visiting a configuration in . The probability of visiting a configuration in in from on the condition that is taken is 1. Thus, on the condition that the prefix is taken, the probability of visiting a configuration in in cannot exceed . This shows us that . Consequently, since the choice of and was arbitrary, we can conclude that .
These two lemmas immediately imply Theorem 5.1.
5.3 Finitary Fairness in Regular Model Checking
We now show how to implement the aforementioned abstract program transformation in our regular model checking framework. Fix a regular representation of an FMDP , which includes two automata over the alphabet representing and , and an automaton over the alphabet representing the regular specification of the fairness conditions and . [Recall that .] We describe the construction of (the construction for is similar). Let be an automaton representing and be an automaton representing the regular specification of fairness. The construction of the automaton for has two stages.
Stage 1: compute an intermediate automaton.
The intermediate automaton will have the alphabet and recognize a subset of . Intuitively, on input , the automaton simultaneously takes a transition over in and any transition in , proceeding into an intermediate state where it remembers the value of , which it outputs in the next step. This process is repeated until both and accept. More precisely, the automaton is defined as where:
, , and