Ranking and Repulsing Supermartingales for Approximating Reachability

by   Toru Takisaka, et al.

Computing reachability probabilities is a fundamental problem in the analysis of probabilistic programs. This paper aims at a comprehensive and comparative account on various martingale-based methods for over- and under-approximating reachability probabilities. Based on the existing works that stretch across different communities (formal verification, control theory, etc.), we offer a unifying account. In particular, we emphasize the role of order-theoretic fixed points---a classic topic in computer science---in the analysis of probabilistic programs. This leads us to two new martingale-based techniques, too. We give rigorous proofs for their soundness and completeness. We also make an experimental comparison using our implementation of template-based synthesis algorithms for those martingales.



There are no comments yet.


page 1

page 2

page 3

page 4


Ranking and Repulsing Supermartingales for Reachability in Probabilistic Programs

Computing reachability probabilities is a fundamental problem in the ana...

Inductive Reachability Witnesses

In this work, we consider the fundamental problem of reachability analys...

A Supervisory Control Algorithm Based on Property-Directed Reachability

We present an algorithm for synthesising a controller (supervisor) for a...

Faster Reachability in Static Graphs

One of the most fundamental problems in computer science is the reachabi...

FlowCFL: A Framework for Type-based Reachability Analysis in the Presence of Mutable Data

Reachability analysis is a fundamental program analysis with a wide vari...

Upper approximating probabilities of convergence in probabilistic coherence spaces

We develop a theory of probabilistic coherence spaces equipped with an a...

Some General Structure for Extremal Sparsification Problems

This paper is about a branch of theoretical computer science that studie...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

1x := 2; y := 2; t := 0; 2while t <= 100 do 3  t := t + 1; 4  z := Unif (-2,1); 5  if * then 6    x := x + z 7  else 8   y := y + z 9  fi
Figure 1: An example of probabilistic programs. The line 4 means that the value of is randomly sampled from the interval .

Computing reachability probabilities is a fundamental problem in the analysis of probabilistic systems. It is known that probabilistic model checking titleems can be solved via reachability probabilities [4], much like nondeterministic model checking problems are reduced to emptiness and hence to reachability [34]

. While the computation of reachability probabilities for finite-state systems is effectively solved by linear programming, the problem becomes much more challenging for

probabilistic programs

—a paradigm that attracts growing attention as a programming language foundation for machine learning 

[17]—because their transition graphs are infinite in general.

Reachability probabilities of probabilistic programs with while loops are clearly not computable, because the problem encompasses termination of (non-probabilistic) while programs. Therefore the existing research efforts have focused on sound approximation methods for reachability probabilities. An approach that is widely used in the literature is to use ranking supermartingales—a probabilistic analogue of ranking functions—as a witness for the qualitative question of almost-sure reachability. Ranking supermartingales are amenable to template-based synthesis [8, 11, 9], making them appealing from the automatic analysis point of view. Recently, methods for quantitatively underapproximating reachability probabilities are also proposed in [13, 33].

The dual question of overapproximating reachability probabilities, which can then be used to qualitatively refute almost-sure reachability, is also considered. In the control theory, supermartingales are used as a probabilistic counterpart of barrier certificates [26, 31]. A similar idea is recently used for the purpose of synthesizing stochastic invariants for probabilistic programs [13]. Here an overapproximation of reachability probability serves as quantitative verification for safety: it gives an upper bound for the probability that the system or the program reaches a bad state.

certificate for from
ranking (super- and sub-) martingale for under-approximation additive supermartingale (ARnkSupM, §5) [24, 8]
-scaled submartingale (-SclSubM, §6) this paper for PP, following categorical observations in [33] for MC
repulsing supermartingale for over-approximation -decreasing supermartingale (-RepSupM, §3) [13], derived from Azuma’s martingale concentration inequality
nonnegative supermartingale (NNRepSupM, §4) this paper, derived from the Knaster–Tarski theorem ([26, 31], without nondeterminism, derived from Markov’s concentration inequality)
Table 1:

Martingale-based techniques for approximation of reachability probabilities. MC stands for Markov chains, and PP stands for probabilistic programs

Table 1 lists four supermartingale-based techniques for over- and underapproximating reachability probabilities. The table is not meant to be exhaustive—still, it shows that multiple methods have been introduced and studied, in different communities (formal verification, control theory, etc.) and with different mathematical backgrounds (ranking functions, martingale concentration inequalities, etc.).

The current work aims at a comprehensive and comparative account of those martingale-based techniques in Table 1. Central to our account is the role of order-theoretic fixed points, a classic topic in theoretical computer science. More specifically, we characterize our objectives—namely reachability probability and expected reaching time—as suitable least fixed points. It turns out that a large part of the theory of martingale-based methods can be developed based on this order-theoretic characterization, without using mathematical gadgets unique to probabilistic settings such as martingale concentration inequalities. Our contributions are summarized as follows.

  • A comprehensive and comparative account of different martingale-based techniques for approximating reachability probabilities. We identify their key mathematical principles to be order-theoretic fixed points and martingale concentration inequalities, and we emphasize the role of the former.

  • We introduce two martingale-based techniques that seem to be new, namely -SclSubM and NNRepSupM in Table 1. Their purely probabilistic versions have been in the literature: -SclSubM is from a category-theoretic account in [33], and NNRepSupM is from control theory [31]. We extend them to probabilistic programs that additionally have nondeterminism. Moreover, completeness of ARnkSupM for probabilistic programs with real-valued variables seems to be new.

  • We formalize those techniques, taking probabilistic programs (with nondeterminism) as the target of analyses. We investigate soundness and completeness of the techniques in Table 1. While the order-theoretic fixed-point foundation gives us clear theoretical guidance, additional nondeterminism requires us to carefully establish measure-theoretic arguments.

  • We implemented template-based automated synthesis algorithms for -SclSubM, -RepSupM and NNRepSupM, following [8, 11]. Our experimental results suggest the advantage of -SclSubM in quantitative reasoning, and the comparative advantage of NNRepSupM over -RepSupM in the quality of bounds.

The paper is organized as follows. Preliminaries are in §2, where we introduce our system models (pCFGs) for operational semantics of probabilistic programs, and review the theory of order-theoretic fixed points (the Knaster–Tarski and Cousot–Cousot theorems). In §36 we discuss the four techniques in Table 1, offering a unifying account based on order-theoretic fixed points, and providing some new techniques and results. In §7 we give implementations and experiment results of template-based synthesis. After discussion of related work in §8, we conclude in §9.

2 Preliminaries

We first fix some notations. We write and for the set of all natural numbers (i.e. nonnegative integers) and reals, respectively. We use subscripts to denote subsets of and ; for example, denotes the set of all nonnegative reals. We write for the sets of all finite, nonempty finite, and infinite sequences of elements of , respectively.

We use the Borel measurable structure of the set of real numbers. This induces the measurable structures of all the other sets used in this paper: where , where is finite, and so on. The induced measurable structures are defined in a standard manner: for example, where is finite, it is given by

. The set of probability distributions on

is denoted by . The Dirac measure on is denoted by . The support of is defined by . The set of all Borel measurable function from to is denoted by . The functions and are the real-valued constant function of which coefficient is and , respectively.

2.1 Probabilistic Control Flow Graphs (pCFGs)

We take the notion of pCFG from [2]

and use it as our model of probabilistic systems. pCFGs can be thought of as a subclass of Markov decision processes (MDPs), but tailored for operational semantics of probabilistic programs (§






Figure 2: The pCFG that models the probabilistic program in Fig. 1. Rectangles, diamonds, and pentagons represent deterministic, nondeterministic and assignment locations, respectively. The variables are initially set , and .
Definition 2.1 (pCFG, [2])

A probabilistic control flow graph (pCFG) is a tuple consisting of the following components.

  • A finite set of locations, equipped with a partition into nondeterministic, probabilistic, deterministic and assignment locations.

  • A finite set of program variables.

  • An initial location , and an

    initial valuation vector


  • A transition relation which is total (each location has a successor). For , we write to denote the set of all successors of , i.e. . We require that each assignment location has a unique successor; in this case, denotes this unique location.

  • An update function , where . Here, three components of represent deterministic, probabilistic and nondeterministic assignment, respectively.

  • A family of probability distributions.

  • A guard function such that, for each , the following hold: (collective exhaustion) ; and (mutual exclusion) , and imply . We write if .

A configuration of a pCFG is a pair of a location and a vector. A successor of a configuration is a one such that and

if then ; if then and ; and if and , then . Here denotes an update of the vector (the -component of is replaced by ), and i) if , ii) if , and iii) if .

A finite path of is a finite sequence of configurations where is a successor of for each . Similarly, A run of is an infinite sequence of configurations such that each is a successor of .

Schedulers resolve nondeterminism. Given a history of configurations, it gives a distribution of the successor’s location or valuation vector. We assume that a scheduler is universally measurable, which is standard in control theory (see e.g. [5]).

If a pCFG and a scheduler for are given, then the behavior of is determined for each initial configuration ; we represent it by the map . For each nonempty sequence the distribution is, intuitively, the distribution of the next configuration given a current history of configurations under the scheduler . For the set of all schedulers for we define the following.

Definition 2.2 (reachability probabilities )

Let be a pCFG. The reachability probability from a configuration to a region under a scheduler is defined by

for the case of , and otherwise. The upper reachability probability from to is defined by ; the lower reachability probability is defined by .

Definition 2.3 (reaching times )

Let be a pCFG. The expected reaching time of from a configuration to under a scheduler is defined by for ; for it is defined by

if , or otherwise. The upper expected reaching time of from to is , and the lower expected reaching time is .

2.2 Probabilistic Programs: APP and PPP

The goal of this paper is the reachability analysis of imperative programs with probabilistic and nondeterministic branching. We consider two languages taken from [13, 11], called affine probabilistic programs (APP) and polynomial probabilistic programs (PPP). The two languages differ only in the arithmetic expressions allowed in the assignment commands and Boolean expressions. For example, the assignment command is allowed in PPP but not in APP; is allowed in both since its right-hand side is an affine expression.

Both APP and PPP have the standard control structure in imperative languages—such as if-branches and while-loops. APP and PPP additionally have nondeterministic and probabilistic if-branches (if  then  and if  then , respectively, where ). They also have nondeterministic and probabilistic assignment commands: where a value is chosen from a set ; and where a value is sampled from a probability distribution over .

The definition of the semantical model pCFG (§2.1) mirrors the structure of these languages. The translation from APP/PPP to pCFGs is straightforward and omitted.

2.3 Order-Theoretic Foundation of Fixed Points

Order-theoretic fixed points are central to computer science, for recursive computation, inductive/coinductive datatypes and reasoning and specification of reactive behaviors, etc. In general, a fixed-point equation can have multiple solutions; often we are interested in extremal solutions: least fixed points (lfp’s, for liveness, induction, etc.) and greatest ones (gfp’s, for safety, coinduction, etc.). The following fundamental results (in a simple setting of complete lattices) give two different characterizations of lfp’s and gfp’s.

Theorem 2.4

Let be a complete lattice, and be a monotone function. Then has the least fixed point and the greatest . Moreover,

  1. (Knaster–Tarski) The lfp is the least pre-fixed point: . Similarly, the gfp is the greatest post-fixed point: .

  2. (Cousot–Cousot [14]) The (potentially transfinite) ascending chain stabilizes to . Here is defined by obvious induction: for a successor ordinal; and for a limit ordinal.

    Similarly, the descending chain stabilizes to . ∎

From these characterizations we can derive the following reasoning principles.

Corollary 2.5
(lfp-KT) implies . (gfp-KT) implies . (lfp-CC) For each ordinal , . (gfp-CC) For each ordinal , . ∎

The arguments so far are symmetric for lfp’s and gfp’s. However, if one turns to the common proof methods for lfp specifications (termination, reachability, liveness) and those for gfp specifications (safety), a strong contrast emerges. Here is an example.

Lemma 2.6
Let be a Kripke frame, and . (Invariant for safety) Let be an invariant, that is, . Here is defined by . Assume also that . Then implies that there is no path from to . (Ranking function for liveness) Let be a ranking function for . That is, 1) for each , there is a successor such that ; and 2) for each , implies . Then, implies that there is a path from to . ∎
Knaster–Tarski Cousot–Cousot
lfp overapprox. underapprox.
gfp underapprox. overapprox.

The difference between the two methods is accounted for by the fact that, in Cor. 2.5, two items give under-approximations while the other two give over-approximations. It is clear that the invariant method in Lem. 2.6 comes from (gfp-KT) of Cor. 2.5. Its dual, (lfp-KT), gives only an overapproximation —it can be used for refutation but not for verification. Similarly, ranking functions come from (lfp-CC)—the role of well-foundedness of the value domain mirrors the structure of ordinals. Its dual (gfp-CC) only gives an overapproximation of . The situation is summarized in the above table.

The above foundations underpin our technical developments: this is because reachability probabilities and reaching times are characterized as least fixed points. We note that our semantical domains in later sections need not be complete lattices. In those cases we exploit the - and -cpo structures, the corresponding continuity of , and the Kleene theorem. The last is understood as a variation of the Cousot–Cousot theorem.

2.4 Invariants and the Nexttime Operations

In §36 the following definitions will be used.

Definition 2.7 ((pure) invariant for pCFG)
Let be a pCFG. A measurable set is called a (pure) invariant for if , and for each , if is a successor of then .
Definition 2.8 (the “nexttime” operation )

Let be a pCFG, be a pure invariant and . For a measurable we define the function of the same type as as follows, provided the right-hand side of each equation is well-defined.

  • For , .

  • For , .

  • For , where is the unique location s.t. .

  • For , let .

    • if is a measurable function.

    • if is a distribution.

    • if is a measurable set.

The function is defined as above, but replacing with in the first line and with in the last.

Proposition 2.9
We define a pointwise partial order on , i.e.  if and only if holds for every . Let be a proper closed convex subset of . Then and are well-defined for every , and the following hold. The operators and are monotone endofunctions over . In particular, and are Borel measurable for any . is -continuous, and is -continuous. ∎

3 -Decreasing Repulsing Supermartingales (-RepSupM)

In §36 we will discuss the four martingale-based techniques in Table 1. Here we briefly review the notion of -decreasing repulsing supermartingale (-RepSupM) from [13]. It is, to the best of our knowledge, the only existing martingale-based notion for overapproximating reachability probabilities.

Definition 3.1 (-RepSupM [13])
Let be a pCFG, be a pure invariant, and be a Borel set. An -repulsing supermartingale (-RepSupM) for supported by is a measurable function such that i) for each , and ii) for each .
Theorem 3.2 (soundness, [13])
Suppose there exists an -RepSupM for supported by such that . Further assume that has -bounded differences for some , i.e. for each and its successor it holds . Let and . We have the following inequality:
If the right-hand side of is greater than , still holds. ∎

We note that for any that has -bounded differences, the function is well-defined. The bound in (1) is derived from Azuma’s concentration inequality, a well-known martingale concentration lemma that exploits -bounded differences. -RepSupM is not complete: there exist a pCFG and a set of configurations such that but no -RepSupM can prove it. See Fig. 3 below.




Figure 3: An example of incompleteness of -RepSupM. Probabilistic locations are depicted by circles. This pCFG satisfies but no -RepSupM can refute its a.s. reachability. Indeed, any -RepSupM for must satisfy due to the -decreasing condition, but such an cannot have -bounded differences at .

4 Nonnegative Repulsing Supermartingales (NNRepSupM)

We move on to another notion for overapproximating reachability probabilities, nonnegative repulsing supermartingale (NNRepSupM). We believe this is new. Compared to the notion of -RepSupM, NNRepSupM has the following features.

  • NNRepSupM is derived from the theory of order-theoretic fixed points (§2.3), unlike -RepSupM that relies on Azuma’s martingale concentration lemma.

  • Consequently, we can show soundness and completeness of NNRepSupM rather easily, while -RepSupM is sound but not complete.

  • We experimentally observe that NNRepSupM often gives better bounds (§7).

The definition of NNRepSupM resembles probabilistic barrier certificates used in control theory [26, 31]. Our technical contributions are the following: i) we develop the theory of NNRepSupM in the presence of nondeterminism, while the settings in [26, 31] are purely probabilistic; and ii) we characterize NNRepSupM in the general terms of order-theoretic fixed points (§2.3), unlike the previous theory in [26, 31] that relies on Markov’s martingale concentration lemma.111We note that the theory of NNRepSupM can also be developed using Markov’s lemma. The latter unveils the mathematical similarity between NNRepSupM and ARnkSupM (§5).

The notion comes with upper and lower variants. They are used to overapproximate and , respectively (Def. 2.2). In this section we use .

Definition 4.1 (NNRepSupM for pCFG)

Let be a pCFG, be a pure invariant, and be a Borel set. An upper nonnegative repulsing supermartingale (U-NNRepSupM) over for supported by is a function s.t.

The function is a lower nonnegative repulsing supermartingale (L-NNRepSupM) if it satisfies the above conditions, but with replaced with .

We shall prove soundness and completeness of NNRepSupM, based on the foundations in §2.3. The following characterization is fundamental.

Proposition 4.2

In the setting of Def. 4.1, we define endofunctions and over as follows:

Then the upper reachability probability 222 Precisely it is the restriction of to ; in what follows we do this identification for , , , and . is the least fixed point (lfp) of . Similarly, is the lfp of .


(Sketch) We first need to show that and are Borel measurable. This is not very easy, as they are defined via supremum or infimum over uncountably many schedulers. We use the technique of -optimal scheduler known from control theory [5].

Checking that and are fixed points is not hard, though laborious. We use -optimal schedulers again for interchange between sup./inf. and integration.

Finally, the proofs for minimality differ for and . For , we first observe that is -continuous (immediate from Prop. 2.9). Therefore by the Kleene theorem, the lfp of is given by (i.e. the chain in Thm. 2.4 stabilizes after steps). We can check the coincidence between and by direct calculation.

For , let be a fixed point of . Then for each , we can construct a scheduler such that for each (at the -th step, chooses a -optimal successor). Since , this proves . ∎

It is easy to see that a U-NNRepSupM is nothing but a pre-fixed point of (i.e. ), and that an L-NNRepSupM is a pre-fixed point of . Therefore, soundness and completeness of NNRepSupM follow essentially from Cor. 2.5.

Corollary 4.3
  1. (Soundness) If is a U-NNRepSupM for supported by , then for each we have .

    Similarly, if is an L-NNRepSupM for supported by , then for each we have . This means, concretely, that for each there is a scheduler such that, for any , we have .

  2. (Completeness) There exists a U-NNRepSupM that gives the optimal bound for . The same for L-NNRepSupM. ∎

5 Additive Ranking Supermartingales (ARnkSupM)

We move on to the notion of additive ranking supermartingale (ARnkSupM) in Table 1. It is the best-known martingale-based notion for analysis of probabilistic programs and is used for overapproximating the expected reaching time. That its value is finite implies almost-sure reachability, too. We review its theory; the reason is to demonstrate that the same order-theoretic structure (see §2.3) underlies ARnkSupM and NNRepSupM in the previous section. The completeness result ((2) of Cor. 5.3) for pCFGs with real-valued variables seems new, too; See §8 for a detailed comparison to existing works. Proofs are done in a much similar manner to the ones in §4. In this section we use .

We note that completeness of U-ARnkSupM we state below is the one for strong almost-sure reachability [3]. U-ARnkSupM is incomplete for positive almost-sure reachability [15], that is, it cannot witness the condition in general.

Definition 5.1 (ARnkSupM for pCFG, [8])

Let be a pCFG, be a pure invariant, and be a Borel set. An upper additive ranking supermartingale (U-ARnkSupM) over for supported by is a function that satisfies for each .

The function is a lower additive ranking supermartingale (L-ARnkSupM) if it satisfies the above conditions, but with replaced with .

Proposition 5.2

In the setting of Def. 5.1, we define endofunctions and over as follows:

Then the upper expected reaching time is the lfp of . Similarly, is the lfp of . ∎

Corollary 5.3
  1. (Soundness, e.g. [2]) If is a U-ARnkSupM for supported by , then for each we have . In particular, for each that satisfies we have .

    Similarly, if is an L-ARnkSupM for supported by , then for each we have . This means, concretely, that for each there is a scheduler such that, for any , we have . In particular, for each that satisfies we have .

  2. (Completeness) There exists a U-ARnkSupM that gives the optimal bound for . The same holds for L-ARnkSupM. ∎

6 -Scaled Submartingales (-SclSubM)

Here we present the theory of -scaled submartingales (-SclSubM). It is for underapproximating reachability (Table 1). Compared to the well-known method of ARnkSupM, the greatest advantage is in quantitative reasoning: the value of a -SclSubM is guaranteed to be below the reachability probability (which can be less than ), while ARnkSupM is useful only if almost reachability holds. In this section we use .

The notion of -SclSubM is first introduced in [33], as an instance of a categorical abstraction of ranking functions. The current paper’s contribution lies in the following: i) the theoretical developments about -SclSubM in concrete (non-categorical) terms; ii) introduction of nondeterminism (the setting of [33] is purely probabilistic); and iii) template-based synthesis of -SclSubM.

Definition 6.1 (-SclSubM for pCFG, [33])
Let be given. An upper -Scaled Submartingale (U--SclSubM) over for supported by is a function that satisfies for each . A lower -Scaled Submartingale (L--SclSubM) over for supported by is a function that satisfies for each .

The derivation of -SclSubM, from a categorical account in [33], can be described in the following concrete terms. A -SclSubM is a post-fixed point of certain functions (namely and below). According to (gfp-KT) in Cor. 2.5, -SclSubM underapproximates a greatest fixed point—but reachability is a least fixed point. The trick here is as follows: 1) thanks to the scaling by , the gfp and lfp of coincide; and 2) the lfp (hence the gfp) of is easily seen to be below the lfp of , that is, the reachability probability that we are after. The overall argument signifies the role of the Knaster–Tarski theorem.

Proposition 6.2

Let and be as defined in Prop. 4.2. Define endofunctions and over as follows: and Then we have i) and , and ii) and . ∎

Corollary 6.3 (soundness)

If is a U--SclSubM for supported by , then for each we have . This means, concretely, that for each there is a scheduler such that, for any , we have .

Similarly, if is an L--SclSubM for supported by , then for each we have .


Just notice that if is an upper- or lower--SclSubM, then so is . The rest is as described in the paragraph before Prop. 6.2. ∎

7 Implementation and Experiments

We implemented template-based automated synthesis algorithms for NNRepSupM (§4) and -SclSubM (§6), and present some experimental results. We implemented the following programs:

  1. synthesis of a U-NNRepSupM for an APP based on a linear template.

  2. synthesis of a U-NNRepSupM for a PPP based on a polynomial template.

  3. synthesis of an L--SclSubM for an APP based on a linear template.

Each algorithm first translates given an APP or a PPP to a pCFG and a terminal configuration , and then solves an optimization problem of finding a U-NNRepSupM (L--SclSubM) over for that gives a small (large) value as possible at the initial configuration. Reduction of optimization problems to LP or SDP ones are done in standard ways in the literature; we use Farkas’ lemma (see e.g. [8, 13]) for the case of APPs, and Schmüdgen’s Positivstellensatz (see e.g. [11, 9]) for PPPs.

We have augmented the syntax of APPs and PPPs (§2.2) so that we can specify an invariant and a terminal configuration . The program does not synthesize an invariant nor prove the correctness of the given invariant, and therefore the user has to provide a correct invariant by hand or by using some algorithm, e.g. [22].

All the programs are implemented in OCaml. We have used glpk (v4.63) [16] and SDPT3 [29] for the LP and SDP solvers respectively. For the implementation of Prog. II, we have also made use of a MATLAB toolbox SOSTOOLS (v3.03) [30].

We tested our implementations for several APPs and PPPs. We have used different benchmark sets for Prog. III and Prog. III because what is overapproximated by Prog. III () and what is underapproximated by Prog. III () are different. The benchmarks implement the following probabilistic processes that are used as benchmarks in the literature. More details and codes are given in §0.G.

  1. (Adversarial random walk) A variation of a random walk, whose analysis is more challenging because of additional adversarial nondeterministic choices [12]. We have considered three variants: (a-1) 1D, (a-2) 2D and (a-3) a variant of 2D. (a-1) is a random walk over modeling a discrete queuing system, and is parametrized by that determines the distribution of the number of packets that arrive in each round. (a-2) and (a-3) are random walks over parametrized by . They determine the distribution of movement distances in each round. We added a queue size limit for (a-1) and a time limit for (a-2) and (a-3). If the queue size exceeds in (a-1) or rounds were consumed in (a-2) or (a-3), the program stops, and it is not counted as termination.

  2. (Room temperature control) A model of an air conditioning system for adjacent two rooms [1, 9]. It is parametrized by real numbers and : the former determines the power of the air conditioner, and the latter determines the size of perturbation. We have also added a time limit of as in (a) above.

We have coded (a)(b) as an APP. Experiments for Prog. I and III were carried out on a MacBook Pro laptop with a Core i5 processor (2.6 GHz, 2 cores) and 16 GiB RAM. That for Prog. II was carried out on an Amazon EC2 c4.large instance (May 2018, 2 vCPUs and 3.75 GiB RAM) running Ubuntu 16.04.4 LTS (64 bit). The results are in Table. 33. For each program, the first column (“time (s)”) shows the total execution time, and the second column (“bound”) shows the calculated probability bound.

(Applicability of NNRepSupM) Table 3 shows the results for Prog. III; the goal of these experiments is to certify the applicability of NNRepSupM to programs with nondeterminism (a-1). We have tested them for (a-1) with two combinations of parameters. Prog. III found a nontrivial bound for the reachability probability when while it failed to find such a bound when . Intuitively, the random walk is more “unfavorable” in the former case in the sense that the opposite direction from a terminal configuration is chosen in higher probabilities. As expected, a polynomial NNRepSupM gives tighter bound than a linear one, but it took much longer. The bound was not improved by increasing the degree of the polynomial template.

(Applicability of -SclSubM) Table 3 shows the results for Prog. III; here we wish to certify applicability of our new method -SclSubM. For each variant of (a), we have tested Prog. III for two combinations of parameters. In each variant, Prog. III gives a nontrivial probability bound for one combination and a trivial bound for the other combination. In fact, all the cases where nontrivial bounds were “favorable” random walks where the direction to a terminal configuration tends to be chosen. In contrast, the cases where no nontrivial bound was found were “unfavorable” random walks. Note that this is the converse of the results for Prog. IIII. Prog. III also succeeded in giving a nontrivial bound for (b). However, if we increase the parameter (i.e. if we strengthened the power of air conditioners), it failed to give a nontrivial bound.

Prog. I (linear) Prog. II (deg.-2 poly.) Prog. II (deg.-3 poly.) param. time (s) bound time (s) bound time (s) bound (a-1) 0.021 530.298 572.393