 # The Satisfiability Problem for Unbounded Fragments of Probabilistic CTL

We investigate the satisfiability and finite satisfiability problem for probabilistic computation-tree logic (PCTL) where operators are not restricted by any step bounds. We establish decidability for several fragments containing quantitative operators and pinpoint the difficulties arising in more complex fragments where the decidability remains open.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Temporal logics are a convenient and useful formalism to describe behaviour of dynamical systems. Probabilistic CTL (PCTL) [17, 16] is the probabilistic extension of the branching-time logic CTL 

, obtained by replacing the existential and universal path quantifiers with the probabilistic operators, which allow us to quantify the probability of runs satisfying a given path formula. At first, the probabilities used were only 0 and 1

, giving rise to the qualitative PCTL (qPCTL). This has been extended to any values from [0, 1] in , yielding the (quantitative) PCTL (onwards denoted just PCTL). More precisely, the syntax of these logics is built upon atomic propositions, Boolean connectives, temporal operators such as X (“next”) and U (“until”), and the probabilistic quantifier where is a numerical comparison such as or , and is a rational constant. A simple example of a PCTL formula is , which says that on almost all runs we reach a state where there is 90% chance to finish in the next step and up to this state ok

holds true. PCTL formulae are interpreted over Markov chains

 where each state is assigned a subset of atomic propositions that are valid in a given state.

In this paper, we study the satisfiability problem, asking whether a given formula has a model, i.e. whether there is a Markov chain satisfying it. If a model does exist, we also want to construct it. Apart from being a fundamental problem, it is a possible tool for checking consistency of specifications or for reactive synthesis. The problem has been shown EXPTIME-complete for qPCTL in the setting where we quantify over finite models (finite satisfiability) [17, 7] as well as over generally countable models (infinite satisfiability) . The problem for (the general quantitative) PCTL remains open for decades. We address this question on fragments of PCTL. In order to get a better understanding of this ultimate problem, we answer the problem for several fragments of PCTL that are

• quantitative, i.e. involving also probabilistic quantification over arbitrary rational numbers (not just 0 and 1),

• step unbounded, i.e. not imposing any horizon for the temporal operators.

Besides, we consider models with unbounded size, i.e. countable models or finite models, but with no a priori restriction on the size of the state space. These are the three distinguishing features, compared to other works. The closest are the following. Firstly, solutions for the qPCTL have been given in [17, 7] and for a more general logic PCTL in [24, 21]. Secondly,  shows decidability for bounded PCTL where the scope of the operators is restricted by a step bound to a given time horizon. Thirdly, the bounded satisfiability problem is to determine, whether there exists a model of a given size for a given formula. This problem has been solved by encoding it into an SMT problem . There is an important implication of this result. Namely, if we are able to determine a maximum required model size for some formula, then it follows that the satisfiability of that formula can also be determined. We take this approach in some of our proofs. Additionally, we use the result of  that the branching degree (number of successors) for a model of a formula can be bounded by , where is the length of .

Our contribution is as follows:

• We show decidability of the (finite and infinite) satisfiability problem for several quantitative unbounded fragments of PCTL, focusing on future- and globally-operators (,).

• We investigate the relationship between finite and infinite satisfiability on these fragments.

• We identify a fundamental issue preventing us from extending our techniques to the general case. We demonstrate this on a formula enforcing a more complicated form of its models. This allows us to identify the “smallest elegant” fragment where the problem remains open and the solution requires additional techniques.

Note that the considered fragments are not that interesting themselves. However, they illustrate the techniques that we developed and how far we can push decidability results when applying only those. Another fragment which might seem simple enough to be reasonable to consider is the pure U-fragment, but despite all efforts, we have not been able to show decidability for any interesting fragment thereof. For this reason, we will not consider general U-operators in this paper. Due to space constraints, the proofs are sketched and then worked out in detail in the Appendix.

### 1.1 Further related work

As for the non-probabilistic predecessors of PCTL, the satisfiability problem is known to be EXPTIME-complete for CTL  as well as the more general modal -calculus [3, 15]. Both logics have the small model property [12, 20], more precisely, every satisfiable formula has a finite-state model whose size is exponential in the size of . The complexity of the satisfiability problems has been investigated also for fragments of CTL  and the modal -calculus .

The satisfiability problem for qPCTL and qPCTL was investigated already in the early 80’s [24, 21, 17], together with the existence of sound and complete axiomatic systems. The decidability for qPCTL over countable models also follows from these general results for qPCTL, but the complexity was not examined until , showing it is also EXPTIME-complete, both for finite and infinite satisfiability.

While the decidability of satisfiability is open, there are only few negative results.  proves undecidability of the problem whether for a given PCTL formula there exists a model with a branching degree that is bounded by a given integer, where the branching degree is the number of successors of a state. However, the authors have not been able to extend their proof and show the undecidability for the general problem.

The PCTL model checking problem is the task to determine, whether a given system satisfies a given formula, i.e. whether it is a model of the formula. This problem has been studied both for finite and infinite Markov chains and decision processes, see e.g. [10, 19, 14, 13, 8]. The PCTL strategy synthesis

problem asks whether the non-determinism in a given Markov decision process can be resolved so that the resulting Markov chain satisfies the formula

[1, 22, 5, 6].

## 2 Preliminaries

In this section, we recall basic notions related to (discrete-time) Markov chains  and the probabilistic CTL . Let be a finite set of atomic propositions.

### 2.1 Markov chains

###### Definition 1 (Markov chain)

A Markov chain is a tuple where is a countable set of states, is the probability transition matrix such that, for all , , is the initial state, and is a labeling function.

Whenever we write , we implicitly mean a Markov chain . The semantics of a Markov chain , is the probability space , where is the set of runs of , is the -algebra generated by the set of cylinders of the form and the probability measure is uniquely determined  by if and otherwise.

We say that a state is reached on a run if it appears in the sequence; a set of states is reached if some of its states are reached. The immediate successors of a state are denoted by and the set of states reachable with positive probability is the reflexive and transitive closure . We will write , and , if is clear from the context.

The unfolding of a Markov chain is the Markov chain with the form of an infinite tree given by and . Each state of maps naturally to a state of (the last one in the sequence), inducing an equivalence relation iff . Consequently, each run of maps naturally to a run of and the unfolding preserves the measure of the respective events.

For a Markov chain , a set is called strongly connected if for all , ; it is a strongly connected component (SCC) if it is maximal (w.r.t. inclusion) with this property. If, moreover, for all then it is a bottom SCC (BSCC). A classical result, see e.g. , states that the set of states visited infinitely often is almost surely, i.e. with probability 1, a BSCC:

###### Lemma 1

In every finite Markov chain, the set of BSCCs is reached almost surely. Further, conditioning on runs reaching a BSCC , every state of is reached infinitely often almost surely.

### 2.2 Probabilistic Computational Tree Logic

The definition of probabilistic CTL (PCTL)  is usually based on the next- and until-operators (, ). In this paper, we restrict our attention to the future- and globally operators (, ), which can be derived from the until-operator. Further, w.l.o.g. we impose the negation normal form and the lower-bound-comparison normal form; for the respective transformations see, e.g., .

###### Definition 2 (Pctl(F,G) syntax and semantics)

The formulae are given by the following syntax:

 Φ::=a∣¬a∣Φ∧Φ∣Φ∨Φ∣F⊳qΦ∣G⊳qΦ

where , , and is an atomic proposition. Let be a Markov chain and its state. We define the modeling relation inductively as follows

1. [label=(M0),align=left]

2. iff

3. iff

4. iff and

5. iff or

6. iff

7. iff

where is with being the initial state, and is the th element of . We say that is a model of if .

We will denote the set of literals by . Instead of the constraint , we often write . Further, we define the set of all subformulae. This definition slightly deviates from the usual definition of subformulae, e.g. the one in , in that does not necessarily imply .

###### Definition 3 (Subformulae)

The set is recursively defined as follows

• if or , then

• if or then

Next, we introduce the satisfiability problems, which are the main topic of the paper.

###### Definition 4 (The satisfiability problems)

A formula is called (finitely) satisfiable, if there is a (finite) model for . Otherwise, it is (finitely) unsatisfiable. The (finite) satisfiability problem is to determine whether a given formula is (finitely) satisfiable.

Instead of simply writing “satisfiable” we sometimes stress the absence of “finitely” and write “generally satisfiable” for satisfiablity on countable, i.e. finite or countably infinite, models. For some proofs, it is more convenient to consider the unfolding of a Markov chain instead of the original one. As we mentioned already, the measure of events is preserved in the unfolding of a chain. Hence, we can state the following lemma.

###### Lemma 2

If is a model of then its unfolding is a model of .

We say that formulae are (finitely) equivalent if they have the same set of (finite) models, written (); that they are (finitely) equisatisfiable if they are both (finitely) satisfiable or both (finitely) unsatisfiable; and that if every model of is also a model of .

## 3 Results

In this section we present our results. A summary is schematically depicted in Fig. 1. We briefly describe the considered fragments; the full formal definitions can be found in the respective sections. Since already the satisfiability for propositional logic in negation normal form has nontrivial instances only when all the constructs and conjunction are present, we only consider fragments with all three included; see the bottom of the Hasse diagram. The fragments are named by the list of constructs they use, where we omit the three constructs above to avoid clutter. Here stands for and stands for for all . Further, denotes the sub-fragment of where the topmost operator is . Finally, denotes the use of with the restriction that inside only can be used.

The fragments are investigated in the respective sections. We examine the problems of the general satisfiability (“inf”) and the finite satisfiability (“fin”); equality denotes the problems are equivalent. We use two results to prove decidability of the problems. Firstly,  shows that given a formula and an integer , one can determine whether or not there is a model for that has at most states. Consequently, we obtain the decidability result whenever we establish an upper bound on the size of smallest models. Here “” denotes the satisfiability of a given on models of size . Secondly,  establishes that for any satisfiable PCTL formula there is a model with branching bounded by . Consequently, we obtain the decidability result whenever it is sufficient to consider trees of a certain height (with back edges) since the number of their nodes is then bounded by . Here “H:” denotes that the models can be limited to a height .

While we obtain decidability in the lower part of the diagram, the upper part only treats finite satisfiability, and in particular for , we only demonstrate that models with more complicated structure are necessary. Namely, the models may be of unbounded sizes for structurally same formulae—i.e. formulae which only differ in the constraints on the temporal operators—or require presence of non-bottom SCCs, see Section 3.6 and the discussion in Section 4.

### 3.1 Finite satisfiability for Gq(Fq,Gq,∨)

This section treats -formulae of the -fragment, i.e. of PCTL. In particular, it includes -formulae. In general, formulae in this fragment (even without quantified and -operators) can enforce rather complicated behaviour . Therefore, we will focus on finitely satisfiable formulae. We will see that they can be satisfied by rather simple models.

###### Definition 5

-formulae are given by the grammar

 Φ ::=G⊳qΨ Ψ ::=a∣¬a∣Ψ∧Ψ∣Ψ∨Ψ∣F⊳qΨ∣G⊳qΨ

The main result of this section is that finitely satisfiable formulae in this fragment can be satisfied by models of size linear in .

###### Theorem 3.1

Let be a finitely satisfiable -formula. Then has a model of size at most .

Intuitively, we obtain the result from the fact that some BSCC is reached almost surely and every state in a BSCC is reached almost surely, once we have entered one. In infinite models, BSCCs are not reached almost surely and therefore the proofs cannot be extended to general satisfiability. The following lemma and its proof demonstrate how we can make use of the BSCC properties in order to obtain an equisatisfiable formula in a simpler fragment.

###### Lemma 3

Let be a -formula. Then, is finitely equisatisfiable to a -formula , such that .

###### Proof

Write as . Assume that we have a finite model for . Intuitively, we can select a BSCC that satisfies . We know that there is a BSCC because we are dealing with a finite model. We also know that there is at least one BSCC satisfying our formula, for otherwise would not be a model for it. In a BSCC, every state is reached almost surely from every other state by Lemma 1. Hence, we can select exactly one state for each -subformula which satisfies that formula’s argument. Then we can create a new BSCC from these states, arranging them, e.g., in a circle. This BSCC models , where replaces all probabilistic operators with their “almost surely” version. Hence, we have created a model for a formula from a model for . The opposite direction follows from the fact that .

Note that the transformation does not produce an equivalent formula. Hence, we cannot replace an occurrence of such a formula in a more complex formula. For instance, the formula is satisfiable, whereas is not. The proof does not work for equality because we are selecting one BSCC while ignoring the rest. This example demonstrates why we cannot ignore certain BSCCs in general. Using the above result, it is easy to prove Theorem 3.1.

###### Proof (Proof Sketch of Theorem 3.1)

This follows immediately from the proof of Lemma 3. The BSCC that we have created has at most as many states as there are -subformulae, which is bounded by .

###### Example 1

Consider the formula

 ϕ:=G≥1/2(F≥1/3a∧F≥1/3¬a). (1)

The large Markov chain in Figure 2 models

. Unlabeled arcs indicate a uniform distribution over all successors. It is clear that the model is unnecessarily complicated. After reducing it according to Lemma

3, we obtain the smaller Markov chain on the right.

The example below shows that satisfiability is not equivalent to finite satisfiability for this fragment, and that the proposed transformation does not preserve equisatisfiability over general models. The decidability of the general satisfiability thus remains open here.

###### Example 2

Note that we made use of the BSCC properties for the proofs of this subsection, such as that some BSCC is reached almost surely. Since this is only the case for finite Markov chains, our transformation only holds for finite satisfiability. If we consider the general satisfiability problem, then the equivalent of Lemma 3 is not true. For instance, the formula

 ϕ:=G>0(¬a∧F>0a) (2)

is satisfiable, but requires infinite models, as pointed out in . One such model is given in Figure 3. Observe that the single horizontal run has measure greater than . Now consider

 ^ϕ:=G=1(¬a∧F=1a)

Obviously, this formula unsatisfiable. Hence, in this case is not equisatisfiable to .

### 3.2 Satisfiability for G1(Fq,G1)

This section treats -formulae of a fragment where only appears with and there is no disjunction. The results are later utilized in a richer fragment in Section 3.4. In fact, the main result of this section is an immediate consequence of the main theorem of Section 3.3. Still, the results are interesting themselves as they show some properties of models for formulae in this fragment which do not apply in the generalized case.

###### Definition 6

-formulae are given by the grammar

 Φ ::=G=1Ψ Ψ ::=a∣¬a∣Ψ∧Ψ∣F⊳qΨ∣G=1Ψ

We prove that satisfiable formulae of this fragment are satisfiable by models of linear size and thus also finitely satisfiable.

###### Theorem 3.2

Let be a satisfiable -formula. Then has a model of size at most .

The idea here is that we can find a state which behaves similarly to a BSCC (even in infinite models) in that it satisfies all -subformulae. We can then use this state’s successors to construct a small model. The outline of the proof is roughly as follows: First we show that from every state and for every subformula we can find a successor that satisfies this subformula. Using this, we can show that there is a state that satisfies all -subformulae.

###### Lemma 4

Let be a satisfiable -formula and its model. Then, for every , and , there is a state , such that .

###### Proof

This follows from the fact that we do not allow disjunctions in this fragment. We apply induction over the depth of a subformula . If the formula is itself, then there is nothing to show. Otherwise, the induction hypothesis yields that the higher-level subformulae are satisfied at some state . From this, we can easily see that in all possible cases the claim follows: If the higher-level formula is a conjunction, then is one of its conjuncts. Since both conjuncts must be satisfied by , in particular must be satisfied at . A similar argument applies to -formulae. If it is of the form , then we know that there must be a reachable state where holds.

This concludes the first part of the proof. We continue with the second part and prove that we can find a state which satisfies all -subformulae.

###### Lemma 5

Let be a satisfiable -formula, its model, and let . Then there is a state such that for all .

###### Proof

It is clear that after encountering a -formula at some state, all successors will also satisfy it. Therefore, the set of satisfied -formulae is monotonically growing and bounded. Hence, we can apply induction over the number of yet unsatisfied -formulae. In every step, we are looking for the next state to satisfy an additional -formula. This is always possible (as long as there are still unsatisfied ones), due to Lemma 4.

Now, we can prove Theorem 3.2.

###### Proof (Proof Sketch of Theorem 3.2)

By Lemma 5, we can find a state that satisfies all -subformulae. In some sense, this state’s subtree resembles a BSCC. We can include exactly one state for each -subformula and create a BSCC out of those states, e.g., arrange them in a circle. We apply induction over . The satisfaction of literals and conjunctions is straightforward. Since every state is reached almost surely, every -formula will be satisfied that way. The satisfaction of the -formulae follows from the fact that all states used to satisfy all -formulae in the original model, and from the induction hypothesis.

For the case of finite satisfiability, we also present an alternative proof, which sheds more light on this fragment and its super-fragments. For details, see Appendix 0.A. Let denote equivalence of PCTL formulae over finite models.

###### Theorem 3.3

Let be a -formula. Then, the following equivalence holds:

 G=1ϕ≡finG=1(⋀l∈Al∧F=1G=1⋀l∈Bl∧⋀i∈IF=1⋀l∈Cil)

for appropriate , and .

###### Proof

The proof is based on the following auxiliary statements

 G=1G=1ϕ ≡G=1ϕ (3) G=1F⊳qϕ ≡finG=1F=1ϕ (4) F=1F⊳qϕ ≡F⊳qϕ (5)

and follows by induction.

The second statement is the most interesting one. Intuitively, it is a zero-one law, stating that infinitely repeating satisfaction with a positive probability ensures almost sure satisfaction. Notably, this only holds if the probabilities are bounded from below, hence for finite models, not necessarily for infinite models.

It is an easy corollary of this theorem that a satisfiable formula has a model of a circle form with and holding in each state and each element in each holding in some state. In general the models can be of a lasso shape where the initial (transient) part only has to satisfy , allowing for easy manipulation in extensions of this fragment.

###### Remark 1

Note that the equivalence does not hold over infinite models. Indeed, consider as simple a formula as , which is satisfied on the Markov chain of Fig. 3 , while this does not satisfy the transformed . Crucially, equivalence (4) does not hold already for this tiny fragment. Interestingly, when we build a model for the transformed formula, which is equisatisfiable but not equivalent, it turns out to be a model of the original formula. If, moreover, we consider then finite and general satisfiability start to differ.

Before we move on to the next fragment, we will prove another consequence of Lemma 4. It is a statement about the BSCCs of models for formulae in this fragment and will be used later for the proof of Theorem 3.5.

###### Corollary 1

Let be a satisfiable -formula and its model. Then, for every BSCC of , the following holds

1. For all , there is a state , such that .

2. For all , and for all states , .

###### Proof

Point 1 follows from the fact that every reachable BSCC must satisfy , and from Lemma 4. Point 2 follows immediately from point 1.

Note that we did not assume finite satisfiability here, so the model might not contain a single BSCC. In that case, the claim is trivially true. However, Theorem 3.2 allows us to focus on finitely satisfiable formulae in this fragment.

### 3.3 Satisfiability for G1(Fq,G1,∨)

This section treats -formulae of the fragment where only appears with . We thus lift a restriction of the previous fragment and allow for disjunctions. We generalize the obtained results to this larger fragment. We mentioned earlier that some of the results of the previous fragment do not apply here. Concretely, Lemma 4 does not hold here; that is, there might be subformulae which are not satisfied almost surely. Therefore, there is not necessarily a state that satisfies all -subformulae. For example, consider . There cannot be a single BSCC to satisfy both disjuncts. Although this is not a problem for the results of this section, it will turn out to be a fundamental problem when dealing with arbitrary formulae of the fragment.

###### Definition 7

-formulae are given by the grammar

 Φ ::=G=1Ψ Ψ ::=a∣¬a∣Ψ∧Ψ∣Ψ∨Ψ∣F⊳qΨ∣G=1Ψ

We prove that satisfiable formulae of this fragment are satisfiable by models of linear size and thus also finitely satisfiable.

###### Theorem 3.4

Let be a satisfiable -formula. Then has a model of size at most .

###### Proof

The proof for this theorem works essentially the same as it did for Theorem 3.2. Recall that we looked for a state to satisfy all -formulae. Though we will not necessarily find a state that does so in this fragment, we can look for a state that satisfies maximal subsets of satisfied -formulae. Then, we can continue in a similar way as we did in the simpler setting.

### 3.4 Satisfiability for Fq,G1

This section treats general formulae of the fragment with no disjunction and where only appears with .

###### Definition 8

-formulae are given by the grammar

 Φ::=a∣¬a∣Φ∧Φ∣F⊳qΦ∣G=1Φ

In Section 3.2 we discussed a special case of this fragment, where the top-level operator is . Two results are particularly interesting for this section: Firstly, the construction of models for such formulae as explained in the proof of Theorem 3.2, and secondly, the properties of BSCCs in models for such formulae as stated in Corollary 1. We will use those in order to simplify models in this generalized setting. We say a Markov chain has height if it is a tree with back edges of height .

###### Theorem 3.5

A satisfiable -formula has a model of height .

###### Proof

Our aim is to transform a given, possibly infinite model into a tree-like shape. To do so, we first construct a tree by considering all non-nested -formulae. Each path in this tree will satisfy each of these -formulae at most once. At the end of each path, we will then insert BSCCs satisfying the -formulae, in the spirit of Theorem 3.2.

The collapsing procedure from a state is as follows: We first determine which of the -formulae that are satisfied at are relevant. Those are the formulae which are not nested in other temporal formulae and have not yet been satisfied on the current path. Once we have determined this set, say , we need to find the successors which are required to satisfy the formulae in . For this, we construct the set . Informally, contains all states s.t. (i) satisfies at least one formula , and (ii) there is no state on the path between and satisfying . Formally, . Then, we connect to every state in directly; i.e. for , we set , and for all states , we set .111In fact, we need to scale in order to obtain a Markov chain, in general, as the probability to reach might be less than . For details, refer to the formal proof in the Appendix. A simple induction on the length of a path yields that every state that is reachable from in the constructed MC is reached with at least the probability as in the original one. From this, one can easily see that every non-nested -formula is satisfied. The new set might be infinite. However, we know that we can prune most of the successors and limit the branching degree to . Then, we repeat the procedure from each of the successors. Since the number of non-nested -formulae decreases with every step, we will reach states which do not have to satisfy non-nested -formulae at all on every branch. The number of steps we need to reach such states, is bounded by the number of non-nested -formulae in . At those states, we can use Theorem 3.2 to obtain models for the respective -formulae. Those are of size linear in the size of the -formulae. The overall height is then bounded by . The fact that the resulting MC is a model can be easily proved by induction over .

The models that we construct have a quite regular shape: They start as a tree and in every step ensure satisfaction of one of the -formulae. As soon as they have satisfied all outer -formulae, on every branch a model of circle shape for the respective -formula follows. Since the branching degree is at most and the number of steps before we repeat a state is bounded by , the overall size is bounded to .

###### Example 3

Let , and . The large Markov chain of Figure 4 is a model for . The grayed states illustrate the set . The other boxes show the sets of the respective grayed state. Everything in between is omitted. The smaller Markov chain is the reduced version of the original model.

### 3.5 Satisfiability for Fq/1,G1,∨

In the previous section we have been able to construct simple models for formulae of the -fragment by exploiting the nature of -formulae thereof as presented in Section 3.2. This works because every formula nested within a is satisfied in every BSCC. Hence, we can simply postpone the satisfaction of those until we reach a BSCCs. In the -fragment, this is not the case anymore, as discussed in Section 3.3. This can cause some complications, which are discussed in more detail in Section 3.6. In order to be able to apply similar techniques as in the previous section, we can simplify the fragment and enforce the property that -formulae occur only with within s.

###### Definition 9

-formulae are given by the grammar

 Φ::=a∣¬a∣Φ∧Φ∣Φ∨Φ∣F⊳qΦ∣G=1Ψ Ψ::=a∣¬a∣Ψ∧Ψ∣Ψ∨Ψ∣F=1Ψ∣G=1Ψ

Again, we show that the necessary minimal height of models can be bounded.

###### Theorem 3.6

A satisfiable -formula has a model of height .

###### Proof

As a first step, we apply the same procedure as in the proof of Theorem 3.5. The outer -formulae are then satisfied for the same reason as in the setting without disjunctions. However, the -nested -formulae might not be satisfied anymore because the BSCCs do not necessarily satisfy each of them. Since the -nested -formulae appear only with , we know that once a state of the original model satisfies such a formula, almost every path satisfies the respective path formula. Let be a state of the reduced chain, and . In the original model, there might be states in between and . If some -nested -formula (say ) which is satisfied at is also satisfied at , we do not need to take care of it. If this is not the case, then we know for sure that some of the states between and must satisfy . We can determine such states for each -nested -formula. We include exactly one of those for each such formula. Then, preserving the order, we chain them in such a way that each one has a unique successor. The last one’s unique successor is . Let be the first one. Then, we set . We repeat this procedure for each state of the reduced chain.

This way, we preserve the reachability probabilities and therefore the satisfaction of the outer -formulae. The newly added states guarantee the satisfaction of nested -formulae. An induction over shows that the constructed MC is again a model. Since we add at most new states between the states of the reduced MC which is of height at most , we obtain the claimed bound on the height.

Figure 5 illustrates the transformation of the models as described in the proof sketch. and are the selections. In the -fragment, we directly connected those sets. Here, we insert simple chains between the selections. The construction guarantees that we have at most one state per

-formula to satisfy. This is obtained by postponing the satisfaction until the last possible moment before

.

###### Remark 2

Note that the resulting models have a tree shape with BSCCs. There are no non-bottom SCCs. Even if the original model did have such, they are removed by this construction: The reduction algorithm takes care that every non-nested -formula occurs at most once on every path. The inserted chains contain at most one state per nested -formula and do not introduce cycles.

###### Example 4

Consider the formula

Figure 6 (a) shows a model for . The boxes illustrate the selection of . Figure 6 (b) shows the corresponding reduced chain. However, it is not a model for . The reason is that neither states satisfying nor such that satisfy are reached almost surely from . By including additional states, the chain in Figure 6 (c) corrects this, and thereby we obtain a model of .

### 3.6 Finite Models for Fq,G1,∨

In this section, we discuss PCTL where appears only with the contraint . Previously for the -fragment, i.e. without disjunctions, we started from a model which was unfolded for a number of steps; we simplified such a model by dropping states (including all non-bottom SCCs) and then we inserted simple chains that guarantee the satisfaction of the nested -formulae. The resulting model thus (i) does not contain any non-bottom SCCs and (ii) the size only depends on the structure of the formula, not on the constraints of the -formulae. However, in the general -fragment, we cannot insert such simple chains to satisfy nested s. Instead, we may have to branch at several places. Intuitively, the reason for such complications is the presence of a repeated, controlled choice. This enables us to find a formula which requires more complicated models, namely models which either have non-bottom SCCs or are of size which also depends on the constraints and not only on the structure of the formula, or can even be infinite.

###### Example 5

Consider . We can try to construct a model for as follows: Firstly, we have to start at a state which satisfies . This enforces the satisfaction of the first disjunct in the -formula. Therefore, almost all paths must lead to a state satisfying . This state must eventually reach a state that satisfies again with positive probability. Hence, we find ourselfs in the same situation as was the case in the initial state. So, we need to either create an SCC of alternating states that satisfy and , or create an infinite model. If we create an SCC, the side constraint enforces us to eventually leave this SCC. Hence, it is a non-bottom SCC. The MC in Figure 7 is a possible model. From it models , for any .

Note that the formula given in the example is qualitative. For this fragment, it is known how to solve the satisfiability problem already . However, we can easily adapt the formula to be quantitative. In this case, we might still be able to obtain a model for the quantitative version by keeping its shape and only adapting the probabilties of the model for the qualitative version. The question arises, whether this is possible in general or not. This question remains open and might be interesting for future work.

## 4 Discussion, Conclusion, and Future Work

We have identified the pattern of the controlled repeated choice, i.e. formulae of the form

 G=1(ϕ1∨⋯∨ϕn)

where at least one of the contains an -formula that has a constraint other than . Additionally, we have “controlling” side constraints, as in Example 5. We have seen that the presence of this pattern enforces more complicated structure of models even in the qualitative setting. This pattern is expressible in the -fragment. Whenever we

1. [label=()]

2. drop the side constraints, keeping only the -part, i.e. consider the -fragment, or

3. drop the disjunction for the choice and consider the -fragment, or

4. drop the quantity of the choice and consider the -fragment,

the structure is simpler and we obtain decidability. For these fragments, we have even shown that the general satisfiability problem is equivalent to the finite satisfiability problem.

Further, adding quantities to -constraints obviously also makes the satisfiability problem more complicated. Already for the qualitative -fragment, satisfiability and finite satisfiability differ. Nevertheless, we established the decidability of finite satisfiability even for the -fragment.

Consequently, instead of attacking the whole quantitative PCTL or even just PCTL(), we suggest two easier tasks, which should lead to a fundamental increase of understanding the general problem, namely:

• finite (and also general) satisfiability of the -fragment, i.e. PCTL() where is limited to the constraint, and

• infinite satisfiability of the -fragment, i.e. -formulae of PCTL().

While the former omits issues stemming from  and only deals with the repeated choice, the latter generalizes the qualitative results for [17, 7] in the presence of general quantitative ’s.

Further, potentially more straight-forward, directions include the generalization of the results obtained in this paper to the until- and release-operators instead of future- and globally-operators, respectively, or the introduction of the next-operator.

## References

•  C. Baier, M. Größer, M. Leucker, B. Bollig, and F. Ciesinski. Controller synthesis for probabilistic systems. In Proceedings of IFIP TCS’2004, pages 493–506, 2004.
•  Christel Baier and Joost-Pieter Katoen. Principles of model checking. MIT press, 2008.
•  B. Banieqbal and H. Barringer. Temporal logic with fixed points. In Temporal Logic in Specification, volume 398, pages 62–74, 1987.
•  Nathalie Bertrand, John Fearnley, and Sven Schewe. Bounded satisfiability for PCTL. In Computer Science Logic (CSL’12) - 26th International Workshop/21st Annual Conference of the EACSL, CSL 2012, September 3-6, 2012, Fontainebleau, France, pages 92–106, 2012.
•  T. Brázdil, V. Brožek, V. Forejt, and A. Kučera. Stochastic games with branching-time winning objectives. In Proceedings of LICS 2006, pages 349–358, 2006.
•  T. Brázdil, V. Forejt, and A. Kučera. Controller synthesis and verification for Markov decision processes with qualitative branching time objectives. In Proceedings of ICALP 2008, 2008.
•  T. Brázdil, V. Forejt, J. Křetínský, and A. Kučera. The satisfiability problem for probabilistic CTL. In LICS, pages 391–402, 2008.
•  T. Brázdil, A. Kučera, and O. Stražovský. On the decidability of temporal properties of probabilistic pushdown automata. In Diekert and Durand , pages 145–157.
•  Souymodip Chakraborty and Joost-Pieter Katoen. On the satisfiability of some simple probabilistic logics. In Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science, pages 56–65. ACM, 2016.
•  C. Courcoubetis and M. Yannakakis. The complexity of probabilistic verification. Journal of the ACM, 42(4):857–907, 1995.
•  Volker Diekert and Bruno Durand, editors. STACS 2005, 22nd Annual Symposium on Theoretical Aspects of Computer Science, Stuttgart, Germany, February 24-26, 2005, Proceedings, volume 3404 of Lecture Notes in Computer Science. Springer, 2005.
•  E. A. Emerson and J. Y. Halpern. Decision procedures and expressiveness in the temporal logic of branching time. In Proceedings of STOC’82, pages 169–180, 1982.
•  J. Esparza, A. Kučera, and R. Mayr. Model-checking probabilistic pushdown automata. 2(1:2):1–31, 2006.
•  K. Etessami and M. Yannakakis. Recursive Markov chains, stochastic grammars, and monotone systems of non-linear equations. In Diekert and Durand , pages 340–352.
•  M. Fischer and R. Ladner. Propositional dynamic logic of regular programs. Journal of Computer and System Sciences, 18:194–211, 1979.
•  H. Hansson and B. Jonsson. A logic for reasoning about time and reliability. 6:512–535, 1994.
•  S. Hart and M. Sharir. Probabilistic propositional temporal logics. Information and Control, 70(2/3):97–155, 1986.
•  T. Henzinger, O. Kupferman, and R. Majumdar. On the universal and existential fragments of the -calculus. Theoretical Computer Science, 354(2):173–186, 2006.
•  M. Huth and M.Z. Kwiatkowska. Quantitative analysis and model checking. In Proceedings of LICS’97, pages 111–122, 1997.
•  D. Kozen. A finite-model theorem for the propositional -calculus. 47(3):233–241, 1988.
•  S. Kraus and D. J. Lehmann. Decision procedures for time and chance (extended abstract). In FOCS, pages 202–209, 1983.
•  A. Kučera and O. Stražovský. On the controller synthesis for finite-state Markov decision processes. In Proceedings of FST&TCS 2005, volume 3821, pages 541–552, 2005.
•  O. Kupferman and M.Y. Vardi. An automata-theoretic approach to modular model checking. 22:87–128, 2000.
•  D. J. Lehmann and S. Shelah. Reasoning with time and chance (extended abstract). In ICALP, volume 154, pages 445–457, 1983.
•  J.R. Norris. Markov Chains. 1998.

## Appendix 0.A Full Proofs

Whenever we write we implicitly mean a Markov chain and similarly stands for , and so on. Moreover, we will frequently refer to the formulae satisfied at by .

### 0.a.1 Finite satisfiability for Gq(Fq,Gq,∨)

Lemma 3. Let be a -formula. Then, is finitely equisatisfiable to a -formula , such that .

###### Proof

Write . Let be a finite model for . Then, there must be at least one BSCC , and a state , such that . For a formula , we define recursively as follows

 ^ξ:=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩a if ξ=a^ζ∧^ϑ if ξ=ζ∧ϑ^ζ∨^ϑ if ξ=ζ∨ϑF=1^ζ if ξ=F⊳rζG=1^ζ if ξ=G⊳rζ.

Let , , and . We will show that .

1. [label=]

2. . Then, , and thus there is nothing to show.

3. . Then, and . By the induction hypothesis, and , and thus .

4. . Then, or . By the induction hypothesis, or . Thus, .

5. . Then, there is a state , such that . By the induction hypothesis, . Since is a BSCC, is reached almost surely. Therefore, .

6. . Assume there was a state , such that . Since is a BSCC, is reached almost surely, and therefore , which is a contradiction. Hence, , for all . By the induction hypothesis, , for all . Finally, this implies that .

We have now shown that if is satisfiable, then so is . Moreover, it is obvious that . The other direction follows immediately from this fact. Hence, is equisatisfiable to .

Theorem 3.1. Let be a finitely satisfiable formula. Then, there is a model of size linear in .

###### Proof

By Lemma 3 we can consider instead of . Let be a model for . Let denote all states that satisfy . Then, we define , such that , , and an arbitrary distribution that generates a BSCC from —e.g. a circle. Note that . In particular it is always finite, even if is infinite.

Let and , such that for all , . Then we will prove that by induction over .

1. [label=]

2. or . Assume . Then, for all , . Hence, and . The case is analagous.

3. . Then, for all , and . By the induction hypothesis, and and therefore .

4. . Then, for all , or . By the induction hypothesis, or . Hence,