    # Parametric Markov Chains: PCTL Complexity and Fraction-free Gaussian Elimination

Parametric Markov chains have been introduced as a model for families of stochastic systems that rely on the same graph structure, but differ in the concrete transition probabilities. The latter are specified by polynomial constraints for the parameters. Among the tasks typically addressed in the analysis of parametric Markov chains are (1) the computation of closed-form solutions for reachabilty probabilities and other quantitative measures and (2) finding symbolic representations of the set of parameter valuations for which a given temporal logical formula holds as well as (3) the decision variant of (2) that asks whether there exists a parameter valuation where a temporal logical formula holds. Our contribution to (1) is to show that existing implementations for computing rational functions for reachability probabilities or expected costs in parametric Markov chains can be improved by using fraction-free Gaussian elimination, a long-known technique for linear equation systems with parametric coefficients. Our contribution to (2) and (3) is a complexity-theoretic discussion of the model checking problem for parametric Markov chains and probabilistic computation tree logic (PCTL) formulas. We present an exponential-time algorithm for (2) and a PSPACE upper bound for (3). Moreover, we identify fragments of PCTL and subclasses of parametric Markov chains where (1) and (3) are solvable in polynomial time and establish NP-hardness for other PCTL fragments.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Finite-state Markovian models are widely used as an operational model for the quantitative analysis of systems with probabilistic behaviour. In many cases, only estimates of the transition probabilities are available. This, for instance, applies to fault-tolerant systems where the transition probabilities are derived from error models obtained using statistical methods. Other examples are systems operating with resource-management protocols that depend on stochastic assumptions on the future workload, or cyber-physical systems where the interaction with its environment is represented stochastically. Furthermore, often the transition probabilities of Markovian models depend on configurable system parameters that can be adjusted at design-time. The task of the designer is to find a parameter setting that is optimal with respect to a given objective. This motivated the investigation of

interval Markov chains (IMCs)  specifying intervals for the transition probabilities (rather than concrete values). More general is the model of parametric Markov chains (pMCs), which has been introduced independently by Daws  and Lanotte et al. , where the transition probabilities are given by polynomials with rational coefficients over a fixed set of real-valued parameters . These concepts can be further generalized to accommodate rational functions, i. e., quotients of polynomials, as transition probabilities (see, e. g., ).

It is well-known that the probabilities for reachability conditions in parametric Markov chains with a finite state space can be characterized as the unique solution of a linear equation system where

is the solution vector, and

is a matrix where the coefficients are rational functions. Likewise, is a vector whose coefficients are rational functions. Note that it is no limitation to assume that the entries in and are polynomials, as rational function entries can be converted to a common denominator, which can then be removed. Now, can be viewed as a linear equation system over the field of rational functions with rational coefficients. As a consequence, the probabilities for reachability conditions are rational functions. This has been observed independently by Daws  and Lanotte et al.  for pMCs. Daws  describes a computation scheme that relies on a state-elimination algorithm inspired by the state-elimination algorithm for computing regular expressions for nondeterministic finite automata. This, however, is fairly the same as Gaussian elimination for matrices over the field of rational functions.

As observed by Hahn et al. , the naïve implementation of Gaussian elimination for pMCs, that treats the polynomials in and as syntactic atoms, leads to a representation of the rational functions as the quotient of extremely (exponentially) large polynomials. In their implementation PARAM  (as well as in the re-implementation within the tool PRISM ), the authors of  use computer-algebra tools to simplify rational functions in each step of Gaussian elimination by identifying the greatest common divisor (gcd) of the numerator and the denominator polynomial. Together with polynomial-time algorithms for the gcd-computation of univariate polynomials, this approach yields a polynomial-time algorithm for computing the rational functions for reachability probabilities in pMCs with a single parameter. Unfortunately, gcd-computations are known to be expensive for the multivariate case (i. e., ) . To mitigate the cost of the gcd-computations, the tool Storm  successfully uses techniques proposed in  such as caching and the representation of the polynomials in partially factorized form during the elimination steps. However, it is possible to completely avoid gcd-computations by using one-step fraction-free Gaussian elimination. Surprisingly, this has not yet been investigated in the context of pMCs, although it is a well-known technique in mathematics. According to Bareiss , this variant of Gaussian elimination probably goes back to Camille Jordan (1838–1922), and has been rediscovered several times since. Like standard Gaussian elimination it relies on the triangulation of the matrix, and finally obtains the solution by back substitution. Applied to matrices over polynomial rings the approach generates matrices with polynomial coefficients (rather than rational functions) and ensures that the degree of the polynomials in all intermediate matrices grows at most linearly. This is achieved by dividing, in each elimination step, by a factor known by construction. Thus, when applied to a pMC with linear expressions for the transition probabilities, the degree of all polynomials in the solution vector is bounded by the number of states. For the univariate case (

), this yields an alternative polynomial-time algorithm for the computation of the rational functions for reachability probabilities. Analogous statements hold for expectations of random variables that are computable via linear equation systems. This applies to expected accumulated weights until reaching a goal, and to the expected mean payoff.

Contribution. The purpose of the paper is to study the complexity of the model checking problem for pMCs and probabilistic computation tree logic (PCTL) , and its extensions by expectation operators for pMCs augmented by weights for its states. In the first part of the paper (Section 3), we discuss the use of Bareiss’ one-step fraction-free Gaussian elimination for the computation of reachability probabilities. The second part of the paper (Section 4) presents complexity-theoretic results for the PCTL model checking problem in pMCs. We describe an exponential-time algorithm for computing a symbolic representation of all parameter valuations under which a given PCTL formula holds, and provide a PSPACE upper bound for the decision variants that ask whether a given PCTL formula holds for some or all admissible parameter valuations. The known NP-/coNP-hardness results for IMCs [27, 7] carry over to the parametric case. We strengthen this result by showing that the existential PCTL model checking problem remains NP-hard even for acyclic pMCs and PCTL formulas with a single probability operator. For the univariate case, we prove NP-completeness for the existential PCTL model checking problem, and identify two fragments of PCTL where the model checking is solvable in polynomial time. The first fragment are Boolean combinations of threshold constraints for reachability probabilities, expected accumulated weights until reaching a goal, and expected mean payoffs. The second fragment consists of PCTL formulas in positive normal form with lower probability thresholds interpreted over pMCs satisfying some monotonicity properties. Furthermore, we observe that the model checking problem for PCTL with expectation operators for reasoning about expected costs until reaching a goal is in P for Markov chains where the weights of the states are given as polynomials over a single parameter, when restricting to Boolean combinations of the expectation operators.

Proofs and further details on the experiments omitted in the main part due to space constraints can be found in the extended version .

Related work. Fraction-free Gaussian elimination is well-known in mathematics, and has been further investigated in various directions for matrices over unique factorization domains (such as polynomial rings), see e. g. [24, 20, 28, 25]. To the best of our knowledge, fraction-free Gaussian elimination has not yet been studied in the context of parametric Markovian models.

Besides the above mentioned work [9, 14, 15, 18, 10] on the computation of the rational functions for reachability probabilities in pMCs,  identifies instances where the parameter synthesis problem for pMCs with 1 or 2 parameters and probabilistic reachability constraints is solvable in polynomial time. These rely on the fact that there are closed-form representations of the (complex) zero’s for univariate polynomials up to degree 4 and rather strong syntactic characterizations of pMCs. In Section 3 we will provide an example to illustrate that the number of monomials in the numerators of the rational functions for reachability probabilities can grow exponentially in the number of states. We hereby reveal a flaw in  where the polynomial-time computability of the rational functions for reachability probabilities has been stated even for the multivariate case.  considers an approach for solving the parametric linear equation system obtained from sparse pMCs via Laplace expansion.

Model checking problems for IMCs and temporal logics have been studied by several authors. Most in the spirit of our work on the complexity of the PCTL model checking problem for pMCs is the paper  which studies the complexity of PCTL model checking in IMCs. Further complexity-theoretic results of the model checking problem for IMCs and temporal logics have been established in  for omega-PCTL (extending PCTL by Boolean combinations of Büchi and co-Büchi conditions), and in  for linear temporal logic (LTL). Our results of the second part can be seen as an extension of the work [27, 7] for the case of pMCs. The NP lower bound for the multivariate case and a single threshold constraint for reachability probabilities strengthen the NP-hardness results of .

There exist several approaches to obtain regions of parameter valuations of a pMC in which PCTL formulas are satisfied or not, resulting in an approximative covering of the parameter space. PARAM [15, 14]

employs a heuristic, sampling based approach, while PROPhESY

 relies on SMT solving via the existential theory of the reals to determine whether a given formula holds for all valuations in a sub region. For the same problem, 

uses a parameter lifting technique that avoids having to solve the parametric equation system by obtaining lower and upper bounds for the values in a given region by a reduction to non-parametric Markov decision processes.

## 2 Preliminaries

The definitions in this section require a general understanding of Markov models, standard model checking, and temporal logics. More details can be found, e. g., in

[21, 2].

Discrete-time Markov chain. A (discrete-time) Markov chain (MC) is a tuple where is a non-empty, finite set of states with the initial state , is a transition relation, and is the transition probability function satisfying if and only if , and for all with nonempty. We refer to as the graph of . A state in which is called a trap (state) of .

An infinite path in is an infinite sequence of states such that for . Analogously, a finite path in is a finite sequence of states in such that for . A path is called maximal if it is infinite or ends in a trap. denotes the set of all maximal paths in starting in . Relying on standard techniques, every MC induces a unique probability measure on the set of all paths.

Parameters, polynomials, and rational functions. Let be parameters that can assume any real value, . We write for the polynomial ring over the rationals with variables . Each can be written as a sum of monomials, i. e., where is a finite subset of and . If is empty, or for all tuples , then is the null function, generally denoted by 0. The degree of is where . A linear function is a function with . A rational function is a function of the form with , . The field of all rational functions is denoted by . We write for the set of all polynomial constraints of the form where , and .

Parametric Markov chain. A (plain) parametric Markov chain on , pMC for short, is a tuple where , , and are defined as for MCs, and is the transition probability function with , i. e., the null function, iff . Intuitively, a pMC defines the family of Markov chains arising by plugging in concrete values for the parameters. A parameter valuation is said to be admissible for if for each state we have if nonempty, and iff , where for all . Let , or briefly , denote the set of admissible parameter valuations for . Given the Markov chain associated with is . The semantics of the pMC is then defined as the family of Markov chains induced by admissible parameter valuations, i. e., .

An augmented pMC is a tuple where , , , and are defined as for plain pMCs, and is a finite set of polynomial constraints. A parameter valuation is admissible for an augmented pMC if it is admissible for the induced plain pMC , and satisfies all polynomial constraints in . As for plain pMC, we denote the set of admissible parameter valuations of an augmented pMC by , or briefly .

A, possibly augmented, pMC is called linear, or polynomial, if all transition probability functions and constraints are linear functions in , or polynomials in , respectively.

Interval Markov chain. An interval Markov chain (IMC)  can be seen as a special case of a linear augmented pMC with one parameter for each edge , and linear constraints for each edge with and . According to the terminology introduced in , this corresponds to the semantics of IMC as an “uncertain Markov chain”. The alternative semantics of IMC as a Markov decision process will not be considered in this paper.

Labellings and weights. Each of these types of Markov chain, whether MC, plain or augmented pMC, or IMC, can be equipped with a labelling function , where is a finite set of atomic propositions. If not explicitly stated, we assume the implicit labelling of the Markov chain defined by using the state names as atomic propositions and assigning each name to the respective state. Furthermore, we can extend any Markov chain with a weight function . The value assigned to a specific state is called the weight of . It is sometimes also referred to as the reward of . In addition to assigning rational values we also consider parametric weight functions .

Probabilistic computation tree logic. We augment the standard notion of probabilistic computation tree logic with operators for the expected accumulated weight and mean payoff, and for comparison. Let be a finite set of atomic propositions. stands for , or , , . Then

where . The basic temporal modalities are (next) and (until). The usual derived temporal modalities (eventually), (release) and (always) are defined by , and , where, e. g., is and is , and .

For an MC with states labelled by we use the standard semantics. We only state the semantics of the probability, expectation, and comparison operators here. For each state , iff , and iff . Here is short for . Furthermore, iff , and iff , where denotes the expected value of the respective random variable. For detailed semantics of the expectation operators, see . We write iff .

Notation: PCTL+EC and sublogics. We use PCTL to refer to unaugmented probabilistic computation tree logic. If we add only the expectation operator we write PCTL+E, and, analogously, PCTL+C if we only add the comparison operator for probabilities. PCTL+EC denotes the full logic defined above.

DAG-representation and length of formulas. We consider for any PCTL+EC state formula the directed acyclic graph (DAG) representing its syntactic structure. Each node of the DAG represents one of the sub-state formulas. The use of a DAG rather than the syntax tree allows the representation of subformulas that occur several times in the formula by a single node. The leaves of the DAG can be the Boolean constant true, and atomic propositions. The inner nodes of the DAG, e. g., of a PCTL formula, are labelled with one of the operators , , , . Nodes labelled with and have a single outgoing edge, while nodes labelled with or have two outgoing edges. For the above-mentioned extensions of PCTL the set of possible inner node labels is extended accordingly. For example, a node representing the PCTL+C formula has three outgoing edges. If then there are two parallel edges from to a node representing . The length of a PCTL+EC formula is defined as the number of nodes in its DAG.

## 3 Fraction-free Gaussian elimination

Given a pMC as in Section 2, the probabilities for reachability conditions are rational functions and computable via Gaussian elimination. As stated in the introduction, this has been originally observed in [9, 23] and realized, e. g., in the tools PARAM  and Storm [10, 11] together with techniques based on gcd-computations on multivariate polynomials. In this section, we discuss the potential of fraction-free Gaussian elimination as an alternative, which is well-known in mathematics [3, 13], but to the best of our knowledge, has not yet been considered in the context of pMCs.

While the given definitions allow for rational functions in the transition probability functions of (augmented) pMCs, we will focus on polynomial (augmented) pMCs throughout the remainder of the paper. Generally, a linear equation systems containing rational functions as coefficients can be rearranged to one containing only polynomials by multiplying each line with the common denominator of the respective rational functions. Due to the multiplications this involves the risk of a blow-up in the coefficient size. To avoid this we add variables in the following way. Let be an (augmented) pMC. For all introduce a fresh variable . By definition for some . Let if , if , . Then is a polynomial augmented pMC.

Linear equation systems with polynomial coefficients. Let be parameters, . We consider linear equation systems of the form , where is a non-singular -matrix with . Likewise, is a vector of length with . The solution vector is a vector of rational functions with . By Cramer’s rule we obtain , where is the determinant of , and the determinant of the matrix obtained when substituting the -th column of by . If the coefficients of and have at most degree , the Leibniz formula implies that and have at most degree .

###### Lemma 1.

There is a family of acyclic linear pMCs where has parameters and states, including distinguished states and , such that is a polynomial for which even the shortest sum-of-monomial representation has monomials.

One-step fraction-free Gaussian elimination is a variant of fraction-free Gaussian elimination that allows for divisions which are known to be exact at the respective point in the algorithm. When using naïve fraction-free Gaussian elimination the new coefficients after the -th step, , are computed as for , where . When applied to systems with polynomial coefficients this results in the degree doubling after each step, so the degree grows exponentially. In a step of one-step fraction-free Gaussian elimination (see Algorithm 1), the computation of the coefficients changes to with . Using Sylvester’s identity one can prove that is again a polynomial, and that is in general the maximal possible divisor. The are updated analogously. If the maximal degree of the initial coefficients of and is , this technique therefore guarantees, that after steps the degree of the coefficients is at most , i. e., it grows linear in during the procedure. For polynomials the division by can be done using standard polynomial division. The time-complexity of the exact multivariate polynomial division in this case is in each step , so for the full one-step fraction-free Gaussian elimination it is .

Proposition 4.3 in  states that the rational functions for reachability probabilities in pMC with a representation of the polynomials , as sums of monomials (called normal form in ) are computable in polynomial time. This contradicts Lemma 1 which shows that the number of monomials in the representation of a reachability probability as a sum of monomials can be exponential in the number of parameters. However, the statement is correct for the univariate case.

###### Lemma 2.

Let be a polynomial pMC over a single parameter and a set of states. Then, the rational functions for the reachability probabilities are computable in polynomial time. Analogously, rational functions for the expected accumulated weight until reaching or the expected mean payoff are computable in polynomial time.

Note that the degrees of the polynomials and computed by one-step fraction-free Gaussian elimination for reachability probabilities are bounded by , where , so the polynomials have representations as sums of at most monomials. In particular, the degree and representation size of the final polynomials and for the rational functions is in where is the number of states of . Another observation concerns the case where only the right-hand side of the linear equation system is parametric. Systems of this form occur, e. g., when considering expectation properties for MCs with parametric weights.

###### Lemma 3.

Let be a parametric linear equation system as defined above where is parameter-free. Then the solution vector consist of polynomials of the form with and can be computed in polynomial time.

Stratification via SCC-decomposition. It is well known (e. g., [8, 18]

) that for probabilistic/parametric model checking a decomposition into strongly-connected components (SCCs) can yield significant performance benefits due to the structure of the underlying models. We have adapted the one-step fraction-free Gaussian elimination approach by a preprocessing step that permutes the matrix according to the topological ordering of the SCCs. This results in the coefficient matrix already having a stair-like form at the start of the algorithm. In the triangulation part of the algorithm, each SCC can now be considered separately, as non-zero entries below the main diagonal only occur within each SCC. While the back-substitution in the general one-step fraction-free elimination will result in each entry on the main diagonal being equal to the last, this property is now only maintained within the SCCs. Formally, this means that the back substitution step in Algorithm

1 is replaced by the following:

 b_m = (a∗(current SCC)⋅b_m−∑_i=m+1na_m,i⋅b_i⋅a∗(current SCC)a∗(SCC at i))/a_m,m

where , and, for , if the -th and -st state belong to the same SCC and otherwise. Intuitively, is the product of the ’s on the diagonal corresponding to the last states in the current SCC and the SCCs below. Of course, the return statement also has to be adjusted accordingly. The advantage of this approach is that the polynomials in the rational functions aside from the ones in the first strongly connected component will have an even lower degree.

Implementation and experiments. For a first experimental evaluation of the one-step fraction-free Gaussian elimination approach (GE-ff) in the context of probabilistic model checking, we have implemented this method (including the SCC decomposition and topological ordering described above) as an alternative solver for parametric linear equation systems in the state-of-the-art probabilistic model checker Storm . We compare GE-ff against the two solvers provided by Storm (v1.0.1) for solving parametric equation systems, i. e., the solver based on the eigen linear algebra library and on state elimination (state-elim. Both of Storm’s solvers use partially factorized representations of the rational functions provided by the CArL library. This approach, together with caching, was shown  to be beneficial due to improved performance of the gcd-computations during the simplification steps.

It should be noted that our implementation is intended to provide first results that allow to gauge whether the fraction-free method, by avoiding gcd-computations, can be beneficial in practice and is thus rather naïve in certain aspects. As an example, it currently relies on a dense matrix representation, with performance improvements for larger models to be expected from switching to sparse representations as used in Storm’s eigen and state-elim solvers. In addition to the fraction-free approach, our solver can also be instantiated to perform a straight-forward Gaussian elimination, using any of the representations for rational functions provided by the CArL library. In all our experiments, we have compared the solutions obtained by the different solvers and verified that they are the same.

Experimental studies. The source code of our extension of Storm and the artifacts of the experiments are available online. As our GE-ff implementation is embedded as an alternative solver in Storm, we mainly report the time actually spent for solving the parametric equation system, as the other parts of model checking (model building, precomputations) are independent of the chosen solver. For benchmarking, we used a machine with two Intel Xeon E5-2680 8-core CPUs at 2.70GHz and with 384GB RAM, a time out of 30 minutes and a memory limit of 30GB. All the considered solvers run single-threaded. We have considered three different classes of case studies for experiments.

Complete pMC. As a first experiment to gauge the efficiency in the presence of a high ratio of parameters to states, we considered a family of pMCs with a complete graph structure (over states) and one parameter per transition, resulting in parameters (for details see ).

Table 1 depicts statistics for the corresponding computations, for the two standard solvers in Storm (eigen and state-elim), as well as our fraction-free implementation (GE-ff). For state-elim, we always use the default elimination order (forward). The time for GE-ff corresponds to the time until a solution rational function (for all states) is obtained. As the numerator and denominator of these rational functions are not necessarily coprime, for comparison purposes we list as well the time needed for simplification (red(GE-ff)) via division by the gcd. As can be seen, here, the fraction-free approach significantly outperforms Storm’s standard solvers and scales to a higher number of parameters. We confirmed using profiling that the standard solvers indeed spend most of the time in gcd-computations.

Multi-parameter Israeli-Jalfon self-stabilizing. The benchmarks used to evaluate parametric model checking implementations in previous papers tend to be scalable in the number of components but use a fixed number of parameters, usually 2. To allow further experiments with an increasing number of parameters, we considered a pMC-variant of the Israeli-Jalfon self-stabilizing protocol with processes, initial tokens and parameters (for details see ).

Table 2 depicts the time spent for computing the rational functions for several instances. As can be seen, the fraction-free approach is competitive for the smaller instances, with performance between the eigen and state-elim solvers for the larger instances. We have also included running times for GE-fac, i. e., for our naïve implementation of Gaussian elimination using the representation for rational functions as used by Storm for the standard solvers, including automatic gcd-based simplification after each step to ensure that numerator and denominator are coprime. GE-fac operates on the same, topologically sorted matrix as the fraction-free GE-ff. Curiously, GE-fac is able to outperform the eigen solver for some of the larger instances. We believe this is mainly due to differences in the matrix permutation and their effect on the elimination order, which is known to have a large impact on performance (e. g., ).

Benchmark case studies from . Furthermore, we considered several case study instances that were used in  to benchmark parametric model checkers, namely the brp, crowds, egl, nand, zeroconf models. Table 3 depicts statistics for selected instances, for further details see . The application of bisimulation quotienting often has a large impact on the size of the linear equation system, so we performed experiments with and without quotienting. For crowds, bisimulation quotienting was particularly effective, with all considered instances having a very small state space and negligible solving times. For the non-quotiented instances, Storm’s standard solvers outperform GE-ff. For the zeroconf instance in Table 3, GE-ff is competitive. Note that the models in the brp, egl and nand case studies are acyclic and that the parametric transition probabilities and rewards are polynomial. As a consequence, the gcd-computations used in Storm’s solvers don’t impose a significant overhead as the rational functions during the computation all have denominator polynomials of degree zero.

Overall, the experiments have shown that there are instances where the fraction-free approach can indeed have a positive impact on performance. Keeping in mind that our implementation has not yet been significantly optimized, we believe that the fraction-free approach is an interesting addition to the gcd-based solver approaches. In particular, the application of better heuristics for the order of processing (i. e., the permutation of the matrix) could still lead to significant performance increases.

## 4 Complexity of the PCTL+EC model checking problem

We now study the complexity of the following variants of the PCTL+EC model checking problem. Given an augmented pMC and a PCTL+EC (state) formula :

(All) Compute a representation of the set of all satisfying parameter valuations,

i. e., the set of all admissible parameter valuations such that .
(MC-E) Does there exist a valuation such that ?
(MC-U) Does hold for all admissible valuations ?

(MC-E) and (MC-U) are essentially duals of each other. Note that the answer for the universal variant (MC-U) is obtained by the negation of the answer for (MC-E) with formula , and vice versa. In what follows, we shall concentrate on (All) and the existential model checking problem (MC-E).

Computing all satisfying parameter valuations. As before, denotes the set of admissible valuations. In what follows, let be the conjunction of the polynomial constraints in as well as the constraints for each non-trap state , and for each edge . We then have if and only if is admissible, i. e., .

Let be a PCTL+EC formula. The satisfaction function is defined by:

 Sat_M(Φ)(¯¯¯ξ) \tiny def={s∈S:s⊨_M(¯¯¯ξ)Φ}=Sat_M(¯¯¯ξ)(Φ)

We now present an algorithm to compute a symbolic representation of the satisfaction function that groups valuations with the same corresponding satisfaction set together. More precisely, we deal with a representation of the satisfaction function by a finite set of pairs where is a Boolean combination of constraints and such that (i) and implies , and (ii) whenever then there is a pair such that .

Given the DAG representation of the PCTL formula , we follow the standard model checking procedure for CTL-like branching-time logics and compute for the subformulas assigned to the nodes in the DAG for in a bottom-up manner. As the leaves of the DAG can be atomic propositions or the formula true, the base cases are , and . Consider now the inner node of the DAG for labelled by the outermost operator of the subformula . Suppose that the children of have already been treated, so when computing the satisfaction sets of the proper subformulas of are known. If is labelled by or , i. e., or , then respectively . If , then

 Sat_M(Ψ) = { (γ_1∧γ_2∧δ_γ_1,T_1,γ_2,T_2,R) : (γ_1,T_1)∈Sat_M(Ψ_1), (γ_2,T_2)∈%Sat_M(Ψ_2), R⊆S }

where is the conjunction of the constraints for each state , and for each state . Here, is the rational function that has been computed using (i) a graph analysis to determine the set of states with and (ii) fraction-free Gaussian elimination (Section 3) to compute the rational functions in the pMC resulting from by turning the states in into traps. If and are polynomials computed by fraction-free Gaussian elimination such that then is a shortform notation for . The treatment of and the expectation operators is similar, and can be found in . After treating a node of the DAG, we can simplify the set by first removing all pairs where is not satisfiable (using algorithms for the existential theory of the reals), and afterwards combining all pairs with the same -component, that is, instead of pairs , we consider a single pair . To answer question (All), the algorithm finally returns the disjunction of all formulas with for .

Complexity bounds of (All) and (MC-E). The existential theory of the reals is known to be in PSPACE and NP-hard, and there is an upper bound on the time-complexity, namely where is the number of constraints, the maximum degree of the polynomials in the constraints, and the number of parameters . Recall from Section 3 that a known upper bound on the time-complexity of one-step fraction-free Gaussian elimination is , where is the number of equations, the maximum degree of the initial coefficient polynomials, and the number of parameters. Combining both approaches, the one-step fraction-free Gaussian elimination for solving linear equation systems with polynomial coefficients, and the existential theory of the reals for treating satisfiability of conjunctions of polynomial constraints, one directly obtains the following bound for the computational complexity of PCTL+EC model checking on augmented polynomial pMCs. Note that this assumes that the number of constraints in is at most polynomial in the size of .

###### Theorem 4 (Exponential-time upper bound for problem (All)).

Let be a PCTL+EC formula. Given an augmented polynomial pMC , where the maximum degree of transition probabilities , and polynomials in the constraints in is , a symbolic representation of the satisfaction function is computable in time , where is the number of probability, expectation and comparison operators in .

###### Theorem 5 (PSPACE upper bound for problem (MC-E)).

The existential PCTL+EC model checking problem (MC-E) for augmented pMC is in PSPACE.

• The main idea of a polynomially space-bounded algorithm is to guess nondeterministically sets of states for the subformulas where the outermost operator is a probability, expectation or comparison operator, and then apply a polynomially space-bounded algorithm for the existential theory of the reals  to check whether there is a parameter valuation such that for all . ∎

NP- and coNP-hardness of (MC-E) follow from results for IMCs [27, 7]. More precisely,  provides a polynomial reduction from SAT to the (existential and universal) PCTL model checking problem for IMCs. In fact, the reduction of  does not require full PCTL, instead Boolean combinations of simple probabilistic constraints without nesting of the probability operators are sufficient. The following theorem strengthens this result by stating NP-hardness of (MC-E) even for formulas consisting of a single probability constraint for a reachability condition.

###### Theorem 6 (NP-hardness for single probabilistic operator, multivariate case).

Given an augmented polynomial pMC  on parameters with initial state and an atomic proposition , and a probability threshold , the problem to decide whether there exists such that is NP-hard, even for acyclic pMCs with the assigned transition probabilities being either constant, or linear in one parameter, i. e., , , for all , and where the polynomial constraints for the parameters are of the form with , .

Univariate pMCs. In many scenarios, the number of variables has a fixed bound instead of increasing with the model size. We consider here the case of univariate pMC, i. e., pMC with a single parameter.

###### Theorem 7 (PCTL+EC model checking without nesting in P, univariate case).

Let be a PCTL+EC formula without nested probability, expectation or comparison operators, and let be a polynomial pMC on the single parameter . The problem to decide whether there exists an admissible parameter valuation such that is in P.

• If we restrict PCTL+EC to Boolean combinations of probability, expectation, and comparison operators, (MC-E) can be dealt with by first computing polynomial constraints for for each probability, expectation, and comparison operator independently (this can be done in polynomial time by Lemma 2), and afterwards applying a polynomial-time algorithm for the univariate existential theory of the reals  once to the appropriate Boolean combination of the constraints. ∎

###### Theorem 8 (NP-completeness for full PCTL+EC, univariate case).

Let be a PCTL+EC formula, and let be a polynomial pMC on the single parameter . The PCTL+EC model checking problem to decide whether there exists an admissible parameter valuation such that is NP-complete. NP-hardness even holds for acyclic polynomial pMCs and the fragment of PCTL+C that uses the comparison operator , but not the probability operator , as well as for (cyclic) polynomial pMC in combination with PCTL.

(MC-E) for monotonic PCTL on univariate pMCs. The parameters in pMC typically have a fixed meaning, e. g., probability for the occurrence of an error, in which case the probability to reach a state where an error has occurred is increasing in . This motivates the consideration of univariate pMCs and PCTL formulas that are monotonic in the following sense.

Given a univariate polynomial pMC , let denote the set of edges such that the polynomial is monotonically increasing in , i. e., whenever and then . Let denote the set of states such that for each finite path with we have for .

As iff there is no value such that , the set is computable in polynomial time using a polynomial-time algorithm for the univariate theory of the reals . Here, is as before the Boolean combination of polynomial constraints characterizing the set of admissible parameter values, and is the first derivative of the polynomial . Thus, the set is computable in polynomial time.

###### Lemma 9.

Let be a univariate polynomial pMC and a monotonic PCTL formula, that is, is in the PCTL fragment obtained by the following grammar:

 Φ ::= a∈S_+ ∣ Φ∧Φ ∣ Φ∨Φ ∣ P⩾c(φ) ∣ P>c(φ) φ ::= ◯Φ ∣ ΦUΦ ∣ ΦRΦ ∣  ◊Φ ∣  □Φ

where . Then, for any two valuations and of with .

Hence, if is monotonic then the satisfaction function , is monotonic. For each monotonic PCTL formula there exist and such that for all and for all . To decide (MC-E) for a given monotonic formula , it suffices to determine the sets for the sub-state formulas of . This can be done in polynomial time. Using this observation, we obtain:

###### Theorem 10 ((MC-E) for monotonic PCTL on univariate pMC).

Let be a univariate polynomial pMC on , and a monotonic PCTL formula. Then the model checking problem to decide whether there exists an admissible parameter valuation for such that is in P.

Model checking PCTL+EC on MCs with parametric weights. We now consider the case where is an ordinary Markov chain augmented with a parametric weight function