1 Introduction
Finitestate Markovian models are widely used as an operational model for the quantitative analysis of systems with probabilistic behaviour. In many cases, only estimates of the transition probabilities are available. This, for instance, applies to faulttolerant systems where the transition probabilities are derived from error models obtained using statistical methods. Other examples are systems operating with resourcemanagement protocols that depend on stochastic assumptions on the future workload, or cyberphysical systems where the interaction with its environment is represented stochastically. Furthermore, often the transition probabilities of Markovian models depend on configurable system parameters that can be adjusted at designtime. The task of the designer is to find a parameter setting that is optimal with respect to a given objective. This motivated the investigation of
interval Markov chains (IMCs) [19] specifying intervals for the transition probabilities (rather than concrete values). More general is the model of parametric Markov chains (pMCs), which has been introduced independently by Daws [9] and Lanotte et al. [23], where the transition probabilities are given by polynomials with rational coefficients over a fixed set of realvalued parameters . These concepts can be further generalized to accommodate rational functions, i. e., quotients of polynomials, as transition probabilities (see, e. g., [15]).It is wellknown that the probabilities for reachability conditions in parametric Markov chains with a finite state space can be characterized as the unique solution of a linear equation system where
is the solution vector, and
is a matrix where the coefficients are rational functions. Likewise, is a vector whose coefficients are rational functions. Note that it is no limitation to assume that the entries in and are polynomials, as rational function entries can be converted to a common denominator, which can then be removed. Now, can be viewed as a linear equation system over the field of rational functions with rational coefficients. As a consequence, the probabilities for reachability conditions are rational functions. This has been observed independently by Daws [9] and Lanotte et al. [23] for pMCs. Daws [9] describes a computation scheme that relies on a stateelimination algorithm inspired by the stateelimination algorithm for computing regular expressions for nondeterministic finite automata. This, however, is fairly the same as Gaussian elimination for matrices over the field of rational functions.As observed by Hahn et al. [15], the naïve implementation of Gaussian elimination for pMCs, that treats the polynomials in and as syntactic atoms, leads to a representation of the rational functions as the quotient of extremely (exponentially) large polynomials. In their implementation PARAM [14] (as well as in the reimplementation within the tool PRISM [22]), the authors of [15] use computeralgebra tools to simplify rational functions in each step of Gaussian elimination by identifying the greatest common divisor (gcd) of the numerator and the denominator polynomial. Together with polynomialtime algorithms for the gcdcomputation of univariate polynomials, this approach yields a polynomialtime algorithm for computing the rational functions for reachability probabilities in pMCs with a single parameter. Unfortunately, gcdcomputations are known to be expensive for the multivariate case (i. e., ) [13]. To mitigate the cost of the gcdcomputations, the tool Storm [11] successfully uses techniques proposed in [18] such as caching and the representation of the polynomials in partially factorized form during the elimination steps. However, it is possible to completely avoid gcdcomputations by using onestep fractionfree Gaussian elimination. Surprisingly, this has not yet been investigated in the context of pMCs, although it is a wellknown technique in mathematics. According to Bareiss [3], this variant of Gaussian elimination probably goes back to Camille Jordan (1838–1922), and has been rediscovered several times since. Like standard Gaussian elimination it relies on the triangulation of the matrix, and finally obtains the solution by back substitution. Applied to matrices over polynomial rings the approach generates matrices with polynomial coefficients (rather than rational functions) and ensures that the degree of the polynomials in all intermediate matrices grows at most linearly. This is achieved by dividing, in each elimination step, by a factor known by construction. Thus, when applied to a pMC with linear expressions for the transition probabilities, the degree of all polynomials in the solution vector is bounded by the number of states. For the univariate case (
), this yields an alternative polynomialtime algorithm for the computation of the rational functions for reachability probabilities. Analogous statements hold for expectations of random variables that are computable via linear equation systems. This applies to expected accumulated weights until reaching a goal, and to the expected mean payoff.
Contribution. The purpose of the paper is to study the complexity of the model checking problem for pMCs and probabilistic computation tree logic (PCTL) [16], and its extensions by expectation operators for pMCs augmented by weights for its states. In the first part of the paper (Section 3), we discuss the use of Bareiss’ onestep fractionfree Gaussian elimination for the computation of reachability probabilities. The second part of the paper (Section 4) presents complexitytheoretic results for the PCTL model checking problem in pMCs. We describe an exponentialtime algorithm for computing a symbolic representation of all parameter valuations under which a given PCTL formula holds, and provide a PSPACE upper bound for the decision variants that ask whether a given PCTL formula holds for some or all admissible parameter valuations. The known NP/coNPhardness results for IMCs [27, 7] carry over to the parametric case. We strengthen this result by showing that the existential PCTL model checking problem remains NPhard even for acyclic pMCs and PCTL formulas with a single probability operator. For the univariate case, we prove NPcompleteness for the existential PCTL model checking problem, and identify two fragments of PCTL where the model checking is solvable in polynomial time. The first fragment are Boolean combinations of threshold constraints for reachability probabilities, expected accumulated weights until reaching a goal, and expected mean payoffs. The second fragment consists of PCTL formulas in positive normal form with lower probability thresholds interpreted over pMCs satisfying some monotonicity properties. Furthermore, we observe that the model checking problem for PCTL with expectation operators for reasoning about expected costs until reaching a goal is in P for Markov chains where the weights of the states are given as polynomials over a single parameter, when restricting to Boolean combinations of the expectation operators.
Proofs and further details on the experiments omitted in the main part due to space constraints can be found in the extended version [17].
Related work. Fractionfree Gaussian elimination is wellknown in mathematics, and has been further investigated in various directions for matrices over unique factorization domains (such as polynomial rings), see e. g. [24, 20, 28, 25]. To the best of our knowledge, fractionfree Gaussian elimination has not yet been studied in the context of parametric Markovian models.
Besides the above mentioned work [9, 14, 15, 18, 10] on the computation of the rational functions for reachability probabilities in pMCs, [23] identifies instances where the parameter synthesis problem for pMCs with 1 or 2 parameters and probabilistic reachability constraints is solvable in polynomial time. These rely on the fact that there are closedform representations of the (complex) zero’s for univariate polynomials up to degree 4 and rather strong syntactic characterizations of pMCs. In Section 3 we will provide an example to illustrate that the number of monomials in the numerators of the rational functions for reachability probabilities can grow exponentially in the number of states. We hereby reveal a flaw in [23] where the polynomialtime computability of the rational functions for reachability probabilities has been stated even for the multivariate case. [12] considers an approach for solving the parametric linear equation system obtained from sparse pMCs via Laplace expansion.
Model checking problems for IMCs and temporal logics have been studied by several authors. Most in the spirit of our work on the complexity of the PCTL model checking problem for pMCs is the paper [27] which studies the complexity of PCTL model checking in IMCs. Further complexitytheoretic results of the model checking problem for IMCs and temporal logics have been established in [7] for omegaPCTL (extending PCTL by Boolean combinations of Büchi and coBüchi conditions), and in [6] for linear temporal logic (LTL). Our results of the second part can be seen as an extension of the work [27, 7] for the case of pMCs. The NP lower bound for the multivariate case and a single threshold constraint for reachability probabilities strengthen the NPhardness results of [27].
There exist several approaches to obtain regions of parameter valuations of a pMC in which PCTL formulas are satisfied or not, resulting in an approximative covering of the parameter space. PARAM [15, 14]
employs a heuristic, sampling based approach, while PROPhESY
[10] relies on SMT solving via the existential theory of the reals to determine whether a given formula holds for all valuations in a sub region. For the same problem, [26]uses a parameter lifting technique that avoids having to solve the parametric equation system by obtaining lower and upper bounds for the values in a given region by a reduction to nonparametric Markov decision processes.
2 Preliminaries
The definitions in this section require a general understanding of Markov models, standard model checking, and temporal logics. More details can be found, e. g., in
[21, 2].Discretetime Markov chain. A (discretetime) Markov chain (MC) is a tuple where is a nonempty, finite set of states with the initial state , is a transition relation, and is the transition probability function satisfying if and only if , and for all with nonempty. We refer to as the graph of . A state in which is called a trap (state) of .
An infinite path in is an infinite sequence of states such that for . Analogously, a finite path in is a finite sequence of states in such that for . A path is called maximal if it is infinite or ends in a trap. denotes the set of all maximal paths in starting in . Relying on standard techniques, every MC induces a unique probability measure on the set of all paths.
Parameters, polynomials, and rational functions. Let be parameters that can assume any real value, . We write for the polynomial ring over the rationals with variables . Each can be written as a sum of monomials, i. e., where is a finite subset of and . If is empty, or for all tuples , then is the null function, generally denoted by 0. The degree of is where . A linear function is a function with . A rational function is a function of the form with , . The field of all rational functions is denoted by . We write for the set of all polynomial constraints of the form where , and .
Parametric Markov chain. A (plain) parametric Markov chain on , pMC for short, is a tuple where , , and are defined as for MCs, and is the transition probability function with , i. e., the null function, iff . Intuitively, a pMC defines the family of Markov chains arising by plugging in concrete values for the parameters. A parameter valuation is said to be admissible for if for each state we have if nonempty, and iff , where for all . Let , or briefly , denote the set of admissible parameter valuations for . Given the Markov chain associated with is . The semantics of the pMC is then defined as the family of Markov chains induced by admissible parameter valuations, i. e., .
An augmented pMC is a tuple where , , , and are defined as for plain pMCs, and is a finite set of polynomial constraints. A parameter valuation is admissible for an augmented pMC if it is admissible for the induced plain pMC , and satisfies all polynomial constraints in . As for plain pMC, we denote the set of admissible parameter valuations of an augmented pMC by , or briefly .
A, possibly augmented, pMC is called linear, or polynomial, if all transition probability functions and constraints are linear functions in , or polynomials in , respectively.
Interval Markov chain. An interval Markov chain (IMC) [27] can be seen as a special case of a linear augmented pMC with one parameter for each edge , and linear constraints for each edge with and . According to the terminology introduced in [27], this corresponds to the semantics of IMC as an “uncertain Markov chain”. The alternative semantics of IMC as a Markov decision process will not be considered in this paper.
Labellings and weights. Each of these types of Markov chain, whether MC, plain or augmented pMC, or IMC, can be equipped with a labelling function , where is a finite set of atomic propositions. If not explicitly stated, we assume the implicit labelling of the Markov chain defined by using the state names as atomic propositions and assigning each name to the respective state. Furthermore, we can extend any Markov chain with a weight function . The value assigned to a specific state is called the weight of . It is sometimes also referred to as the reward of . In addition to assigning rational values we also consider parametric weight functions .
Probabilistic computation tree logic. We augment the standard notion of probabilistic computation tree logic with operators for the expected accumulated weight and mean payoff, and for comparison. Let be a finite set of atomic propositions. stands for , or , , . Then
where . The basic temporal modalities are (next) and (until). The usual derived temporal modalities (eventually), (release) and (always) are defined by , and , where, e. g., is and is , and .
For an MC with states labelled by we use the standard semantics. We only state the semantics of the probability, expectation, and comparison operators here. For each state , iff , and iff . Here is short for . Furthermore, iff , and iff , where denotes the expected value of the respective random variable. For detailed semantics of the expectation operators, see [17]. We write iff .
Notation: PCTL+EC and sublogics. We use PCTL to refer to unaugmented probabilistic computation tree logic. If we add only the expectation operator we write PCTL+E, and, analogously, PCTL+C if we only add the comparison operator for probabilities. PCTL+EC denotes the full logic defined above.
DAGrepresentation and length of formulas. We consider for any PCTL+EC state formula the directed acyclic graph (DAG) representing its syntactic structure. Each node of the DAG represents one of the substate formulas. The use of a DAG rather than the syntax tree allows the representation of subformulas that occur several times in the formula by a single node. The leaves of the DAG can be the Boolean constant true, and atomic propositions. The inner nodes of the DAG, e. g., of a PCTL formula, are labelled with one of the operators , , , . Nodes labelled with and have a single outgoing edge, while nodes labelled with or have two outgoing edges. For the abovementioned extensions of PCTL the set of possible inner node labels is extended accordingly. For example, a node representing the PCTL+C formula has three outgoing edges. If then there are two parallel edges from to a node representing . The length of a PCTL+EC formula is defined as the number of nodes in its DAG.
3 Fractionfree Gaussian elimination
Given a pMC as in Section 2, the probabilities for reachability conditions are rational functions and computable via Gaussian elimination. As stated in the introduction, this has been originally observed in [9, 23] and realized, e. g., in the tools PARAM [14] and Storm [10, 11] together with techniques based on gcdcomputations on multivariate polynomials. In this section, we discuss the potential of fractionfree Gaussian elimination as an alternative, which is wellknown in mathematics [3, 13], but to the best of our knowledge, has not yet been considered in the context of pMCs.
While the given definitions allow for rational functions in the transition probability functions of (augmented) pMCs, we will focus on polynomial (augmented) pMCs throughout the remainder of the paper. Generally, a linear equation systems containing rational functions as coefficients can be rearranged to one containing only polynomials by multiplying each line with the common denominator of the respective rational functions. Due to the multiplications this involves the risk of a blowup in the coefficient size. To avoid this we add variables in the following way. Let be an (augmented) pMC. For all introduce a fresh variable . By definition for some . Let if , if , . Then is a polynomial augmented pMC.
Linear equation systems with polynomial coefficients. Let be parameters, . We consider linear equation systems of the form , where is a nonsingular matrix with . Likewise, is a vector of length with . The solution vector is a vector of rational functions with . By Cramer’s rule we obtain , where is the determinant of , and the determinant of the matrix obtained when substituting the th column of by . If the coefficients of and have at most degree , the Leibniz formula implies that and have at most degree .
Lemma 1.
There is a family of acyclic linear pMCs where has parameters and states, including distinguished states and , such that is a polynomial for which even the shortest sumofmonomial representation has monomials.
Onestep fractionfree Gaussian elimination is a variant of fractionfree Gaussian elimination that allows for divisions which are known to be exact at the respective point in the algorithm. When using naïve fractionfree Gaussian elimination the new coefficients after the th step, , are computed as for , where . When applied to systems with polynomial coefficients this results in the degree doubling after each step, so the degree grows exponentially. In a step of onestep fractionfree Gaussian elimination (see Algorithm 1), the computation of the coefficients changes to with . Using Sylvester’s identity one can prove that is again a polynomial, and that is in general the maximal possible divisor. The are updated analogously. If the maximal degree of the initial coefficients of and is , this technique therefore guarantees, that after steps the degree of the coefficients is at most , i. e., it grows linear in during the procedure. For polynomials the division by can be done using standard polynomial division. The timecomplexity of the exact multivariate polynomial division in this case is in each step , so for the full onestep fractionfree Gaussian elimination it is .
Proposition 4.3 in [23] states that the rational functions for reachability probabilities in pMC with a representation of the polynomials , as sums of monomials (called normal form in [23]) are computable in polynomial time. This contradicts Lemma 1 which shows that the number of monomials in the representation of a reachability probability as a sum of monomials can be exponential in the number of parameters. However, the statement is correct for the univariate case.
Lemma 2.
Let be a polynomial pMC over a single parameter and a set of states. Then, the rational functions for the reachability probabilities are computable in polynomial time. Analogously, rational functions for the expected accumulated weight until reaching or the expected mean payoff are computable in polynomial time.
Note that the degrees of the polynomials and computed by onestep fractionfree Gaussian elimination for reachability probabilities are bounded by , where , so the polynomials have representations as sums of at most monomials. In particular, the degree and representation size of the final polynomials and for the rational functions is in where is the number of states of . Another observation concerns the case where only the righthand side of the linear equation system is parametric. Systems of this form occur, e. g., when considering expectation properties for MCs with parametric weights.
Lemma 3.
Let be a parametric linear equation system as defined above where is parameterfree. Then the solution vector consist of polynomials of the form with and can be computed in polynomial time.
Stratification via SCCdecomposition. It is well known (e. g., [8, 18]
) that for probabilistic/parametric model checking a decomposition into stronglyconnected components (SCCs) can yield significant performance benefits due to the structure of the underlying models. We have adapted the onestep fractionfree Gaussian elimination approach by a preprocessing step that permutes the matrix according to the topological ordering of the SCCs. This results in the coefficient matrix already having a stairlike form at the start of the algorithm. In the triangulation part of the algorithm, each SCC can now be considered separately, as nonzero entries below the main diagonal only occur within each SCC. While the backsubstitution in the general onestep fractionfree elimination will result in each entry on the main diagonal being equal to the last, this property is now only maintained within the SCCs. Formally, this means that the back substitution step in Algorithm
1 is replaced by the following:where , and, for , if the th and st state belong to the same SCC and otherwise. Intuitively, is the product of the ’s on the diagonal corresponding to the last states in the current SCC and the SCCs below. Of course, the return statement also has to be adjusted accordingly. The advantage of this approach is that the polynomials in the rational functions aside from the ones in the first strongly connected component will have an even lower degree.
Implementation and experiments. For a first experimental evaluation of the onestep fractionfree Gaussian elimination approach (GEff) in the context of probabilistic model checking, we have implemented this method (including the SCC decomposition and topological ordering described above) as an alternative solver for parametric linear equation systems in the stateoftheart probabilistic model checker Storm [11]. We compare GEff against the two solvers provided by Storm (v1.0.1) for solving parametric equation systems, i. e., the solver based on the eigen linear algebra library^{2}^{2}2http://eigen.tuxfamily.org/ and on state elimination (stateelim) [15]. Both of Storm’s solvers use partially factorized representations of the rational functions provided by the CArL library^{3}^{3}3https://github.com/smtrat/carl. This approach, together with caching, was shown [18] to be beneficial due to improved performance of the gcdcomputations during the simplification steps.
It should be noted that our implementation is intended to provide first results that allow to gauge whether the fractionfree method, by avoiding gcdcomputations, can be beneficial in practice and is thus rather naïve in certain aspects. As an example, it currently relies on a dense matrix representation, with performance improvements for larger models to be expected from switching to sparse representations as used in Storm’s eigen and stateelim solvers. In addition to the fractionfree approach, our solver can also be instantiated to perform a straightforward Gaussian elimination, using any of the representations for rational functions provided by the CArL library. In all our experiments, we have compared the solutions obtained by the different solvers and verified that they are the same.
Experimental studies. The source code of our extension of Storm and the artifacts of the experiments are available online.^{4}^{4}4http://wwwtcs.inf.tudresden.de/ALGI/PUB/GandALF17/ As our GEff implementation is embedded as an alternative solver in Storm, we mainly report the time actually spent for solving the parametric equation system, as the other parts of model checking (model building, precomputations) are independent of the chosen solver. For benchmarking, we used a machine with two Intel Xeon E52680 8core CPUs at 2.70GHz and with 384GB RAM, a time out of 30 minutes and a memory limit of 30GB. All the considered solvers run singlethreaded. We have considered three different classes of case studies for experiments.
Complete pMC. As a first experiment to gauge the efficiency in the presence of a high ratio of parameters to states, we considered a family of pMCs with a complete graph structure (over states) and one parameter per transition, resulting in parameters (for details see [17]).
rows  param.  eigen  stateelim  GEff  red(GEff)  

4  4  20  0.47  0.64  0.06  0.01 
5  5  30  44.47  42.09  2.13  1.52 
6  6  42  timeout  timeout  221.27  21.53 
Table 1 depicts statistics for the corresponding computations, for the two standard solvers in Storm (eigen and stateelim), as well as our fractionfree implementation (GEff). For stateelim, we always use the default elimination order (forward). The time for GEff corresponds to the time until a solution rational function (for all states) is obtained. As the numerator and denominator of these rational functions are not necessarily coprime, for comparison purposes we list as well the time needed for simplification (red(GEff)) via division by the gcd. As can be seen, here, the fractionfree approach significantly outperforms Storm’s standard solvers and scales to a higher number of parameters. We confirmed using profiling that the standard solvers indeed spend most of the time in gcdcomputations.
Multiparameter IsraeliJalfon selfstabilizing. The benchmarks used to evaluate parametric model checking implementations in previous papers tend to be scalable in the number of components but use a fixed number of parameters, usually 2. To allow further experiments with an increasing number of parameters, we considered a pMCvariant of the IsraeliJalfon selfstabilizing protocol with processes, initial tokens and parameters (for details see [17]).
rows  param.  eigen  stateelim  GEfac  GEff  red(GEff)  

4  3  21  4  1.01  0.86  0.73  0.16  0.20 
4  4  15  4  0.94  0.58  0.58  0.16  0.13 
5  2  16  5  19.13  30.83  29.46  9.36  0.33 
5  3  36  5  360.43  747.32  172.16  485.78  95.92 
5  4  51  5  457.55  1542.97  442.80  614.01  742.69 
5  5  31  5  368.70  1597.29  252.92  622.00  414.72 
Table 2 depicts the time spent for computing the rational functions for several instances. As can be seen, the fractionfree approach is competitive for the smaller instances, with performance between the eigen and stateelim solvers for the larger instances. We have also included running times for GEfac, i. e., for our naïve implementation of Gaussian elimination using the representation for rational functions as used by Storm for the standard solvers, including automatic gcdbased simplification after each step to ensure that numerator and denominator are coprime. GEfac operates on the same, topologically sorted matrix as the fractionfree GEff. Curiously, GEfac is able to outperform the eigen solver for some of the larger instances. We believe this is mainly due to differences in the matrix permutation and their effect on the elimination order, which is known to have a large impact on performance (e. g., [10]).
Benchmark case studies from [10]. Furthermore, we considered several case study instances that were used in [10] to benchmark parametric model checkers, namely the brp, crowds, egl, nand, zeroconf models. Table 3 depicts statistics for selected instances, for further details see [17]. The application of bisimulation quotienting often has a large impact on the size of the linear equation system, so we performed experiments with and without quotienting. For crowds, bisimulation quotienting was particularly effective, with all considered instances having a very small state space and negligible solving times. For the nonquotiented instances, Storm’s standard solvers outperform GEff. For the zeroconf instance in Table 3, GEff is competitive. Note that the models in the brp, egl and nand case studies are acyclic and that the parametric transition probabilities and rewards are polynomial. As a consequence, the gcdcomputations used in Storm’s solvers don’t impose a significant overhead as the rational functions during the computation all have denominator polynomials of degree zero.
model  rows  param.  eigen  stateelim  GEff  red(GEff) 

Crowds (3,5), weakbisim  40  2  0.08  0.06  0.02  0.13 
Crowds (5,5), weakbisim  40  2  0.08  0.06  0.02  0.11 
Crowds (10,5), weakbisim  40  2  0.08  0.06  0.02  0.11 
Crowds (3,5)  715  2  0.99  0.80  11.44  63.39 
Crowds (5,5)  2928  2  6.36  5.51  1271.95  timeout 
Crowds (10,5)  25103  2  139.82  173.15  timeout  — 
Zeroconf (1000)  1002  2  81.03  45.01  49.43  11.35 
Overall, the experiments have shown that there are instances where the fractionfree approach can indeed have a positive impact on performance. Keeping in mind that our implementation has not yet been significantly optimized, we believe that the fractionfree approach is an interesting addition to the gcdbased solver approaches. In particular, the application of better heuristics for the order of processing (i. e., the permutation of the matrix) could still lead to significant performance increases.
4 Complexity of the PCTL+EC model checking problem
We now study the complexity of the following variants of the PCTL+EC model checking problem. Given an augmented pMC and a PCTL+EC (state) formula :
(All)  Compute a representation of the set of all satisfying parameter valuations, 


i. e., the set of all admissible parameter valuations such that . 
(MCE)  Does there exist a valuation such that ? 
(MCU)  Does hold for all admissible valuations ? 
(MCE) and (MCU) are essentially duals of each other. Note that the answer for the universal variant (MCU) is obtained by the negation of the answer for (MCE) with formula , and vice versa. In what follows, we shall concentrate on (All) and the existential model checking problem (MCE).
Computing all satisfying parameter valuations. As before, denotes the set of admissible valuations. In what follows, let be the conjunction of the polynomial constraints in as well as the constraints for each nontrap state , and for each edge . We then have if and only if is admissible, i. e., .
Let be a PCTL+EC formula. The satisfaction function is defined by:
We now present an algorithm to compute a symbolic representation of the satisfaction function that groups valuations with the same corresponding satisfaction set together. More precisely, we deal with a representation of the satisfaction function by a finite set of pairs where is a Boolean combination of constraints and such that (i) and implies , and (ii) whenever then there is a pair such that .
Given the DAG representation of the PCTL formula , we follow the standard model checking procedure for CTLlike branchingtime logics and compute for the subformulas assigned to the nodes in the DAG for in a bottomup manner. As the leaves of the DAG can be atomic propositions or the formula true, the base cases are , and . Consider now the inner node of the DAG for labelled by the outermost operator of the subformula . Suppose that the children of have already been treated, so when computing the satisfaction sets of the proper subformulas of are known. If is labelled by or , i. e., or , then respectively . If , then
where is the conjunction of the constraints for each state , and for each state . Here, is the rational function that has been computed using (i) a graph analysis to determine the set of states with and (ii) fractionfree Gaussian elimination (Section 3) to compute the rational functions in the pMC resulting from by turning the states in into traps. If and are polynomials computed by fractionfree Gaussian elimination such that then is a shortform notation for . The treatment of and the expectation operators is similar, and can be found in [17]. After treating a node of the DAG, we can simplify the set by first removing all pairs where is not satisfiable (using algorithms for the existential theory of the reals), and afterwards combining all pairs with the same component, that is, instead of pairs , we consider a single pair . To answer question (All), the algorithm finally returns the disjunction of all formulas with for .
Complexity bounds of (All) and (MCE). The existential theory of the reals is known to be in PSPACE and NPhard, and there is an upper bound on the timecomplexity, namely where is the number of constraints, the maximum degree of the polynomials in the constraints, and the number of parameters [4]. Recall from Section 3 that a known upper bound on the timecomplexity of onestep fractionfree Gaussian elimination is , where is the number of equations, the maximum degree of the initial coefficient polynomials, and the number of parameters. Combining both approaches, the onestep fractionfree Gaussian elimination for solving linear equation systems with polynomial coefficients, and the existential theory of the reals for treating satisfiability of conjunctions of polynomial constraints, one directly obtains the following bound for the computational complexity of PCTL+EC model checking on augmented polynomial pMCs. Note that this assumes that the number of constraints in is at most polynomial in the size of .
Theorem 4 (Exponentialtime upper bound for problem (All)).
Let be a PCTL+EC formula. Given an augmented polynomial pMC , where the maximum degree of transition probabilities , and polynomials in the constraints in is , a symbolic representation of the satisfaction function is computable in time , where is the number of probability, expectation and comparison operators in .
Theorem 5 (PSPACE upper bound for problem (MCE)).
The existential PCTL+EC model checking problem (MCE) for augmented pMC is in PSPACE.

The main idea of a polynomially spacebounded algorithm is to guess nondeterministically sets of states for the subformulas where the outermost operator is a probability, expectation or comparison operator, and then apply a polynomially spacebounded algorithm for the existential theory of the reals [4] to check whether there is a parameter valuation such that for all . ∎
NP and coNPhardness of (MCE) follow from results for IMCs [27, 7]. More precisely, [7] provides a polynomial reduction from SAT to the (existential and universal) PCTL model checking problem for IMCs. In fact, the reduction of [7] does not require full PCTL, instead Boolean combinations of simple probabilistic constraints without nesting of the probability operators are sufficient. The following theorem strengthens this result by stating NPhardness of (MCE) even for formulas consisting of a single probability constraint for a reachability condition.
Theorem 6 (NPhardness for single probabilistic operator, multivariate case).
Given an augmented polynomial pMC on parameters with initial state and an atomic proposition , and a probability threshold , the problem to decide whether there exists such that is NPhard, even for acyclic pMCs with the assigned transition probabilities being either constant, or linear in one parameter, i. e., , , for all , and where the polynomial constraints for the parameters are of the form with , .
Univariate pMCs. In many scenarios, the number of variables has a fixed bound instead of increasing with the model size. We consider here the case of univariate pMC, i. e., pMC with a single parameter.
Theorem 7 (PCTL+EC model checking without nesting in P, univariate case).
Let be a PCTL+EC formula without nested probability, expectation or comparison operators, and let be a polynomial pMC on the single parameter . The problem to decide whether there exists an admissible parameter valuation such that is in P.

If we restrict PCTL+EC to Boolean combinations of probability, expectation, and comparison operators, (MCE) can be dealt with by first computing polynomial constraints for for each probability, expectation, and comparison operator independently (this can be done in polynomial time by Lemma 2), and afterwards applying a polynomialtime algorithm for the univariate existential theory of the reals [5] once to the appropriate Boolean combination of the constraints. ∎
Theorem 8 (NPcompleteness for full PCTL+EC, univariate case).
Let be a PCTL+EC formula, and let be a polynomial pMC on the single parameter . The PCTL+EC model checking problem to decide whether there exists an admissible parameter valuation such that is NPcomplete. NPhardness even holds for acyclic polynomial pMCs and the fragment of PCTL+C that uses the comparison operator , but not the probability operator , as well as for (cyclic) polynomial pMC in combination with PCTL.
(MCE) for monotonic PCTL on univariate pMCs. The parameters in pMC typically have a fixed meaning, e. g., probability for the occurrence of an error, in which case the probability to reach a state where an error has occurred is increasing in . This motivates the consideration of univariate pMCs and PCTL formulas that are monotonic in the following sense.
Given a univariate polynomial pMC , let denote the set of edges such that the polynomial is monotonically increasing in , i. e., whenever and then . Let denote the set of states such that for each finite path with we have for .
As iff there is no value such that , the set is computable in polynomial time using a polynomialtime algorithm for the univariate theory of the reals [5]. Here, is as before the Boolean combination of polynomial constraints characterizing the set of admissible parameter values, and is the first derivative of the polynomial . Thus, the set is computable in polynomial time.
Lemma 9.
Let be a univariate polynomial pMC and a monotonic PCTL formula, that is, is in the PCTL fragment obtained by the following grammar:
where . Then, for any two valuations and of with .
Hence, if is monotonic then the satisfaction function , is monotonic. For each monotonic PCTL formula there exist and such that for all and for all . To decide (MCE) for a given monotonic formula , it suffices to determine the sets for the substate formulas of . This can be done in polynomial time. Using this observation, we obtain:
Theorem 10 ((MCE) for monotonic PCTL on univariate pMC).
Let be a univariate polynomial pMC on , and a monotonic PCTL formula. Then the model checking problem to decide whether there exists an admissible parameter valuation for such that is in P.
Model checking PCTL+EC on MCs with parametric weights. We now consider the case where is an ordinary Markov chain augmented with a parametric weight function
Comments
There are no comments yet.