# Verification of Continuous-time Markov Chains

In this paper, we fill the long-standing gap in the field of the verification of continuous-time systems by proposing a linear-time temporal logic, named continuous linear logic (CLL) and the corresponding model checking algorithm for continuous-time Markov chains. The correctness of our model checking algorithm depends on Schanuel's conjecture, a central open question in transcendental number theory.

There are no comments yet.

## Authors

• 8 publications
• 13 publications
05/02/2021

### Model Checking Quantum Continuous-Time Markov Chains

Verifying quantum systems has attracted a lot of interests in the last d...
11/26/2018

### Integrating Topological Proofs with Model Checking to Instrument Iterative Design

System development is not a linear, one-shot process. It proceeds throug...
07/06/2021

### Scaling up Continuous-Time Markov Chains Helps Resolve Underspecification

Modeling the time evolution of discrete sets of items (e.g., genetic mut...
10/28/2019

### Bounding Mean First Passage Times in Population Continuous-Time Markov Chains

We consider the problem of bounding mean first passage times for a class...
04/19/2021

### Sampling Polynomial Trajectories for LTL Verification

This paper concerns the verification of continuous-time polynomial splin...
04/22/2022

### MITL Verification Under Timing Uncertainty

A Metric Interval Temporal Logic (MITL) verification algorithm is presen...
06/17/2019

### Statistical Verification of Hyperproperties for Cyber-Physical System

Many important properties of cyber-physical systems (CPS) are defined up...
##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Model checking is a method for checking whether a finite-state model of a system meets a given specification, which is usually expressed by a temporal logic formula. Since the 1970s, the area of checking program has been promoted significantly through the discovery and development of the algorithmic methods for verifying discrete-time temporal logic properties of finite-state systems [Pnu77, CES86, LP85, QS82, VW86]. Regarding the nature of time, two dominant logic emerged. Linear-time temporal logics (LTL) was first proposed by Amir Pnueli in 1977 [Pnu77]. After that, in 1981, Clarke and Emerson introduced computational tree logic (CTL), a branching-time logic [CE81]. Not surprisingly, LTL and CTL are expressively incomparable [EH86, CD88, Lam80]. CTL and LTL are used to specify qualitative properties for non-probabilistic model checking, for example, safety (nothing bad happens) and liveness (something good eventually happens). When considering probabilistic transition systems, in practice, discrete-time Markov chains with finite states are the most successful model for probabilistic dynamical systems. By allowing for probabilistic quantification on transition paths, Hansson and Jonsson introduced probabilistic computational tree logic (PCTL) in 1994 [HJ94]. The verification of the linear-time property of discrete-time Markov chains was achieved by Agrawal, Akshay, Genest, and Thiagarajan in [AAGT15]. They introduced probabilistic linear-time logic (PLTL) to reason the dynamics of discrete-time Markov chains by adding probabilistic quantifiers on the state space. The first three model-checking algorithms can be found in [BK08] and the last one is given in [AAGT15]. Recently, Yu provided quantum temporal logic (QTL), a quantum generalization of LTL, mainly targeting on the verification of quantum programs in [Yu19].

Model checking continuous-time (real-time) systems is even nastier. For specifying the temporal properties of (non)deterministic systems, LTL and CTL both have been generalized from discrete-time domain to continuous-time domain in the 1990s, named metric temporal logic (MTL)[Koy90] and timed computational temporal logic (TCTL) [AH94], respectively. The syntaxes of the two continuous-time logics both use time-constrained temporal operators to replace the corresponding versions in LTL and CTL. The semantics are dense, i.e., the path is defined on infinitely countable time instants rather than the non-negative real numbers. Model checking for TCTL is decidable, while model checking for MTL is undecidable. However, model checking for Metric Interval Temporal Logic (MITL), a fragment of MTL, is decidable. MITL was proposed by Alur et al. in [AFH96].

After that, temporal logics are also brought in the field of the verification of probabilistic continuous-time systems. Continuous-time Markov chains (CTMCs) form a fundamental class of probabilistic dynamical systems that have been widely studied theoretically and practically since Kolmogorov [Kol31]. Due to a broad range of applications, continuous-time model checking is invented for the verification of continuous-time Markov chains. Lots of efforts have been devoted in this field to constitute the underlying semantical model of a plethora of modeling formalisms for continuous-time probabilistic systems, in the last two decades, cf. the recent survey [Kat16]

. In a majority of the verification related studies, the continuous-time Markov chain is viewed as a probabilistic interval transition system. The time is measured in discrete intervals, so at each time interval the system is in a particular state and the state will transfer to another state with an appropriate probability after the time interval. The paths of this transition system are viewed as computations and the goal is to use the branching-time probabilistic temporal logic, named

continuous stochastic logic (CSL), to reason about these computations [BHHK03]. Thus the semantic of CSL is also dense. Reusing the name presented in [ASSB00], CSL generalizes PCTL to fit for specifying properties of continuous-time Markov chains. Thanks to this time discretization method, efficient techniques for analyzing discrete-time Markov chains have been developed for model checking continuous-time Markov chains. Furthermore, model-checking algorithms developed in [BHHK03] has been implemented in model checkers PRISM [KNP02] and MRMC [KZH11]. As the semantics of all three continuous-time logics MITL, TCTL and CSL are dense, all model-checking algorithms for them are in general approximate. Table 1 overviews the current status of temporal logics.

There is an obvious gap in the above table.

###### Problem 1

Developing a linear-time temporal logic and the corresponding model-checking algorithm for continuous-time probabilistic systems?

This is formulated as a challenging open problem mentioned in [CHKM11a]. We remark that linear-time temporal logics and branching-time temporal logics are naturally incomparable from many perspectives including the expressiveness. The techniques and results of branching-time temporal logic can not be used to solve the above problem.

### 1.1 Conceptual Contribution

In this paper, we introduce continuous linear logics (CLL) filling the mentioned gap in temporal logics.

We adopt an alternative approach to viewing the state space of the continuous-time Markov chain to be the set of probability distributions over the states of the chain, which was used in analyzing discrete-time Markov chains

[AAGT15]. Thus the continuous-time Markov chain transforms initial probability distribution continuously. CLL studies the properties of the trajectory at all times generated from a given initial distribution. Many interesting dynamical properties can be formulated in this setting regarding the behaviors of the chain. For instance, in some time interval, no time will it be the case that the probability of being in the state and the probability of being in the state are both low, or starting from some time the system is most likely to be in state or state . To our best knowledge, such temporal properties have not been discussed in previous literature on the verification of continuous-time Markov chains.

We apply the method of symbolic dynamics on the distributions of continuous-time Markov chains. To be specified in our setting, we symbolize the probability value space into a finite set of intervals . A probability distribution over its set of states is then represented symbolically as a set of symbols which asserting that the -th element of falls in interval . For example, a special case of symbolization of distributions has been considered in [AAGT15]: choosing a disjoint cover of : . is regarded as the set of atomic propositions. It is direct to define if a probability distribution satisfies . A crucial fact is that the set of symbols is finite. Consequently, the path over real numbers generated by an initial probability distribution will induce a sequence over the finite alphabet . Hence, given an initial distribution, the symbolic dynamics of continuous-time Markov chains can be studied in terms of a language over the alphabet . Our focus here is on continuous behaviors over real numbers.

Our main motivation for studying continuous-time Markov chains in this fashion is that in many practical applications such as biochemical networks and queuing systems, obtaining exact estimations of the probability distributions (including the initial distribution) may be neither feasible nor necessary. Actually, interesting properties are stated in terms of probability ranges, such as “low, medium, or high” or “above the threshold 0.9” rather than exact probabilities.

We formulate the CLL in which an atomic proposition will assert that “the current probability of the state lies in some interval”. Different from temporal logic in the discrete-time domain, CLL has two types of formulas: states and paths. The state type formulas are constructed under propositional connectives. The path type formulas are obtained under propositional connectives and the temporal model timed until in the usual way. The timed until is bounded. The usual next-step temporal operator is not meaningful in our logic. This is not surprised as in continuous-time, the steps of time can not be defined because the time domain (real numbers) is uncountable. On the other hand, keeping the time continuous instead of discretizing time, our logic’s expressive power is incomparable with logics such as CSL interpreted over the states transitions of the continuous-time Markov chain.

### 1.2 Technical contribution

Based on Schanuel’s conjecture, we develop an algorithm to model checking continuous-time Markov chains against CLL formulas.

We reduce the model checking problem to the real root isolation problem of real polynomial-exponential functions (PEFs) over the field of the algebraic number, a widely studied problem in recent symbolic and algebraic computation community [GCL17, AMW08, LHXL16]. We resolve the latter problem under the assumption of the validity of Schanuel’s conjecture, a central open question in transcendental number theory  [Lan66].

### 1.3 Comparison with Related Work

Symbolic dynamics is a classical topic in the theory of dynamical systems [MH38] with data storage, transmission and coding being the major application areas [LMDB95]. The basic idea is to partition the “smooth” state space into a finite set of blocks and represent a trajectory as a sequence of such blocks. The symbolic dynamics of discrete-time Markov chains was verified in [AAGT15]. This idea regards distributions as the state space instead of the states of discrete-time Markov chains and partitioning the probability value space into finite disjoint covers (intervals). Therefore, for a given initial distribution, there is only one single path, which PLTL can be introduced to reason the behavior of the dynamics of distributions. In our setting, we follow this view and symbolizing the probability value space into finite intervals, not needed to be a partition. This generalization endows CLL’s stronger power of expressiveness.

The verification of continuous-time Markov chains was studied in [ASSB00] using CSL, a branching-time logic, i.e., asserting the exact temporal properties with time continuous. The essential feature of CSL is that the path formula is the form of nesting of bounded timed until operators only reasoning the absolutely temporal properties (all time instants basing on one starting time). In our CLL logic, the nesting of bounded timed until operator can also specify relatively temporal properties. This setting significantly enriches the expressiveness of temporal properties.

CSL was extended by adding next-step operator in [BHHK03]. For introducing the operator, they make time discretization such that the time is measured in discrete intervals. So at each time interval, the system is in a particular state. Each state is associated with a set of atomic propositions and the evolution of the system is described by an infinite sequence of symbols, which is represented as strings. The distinct feature in our CLL setting is that the discretization is applied on the distributions over states and we leave the time continuous. The set of all distributions is inherently not discrete. We fix finite intervals of representing a distribution by satisfying these intervals, to get a coarse-grained description of the system. This method ensures the time is real continuous such that we can exactly model checking continuous-time Markov chains instead of approximate verification by CSL. For more comparison, please see Table 2.

Linear-time model checking for continuous-time Markov chains has also been studied through time discretization in [CHKM11a]. The specification is given by deterministic timed automaton (DTA) with finite and Muller acceptance criteria. The central question they addressed is: what is the probability of the set of paths of continuous-time Markov chains that are accepted by a DTA. The model checker was also developed in [BCH11]

. Furthermore, the verification of the continuous-time Markov decision process (CTMDP) against DTA has been studied in

[CHKM11b]. A DTA can only express a property. The logic system CLL provides the opportunity of covering many more properties we are interested in.

### 1.4 Organization of this paper

In the next section, we give the mathematical preliminaries used in this paper. In Section 3, we introduce the symbolic dynamics of continuous-time Markov chains by symbolizing distributions of them. In the subsequent section, we present our continuous-time temporal logic CLL and illustrate the model checking problem. In Section 5, with the help of the real root isolation of polynomial exponential functions, we develop an algorithm to solve the model checking problem. In Section 6, we illustrate the real root isolation of polynomial exponential functions over algebraic numbers and prove the correctness basing on Schanuel’s conjecture. In the concluding section, we summarize our results and point to future research directions.

## 2 Preliminaries

We use the set of the nonnegative real numbers (including ) to denote the time domain. A bounded (time) interval is a subset of . Intervals may be open, half-open, or closed. Each interval has one of the following forms:

 [t1,t2],[t1,t2),(t1,t2],(t1,t2),

where . For an interval of the above forms, is the left endpoint of , and is the right endpoint of . The left and right endpoints of are denoted by and , respectively.

The expression , for , denotes the interval . Similarly, stands for the intervals . Two intervals and are disjoint if their intersection is an empty set, i.e., .

Throughout this paper, we write and for the fields of complex, real, rational and algebraic numbers, respectively.

###### Definition 1

An algebraic number is any complex number that is a root of a non-zero polynomial in one variable with rational coefficients (or equivalently with integer coefficients, by eliminating denominators). An algebraic number is represented by where is the minimal polynomial of , and is an approximation of such that and is the only root of in the open ball . The minimal polynomial of is the polynomial with the lowest degree in such that is a root of the polynomial.

Furthermore, for any field , we use to denote the set of positive elements (including 0) of and to denote the set of polynomials in with coefficients in ; let be -by- matrix with every entry on the filed of . For any complex number where and is the imaginary unit, we denote the real part and the imaginary part of by and , respectively. It is well known that a root of is also algebraic. Moreover, given the representations of , the representations of and can be computed in polynomial time, so is the equality checking [Coh13].

###### Definition 2

is called a root of a function if . Furthermore, the multiplicity of is the maximum number such that is a factor of , i.e., there exists a function such that . Furthermore, if , then we call that is a single root; otherwise is a multiple root.

### 2.1 The Real Root Isolation of Polynomial Exponential Functions

###### Definition 3

A function is a polynomial-exponential function (PEF) if has the following form:

 f(t)=K∑k=0fk(t)eλkt (1)

where for all , , and are fields. Furthermore, denotes the set of power factors of , i.e., .

We define the degree of be the maximum degree of s.

We usually write this PEF as .

Generally, if the range of is in complex numbers , then let is a PEF with the range in real numbers , where is the complex conjugate of . The factor is omitted whenever convenient, i.e., .

PEFs often appear in transcendental number theory as auxiliary functions in proofs involving the exponential function [Bak90].

###### Definition 4

A (real) root isolation of function in interval is a set of mutually disjoint intervals, denoted by for such that

• for any , there is at least and only one root of in ;

• for any root of , for some .

Furthermore, if has no any root in , then .

Although there are infinite kinds of real root isolations of in , the number of isolation intervals equals to the number of distinct roots of in .

Finding a real root isolation of PEFs is a long-standing problem and can at least backtrack to Ritt’s paper [Rit29] in 1929. After that, some following results were obtained since the last century (e.g. [AH80, Tij71]). This problem plays an essential role in the reachability analysis of dynamical systems, one active field of symbolic and algebraic computation. In the case of and in [AMW08], an algorithm named ISOL was proposed to isolate all real roots of . Later, this algorithm has been extended to the case of and  [GCL17]. A variant of the problem has also been studied in [LHXL16]. The correctness of all these algorithms is based on very famous Schanuel’s conjecture from transcendental number theory.

We will also pursue this problem in the context of continuous-time Markov chains. The distinct feature of solving the problem in our paper is to deal with complex numbers , more specifically algebraic numbers , i.e., , while all the previous works can only handle the case over . Up to our best knowledge, finding a real root isolation of PEFs over has not been solved. From now on, we always assume that PEFs are over , i.e., .

We remark that the algorithms for finding real root isolations of PEFs over can not be directly generalized to the case over . The main reason is that there are only finitely many real roots of PEF over in any interval  [GCL17], while there are infinitely many real roots over or , for example

 f(t)=eiθt+e−iθt=2cos(θt) ∀θ∈R and θ≠0

the real roots of is , i.e., infinitely many.

### 2.2 Existence Checking of Real Roots

Usually, finding a (real) root isolation of a function bases on the decision problem: checking whether or not there is a root of in some given interval . This can be done by the following proposition in [COW15].

###### Proposition 1 ([Cow15])

Let be a function defined on a closed interval of reals with endpoints . There is a procedure to decide the existence of the roots of if satisfies the following conditions:

• there exists such that is -Lipschitz, i.e., for all ;

• given and positive error bound , we can compute such that ;

• ;

• for any such that , exists and is non-zero, i.e., has no tangential zeros.

For our use in the latter discuss, we formate the procedure presented in [COW15] as Algorithm 1.

### 2.3 Jordan Decomposition

To reduce our continuous-time model checking problem to the real root isolation of PEFs, we need to use the Jordan decomposition.

###### Definition 5

A Jordan block is a square matrix of the following form

 ⎡⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢⎣λ10⋯00λ1⋯0⋱⋱⋱1\par000⋯λ⎤⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥⎦.

A square matrix is in Jordan norm form if

 J=⎡⎢ ⎢ ⎢ ⎢ ⎢⎣J1J2⋱Jn⎤⎥ ⎥ ⎥ ⎥ ⎥⎦,

where is a Jordan blocks for each .

Because is algebraic closed, we know that

###### Proposition 2

Any matrix is algebraically similar to a matrix in Jordan normal form over algebraic number field . Namely, there exists some invertible and in Jordan form such that , where is the set of all -by- matrices with every entry being algebraic number.

### 2.4 Transcendental Number Theory

In the latter discussion, we will see that transcendental number theory can be applied to compare time instants when the values are transcendental numbers. A transcendental number is a complex number that is not an algebraic number, such as .

In general, it is extremely difficult to verify relationships between two transcendental numbers [Ric97]. But for some transcendental numbers represented by PEFs can be compared with the help of Lindemann-Weierstrass theorem.

###### Theorem 1 (Lindemann-Weierstrass theorem)

Let be pairwise distinct algebraic complex numbers. Then there exists no equation in which are algebraic numbers and are not all zero.

The following fact was observed in [ASSB00] by Lindemann-Weierstrass theorem.

###### Observation 1

Given a real number of the form where and are algebraic complex numbers, and the are pairwise distinct, there is an effective procedure to compare the values of and for any .

First by Lindemann-Weierstrass theorem, we know that if and only if or for some with . Then the main idea of the above observation is for each , can be approximated with an error of less than (when ) by taking the first terms of the Maclaurin expansion for . This can be extended to obtain an upper bound on the number of terms needed to approximate to within . Since the individual terms in the Maclaurin expansion are algebraic functions of the ’s, it follows that the approximations are algebraic. Then we can check if by the comparision between the approximations and .

For finding a real root isolation of PEFs, we also need the famous open problem Schanuel’s conjecture to factoring a PEF into finite irreducible PEFs. Before introducing the conjecture, we need more concepts.

###### Definition 6 (Algebraic independence)

A set of complex numbers is algebraically independent over if the elements of do not satisfy any nontrivial (non-constant) polynomial equation with coefficients in .

By the above definition, for any transcendental number , is algebraically independent over , for any algebraic number is not. Thus, a number in an algebraically independent set over must be a transcendental number. Excluding singlet sets, is also algebraically independent over for any positive integer  [Nes96]. Checking the algebraic independence is a challenging problem, and there are many open problems. It is still not known whether is algebraically independent over .

###### Definition 7 (Transcendence degree)

Let be a field extension of , the transcendence degree of over is defined as the largest cardinality of an algebraically independent subset of over .

For instance, let and be two field extensions of . Then the transcendence degree of them are and , respectively, by noting that is a transcendental number and is a algebraic number.

###### Conjecture 2 (Schanuel’s conjecture)

Given any complex numbers that are linearly independent over , the extension field has transcendence degree of at least over .

This conjecture was proposed by Stephen Schanuel during a course given by Serge Lang at Columbia in the 1960s [Lan66]. Schanuel’s conjecture concerning the transcendence degree of certain field extensions of the rational numbers. The conjecture, if proven, would generalize the most known results in transcendental number theory [Ter08, MW96]. For example, is algebraically independent simply by setting and , and using Euler’s identity .

###### Corollary 1

[GCL17, Corollary 3] Let be algebraic numbers that are linearly independent over . Based on Schanuel’s conjecture, the transcendence degree of the field extension is at least if .

### 2.5 Factoring Multivariate Polynomials over Algebraic Number Fields

Recall that PEF in Eq.(1) over algebraic number field has the form:

 f(t)=K∑k=0fk(t)eλkt (2)

where and for all .

is called exponential if there exists some such that

 f(t)≡a⋅eηt.

is called irreducible in if it has only trivial factoring as follows

 f(t,ea1t,…,eant)=eμt⋅g(t,eb1t,…,ebnt),

for some and a PEF .

We are interested in factoring a non exponential PEF into product of PEFs, i.e.,

 f(t,ea1t,…,eant)=eμt⋅fm11(t,eb1,1t,…,eb1,n1t)⋯fmss(t,ebs,1t,…,ebs,nst),

for some , and non-exponential .

###### Definition 8

A PEF is square free if

 f(t)≢f21(t)⋯fs(t)

where s are PEFs and is not exponential.

There is an efficient algorithm to factor multivariate polynomials over with polynomial-time complexity in the degrees of the polynomial to be factored [Len87]. The main idea is that the multivariate polynomial is first reduced to a polynomial in just one variable by substituting properly selected integers for all but one variable . The resulting univariate polynomial is then factored over . Then the well-known algorithm for factoring univariate polynomial over rational numbers [LLL82] can be generalized to factor .

We can employ this algorithm for factoring multivariate polynomials into the factoring of PEFs, an important step to get real root isolation of PEF over algebraic number field .

If is exponential, the factoring is trivial. Assume is not exponential, and we have the following factoring,

 f(t)=K∑k=0fk(t)eλkt=fm11(t,eb1,1t,…,eb1,n1t)⋯fmss(t,ebs,1t,…,ebs,nst),

where s are not exponential.

We know that can be written as some natural number combinations of ’s and this is sufficient requirement for ’s by expanding this equation observing that if and only if for all .

We first compute an integral basis of , says . Then are linearly independent over , and is a multivariate polynomial of , denoted by . We can always choose to be some natural number combination of , thus, are all multivariate polynomials of , denoted by . Therefore,

 P(t,ea1t,…,eant,e−a1t,…,e−ant)=Πsi=1Pmii(t,ea1t,…,eant,e−a1t,…,e−ant)

Comparing the polyomial degrees of both sides. The degree of the left hand side is the maximum degree of s, says . The right hand side is at least because the degree of PEFs is additive and each non-exponential PEF has degree at least 1.

Therefore,

 s∑i=1ms≤d.

There exists a positive integer such that

 P(t,ea1t,…,eant,e−a1t,…,e−ant)⋅e∑ni=1rdait=Q(t,ea1t,…,eant) Pi(t,ea1t,…,eant,e−a1t,…,e−ant)⋅e∑ni=1rmiait=Qi(t,ea1t,…,eant)

for some multivariate polynomials and s.

Now we have

 Q(t,ea1t,…,eant)=Πsi=1Qi(t,ea1t,…,eant)⋅e∑ni=1hait,

for some integer .

One can use the factoring algorithm for multivariate polynomials by regarding and ’s as multivariate polynomial on .

In Section 6, we will prove that assuming Schanuel’s conjecture, has only single roots excepting if is square free. Thus the algorithm of factoring multivariate polynomials over can be used to getting from PEF such that inherits all roots of and each root is single excepting 0. As we will see latter, this simplification plays essential role in finding a root isolation of .

## 3 Symbolic Dynamics of Continuous-time Markov Chains

We begin with continuous-time Markov chains. A continuous-time Markov chain is a Markovian (i.e. memoryless) stochastic process that takes values on a finite state set () and evolves in continuous-time . More formally:

###### Definition 9

A stochastic process with finite state set is a continuous-time Markov chain if it satisfies the Markov property, i.e. for any :

 p(Xt+k=j|Xt=i,{Xh:0≤h

where and is the conditional probability of events. The number of states is the dimension of the chain.

It turns out that a continuous-time Markov chain is fully characterized by a transition rate matrix  [GS07]. is -by- matrix and the off-diagonal entries are nonnegative rational numbers, representing the transition rate from state to state , while the diagonal entries are constrained to be for all . Consequently, the row summations of

are all zero. The dynamic of a continuous-time Markov chain is fully described by a master equation, which is a system of coupled ordinary differential equations that describe how the probability distribution changes over time for each of the states of the system. Specifically, the master equation is:

 dP(t)=P(t)Qdt,

where is the -by- transition probability matrix at time ; the quantity denotes the probability from state at time to state at time . The master equation is solved subject to initial conditions

, the identity matrix:

 P(t)=eQt.

Thus, given an initial distribution , the distribution at time is:

 μt=P(t)μ0=eQtμ0.

Therefore, we have the brief definition of continuous-time Markov chains:

###### Definition 10

A continuous-time Markov chain is a pair , where () is a finite state set, a -by- transition rate matrix.

W.l.o.g, we always denote in this paper.

###### Example 1

Consider continuous-time Markov chain , where ,

 Q=⎛⎜⎝−0.0250.020.0050.3−0.50.20.020.4−0.42⎞⎟⎠

Given continuous-time Markov chain , we denote the set of all distributions over , which is called the state space of continuous-time Markov chains, while in previous work (e.g.[ASSB00, CHKM11a, BHHK03]), is regarded as the state apace of the chain. The path of is a continuous function indexed by an initial distribution :

 σμ:R+→D(S),σμ(t)=eQtμ. (3)

We remark that defining path over distributions transitions is different to the usual way over the transitions of states set , i.e., functions with the starting state . Furthermore, must have finite variability, i.e., its set of discontinuities has no accumulation points (in other words, on any finite interval the value of can only change a finite number of times). This restriction is for applying analyzing methods of discrete-time Markov chains on the continuous-time counterpart as the domain is actually rather than . However, we do not put the constrain on the path such that we track all time instants not a part domain of continuous-time Markov chains. Thus in any time interval, the number of changing may be infinite.

We move to the symbolic dynamic (path) of continuous-time Markov chains. The symbolization of distributions is a generalization of the discretization of distributions in [AAGT15]. First, we fix a finite set of intervals . With the states , we define the symbolization of distributions as a function:

 S:D(S)→2S×IS(μ)={⟨j,Ik⟩:μ(j)∈Ik}. (4)

where denotes the Cartesian product, and is the set of all subsets of .

asserts that the -th element of is in the interval . Specially, in [AAGT15], must be fixed as a partition of , i.e., for all . Thus for any , we can label it by finite symbols from .

###### Example 2

Considering distributions

 μ0=⎛⎜⎝0.30.30.4⎞⎟⎠,μ1=⎛⎜⎝00.70.3⎞⎟⎠

suppose

 I={[0,0.2),[0.4,0.4],(0.5,0.7],(0.3,1]}.

Then, , .

The benefits of the symbolization are that in practice, we do not care the exact probability of some state but the range of the probability, which can be represented by intervals. In the next section, we will further explain this with our temporal logic CLL.

With the symbolization, we have

###### Definition 11

A symbolized continuous-time Markov chain is a tuple , where is a continuous-time Markov chain and is a finite set of intervals in .

Throughout this paper, we always assume that is rational, i.e., all elements of , and for all is a rational interval, i.e., the endpoints must be rational numbers. Thus , , and are all valid rational intervals for all . Furthermore, the elements of all the distributions are also rational.

Next, we extend this symbolization to the path :

 S∘σμ:R+→2S×I. (5)
###### Definition 12

Given a symbolized continuous-time Markov chain , is a symbolic dynamic (path) of .

In the end of this section, we prove that the path (continuous function) is a system of PEFs over algebraic number .

###### Lemma 1

Given a continuous-time Markov chain with an initial distribution , for any , , the -th element of , can be expressed as a PEF of over .

Proof. As the elements of are rational, we only need to prove that any entry of can be expressed as a finite sum of for and .

By Proposition 2, we have that there is a such that such that

 Jk=⎡⎢ ⎢ ⎢ ⎢ ⎢⎣λk10⋯00λk1⋯0⋱000⋯λk\par⎤⎥ ⎥ ⎥ ⎥ ⎥⎦

Note that

is an eigenvalue of

and , so is algebraic. Furthermore, , where is the dimension of .

Therefore, . So we complete the proof by proving that for each , any entry of can be expressed as a finite sum of for and . Computing , we obtain that

 eJkt=⎡⎢ ⎢ ⎢ ⎢ ⎢ ⎢⎣eλktteλktt2eλkt/2!⋯tnkeλkt/nk!0eλktteλkt⋯tnk−1eλkt/(nk−1)!⋱000⋯eλkt\par⎤⎥ ⎥ ⎥ ⎥ ⎥ ⎥⎦.

By the above lemma, analyzing dynamics of continuous-time Markov chains is equivalent to studying corresponding PEFs.

## 4 Continuous Linear Logic

In this section, we introduce continuous linear logic (CLL) to specify the temporal properties of symbolized continuous-time Markov chain . In summary, CLL is a linear-time temporal logic. Unlike LTL, CLL has the path and state formulas expressing temporal properties of . However, the path formulas in CLL are simpler because the only applied temporal operator is a timed version of until operator. Furthermore, the nesting of until temporal operator is allowed, which does not appear in the previous work [BHHK03]. More importantly, CLL formulas can express not only absolutely temporal properties (all-time instants basing on one starting time) but also relative versions. Up to our best knowledge, this is the first logic with this distinctive feature in the context of the verification of continuous-time Markov chains.

Remark: The next-step operator is important in the expressiveness of LTL and CSL with time-domain non-negative positive numbers . However, the time domain of CLL is non-negative real numbers , so “next-step” is not meaningful.

###### Definition 13

CLL path formulas are formed according to the following grammar:

 φ:=true∣Φ0UT1Φ1UT2Φ2…UTnΦn∣¬φ∣φ1∧φ2

where is a positive integer, for all , is a state formula, and ’s are arbitrary rational time intervals (with rational endpoints) in , i.e., is one of the forms:

 (a,b),[a,b],(a,b],[a,b)∀a,b∈Q+.

The syntax of CLL state formulas is described by the following grammar:

 Φ:=true∣a∈AP∣¬Φ∣Φ1∧Φ2

where denotes the set of atomic propositions.

For , the until operator is a timed variant of the until operator of LTL; the path formula asserts that is satisfied at some time instant in the interval and that at all preceding time instants holds . This can be extended to arbitrary by noting that is right-associative, e.g., stands for . One important point of CLL is that we consider the relative time, i.e., and do not have to be disjoint, and the starting time point of is based on some time event in . For instance is a valid path formula, which is different to CLL path formula . This picture will be more clear by the following semantics of CLL.

Given a symbolized continuous-time Markov chain , the semantics of CLL path formulas is defined on paths .

• for all distributions ;

• iff there is a time such that , and for any , , where iff , ;

• iff it is not the case that (written );

• iff and .

As we see, the CLL path formula is explained over the induction on . This makes time instants relative. This can be further explained by comparing and .

• asserts that there are time instants such that and for any and , and , where This is more clear in the following picture.