Runtime Verification (RV) [LeuS09, BarFF18, HavR18] is a lightweight formal technique in which program or system execution is monitored and analyzed. RV uses information extracted from an execution to check whether certain properties are satisfied or violated after a finite number of steps, possibly leading to online responses, such as signaling notifications or alarms, logging, computing statistical information, profiling, and performing error protection or recovery. In RV, properties are usually expressed using formalisms [HavR17] such as Linear Temporal Logic (LTL) formulas [Pnu77, ManP92, Dru00, BauLS11], Nondeterministic Büchi Automata (NBAs), and -regular expressions, which represent -regular languages [AmoR05, Bau10]. RV tools automatically synthesize monitors (i.e., code fragments) from formal specifications and then weave the code into the system through instrumentation [Gei01, HavR02, Hav08]. The inserted code typically maintains a set of monitor objects that can detect property satisfaction or violation during system execution. Such approaches have been extended to parametric RV, in which properties are checked over every parameter instance (i.e., a combination of parameter values) by maintaining a monitor object for every parameter instance [ChenR07, MerJGCR12, RosC12, ChenWZX16, Chen17, HavRT18].
Figure 1 shows a monitor specification, written in the Movec language [ChenWZX16], for the parametric RV of an event-driven system that dispatches a variety of events (e.g., sensor status, keystrokes, program loadings etc.) to components (e.g., libraries, mobile apps, microservices etc.).
Similar specifications can be written for other tools such as JavaMOP [ChenR07, MerJGCR12] and TraceMatches [AllAC05, AvgTM07].
This specification defines a parametric monitor, named
priority, which takes two parameters: a component ID
c and an event ID
e that should be instantiated with the values (i.e., actual arguments) generated by system execution.
The specification body begins with four actions, which extract information regarding function calls that occur during runtime:
r records a component being registered to an event (it also creates a monitor object by instantiating the monitor parameters with the arguments of the call),
u records an unregister,
b records the broadcast of an event (the argument of the call) to all components,
n records a certain component being notified of a specific event.
This specification is used to monitor system execution to check whether the property, specified as LTL formula , is satisfied or violated after a finite number of steps, i.e., any infinite future continuation makes the property satisfied or violated, respectively.
The property requires that if a component
c registers to an event
e and unregisters later, then before the unregister, the event
e cannot be broadcasted until
c has been notified (i.e.,
c has a higher priority than unregistered components in the context of receiving
In practice, if the satisfaction or violation of a property is detected by a monitor object then an associated handler (i.e., a piece of code) is automatically triggered to perform some online response [ChenWZX16, ChenR07, MerJGCR12]. For example, Figure 1 includes two handlers for the satisfaction (i.e., validation) and violation of the LTL formula: if the property is satisfied then a message is logged; if it is violated then an alarm is signaled and this prints the IDs of the component and the event. The two handlers may also be extended to more advanced operations, e.g., profiling and error recovery.
We may also monitor the system against other properties, e.g., that a component should receive notifications infinitely often after its registration, that a component unregisters after its registration, and that a registered component receives at least one notification before its deregistration. The developer may also write handlers for the satisfaction and violation of each property.
When specifying properties, the developer is usually concerned with their monitorability [PnuZ06, BauLS11, Bau10, DieL14], i.e., after any number of steps, whether the satisfaction or violation of the monitored property can still be detected after a finite future continuation. When writing handlers for these properties, the developer might consider the following question: “Can the handlers for satisfaction and violation be triggered during system execution?” We say that a verdict and its handler are active if there is some continuation that would lead to the verdict being detected and thus its handler being triggered. This question can be partly answered by deciding monitorability (with the traditional two-valued notion). For example, (above) is non-monitorable, i.e., there is some finite sequence of steps after which no verdict is active. Worse, is also weakly non-monitorable [ChenWW18], i.e., no verdict can be detected after any number of steps. Thus writing handlers for is a waste of time as they will never be triggered. More seriously, monitoring at runtime adds no value but increases runtime overhead. In contrast, , and are monitorable, i.e., some verdicts are always active. Thus their handlers must be developed as they may be triggered. However, this answer is still unsatisfactory, as the existing notion of monitorability suffers from two inherent limitations: limited informativeness and coarse granularity.
The existing notion of monitorability is not sufficiently informative, as it is two-valued, i.e., a property can only be evaluated as monitorable or non-monitorable. This means, for a monitorable property, we only know that some verdicts are active, but no information is available regarding whether only one verdict (satisfaction or violation) is active. As a result, the developer may still write unnecessary handlers for inactive verdicts. For example, , and are monitorable. We only know that at least one of satisfaction and violation is active, but this does not tell us which ones are active and thus which handlers are required. As a result, the developer may waste time in handling inactive verdicts, e.g., the violation of and the satisfaction of . Thus, the existing answer is far from satisfactory.
Limited informativeness also weakens the support for property debugging. For example, when writing a property the developer may expect that both verdicts are active but a mistake may lead to only one verdict being active. The converse is also the case. Unfortunately, these kinds of errors cannot be revealed by two-valued monitorability, as the expected property and the written (erroneous) property are both monitorable. For example, the developer may write formula while having in mind another one , i.e., what she/he really wants is wrongly prefixed by one G. These two formulas cannot be discriminated by deciding two-valued monitorability as both are monitorable.
The existing notion of monitorability is defined at the language-level, i.e., a property can only be evaluated as monitorable or non-monitorable as a whole, rather than a notion for (more fine-grained) states in a monitor. This means that we do not know whether satisfaction or violation can be detected starting from the current state during system execution. As a result, every monitor object must be maintained during the entire execution, again increasing runtime overhead. For example, is weakly monitorable, thus all its monitor objects (i.e., instances of the Finite State Machine (FSM) in Figure 2), created for every pair of component and event, are maintained.
Note that parametric runtime verification is NP-complete for detecting violations and coNP-complete for ensuring satisfaction [Chen17]. This high complexity primarily comes from the large number of monitor objects maintained for all parameter instances.
For state-level optimizations of monitoring algorithms, if no verdict can be detected starting from the current state of a monitor object, then the object can be switched off and safely removed to improve runtime performance.
For example, in Figure 2, only satisfaction can be detected starting from states
T, whereas no verdict can be detected starting from state
N. Thus a monitor object can be safely removed when it enters state
N. Unfortunately, the existing notion does not support such optimizations.
In this paper, we propose a new notion of four-valued monitorability for -languages, and apply it at the state-level, overcoming the two limitations discussed above.
First, the proposed approach is more informative than two-valued monitorability. Indeed, a property can be evaluated as a four-valued result, denoting that only satisfaction, only violation, or both are active for a monitorable property.
Thus, if satisfaction (resp. violation) is inactive, then writing handlers for satisfaction (resp. violation) is not required.
This can also enhance property debugging. For example, and can now be discriminated by their different monitorability results, as can never be satisfied but can be satisfied and can also be violated.
Thus, additional developer mistakes can be revealed.
Second, we can compute state-level weak monitorability, i.e., whether satisfaction or violation can be detected starting from a given state in a monitor.
For example, in Figure 2, state
N is weakly non-monitorable, thus a monitor object can be safely removed when it enters state
N, which achieves a state-level optimization.
In summary, we make the following contributions.
We propose a new notion of four-valued monitorability for -languages (Section 3), which provides more informative answers as to which verdicts are active. This notion is defined using six types of prefixes, which complete the classification of finite sequences.
We propose a procedure for computing four-valued monitorability of -regular languages, given in terms of LTL formulas, NBAs or -regular expressions (Section 4), based on a new six-valued semantics.
We propose a new notion of state-level four-valued weak monitorability and its computation procedure for -regular languages (Section 5), which describes which verdicts are active for a state. This can enable state-level optimizations of monitoring algorithms.
We have developed a new tool, Monic, that implements the proposed procedure for computing monitorability of LTL formulas. We evaluated its effectiveness using a set of 97 LTL patterns and formulas to (above). Experimental results show that Monic can correctly report both two-valued and four-valued monitorability (Section 6).
Let be a non-empty finite set of atomic propositions, and be a finite alphabet, i.e., the set of all states, where each state is a complete assignment of truth values to the propositions in . is the set of finite words (i.e., sequences of states in ), including the empty word , and is the set of infinite words over . We denote atomic propositions by , , , finite words by , , and infinite words by , unless explicitly specified. We write a finite or infinite word in the form , where a proposition appears in a state iff it is assigned true. We drop the brackets around singletons, i.e., .
[-languages] An -language (i.e., a linear-time infinitary property) over is a subset of , i.e., . That is, is a set of infinite words over . A finite word satisfies if , or violates if .
Linear Temporal Logic (LTL) [Pnu77, ManP92] is a typical representation of -languages. LTL extends propositional logic, which uses boolean connectives (not) and (conjunction), by introducing temporal connectives such as X (next), U (until), R (release), F (future, or eventually) and G (globally, or always). Intuitively, says that holds at the next state, says that at some future state holds and before that state always holds. Using the temporal connectives X and U, the full power of LTL is obtained. For convenience, we also use some common abbreviations: true, false, standard boolean connectives and , and additional temporal connectives (the dual to U), ( eventually holds), and ( always holds).
Good, bad and ugly prefixes
Kupferman and Vardi [KupV01] proposed the notions of good and bad prefixes, i.e., finite words that satisfy and violate an -language, respectively. Ugly prefixes were later introduced for runtime verification [BauLS07]. [Good, bad and ugly prefixes] Let be an -language. A finite word is a good prefix for if , a bad prefix for if , or an ugly prefix for if no finite extension makes it good or bad, i.e., and .
We denote by , and the set of good, bad and ugly prefixes for , respectively. Note that they do not constitute a complete classification of finite words. For example, any finite word of the form is neither a good nor a bad prefix for the LTL formula , and also is not an ugly prefix as it can be extended to a good prefix (ended with ) or a bad prefix (ended with ).
Bauer et al. [BauLS11] proposed a three-valued semantics for LTL, which evaluates an LTL formula wrt. a finite word as one of the truth values: true , false and inconclusive . This three-valued semantics can be generalized to -languages as follows: [Three-valued semantics] The truth value of an -language wrt. a finite word , denoted by , is an element of , defined as follows:
If is given by an LTL formula , then denotes the -language accepted by . Note that the truth values and denote that satisfies and violates , and so they identify good and bad prefixes for , respectively. For example, and is a good prefix for . However, the value ? does not identify ugly prefixes. For example, but is not an ugly prefix.
Pnueli and Zaks [PnuZ06] were the first to formalize a notion of monitorability. An -language is monitorable wrt. a finite word if that finite word can be extended to a finite word that satisfies or violates . Note that their notion depends on the finite word. Bauer et al. [BauLS11, Bau10] formalized a more complete notion of monitorability, defined as follows. [Monitorability] Let be an -language. is
positively determined by , if . ( is a good prefix)
negatively determined by , if . ( is a bad prefix)
-monitorable for , if , s.t. is positively or negatively determined by . ( has a good or bad extension, respectively)
monitorable if it is -monitorable for every .
In other words, is -monitorable if can be extended to a finite word that satisfies or violates , i.e., a good or bad prefix. is monitorable if every finite word can be extended to a good or bad prefix. Note that an ugly prefix can never be extended to a good or bad prefix. Thus, is non-monitorable iff there exists an ugly prefix for .
Deciding two-valued monitorability
Bauer et al. [BauLS11] presented a procedure for deciding the monitorability of -regular languages specified as LTL formulas, based on the three-valued semantics. The first step is a monitor construction procedure that transforms an LTL formula into a three-valued monitor, i.e., a deterministic FSM whose states are marked by outputs , or . The monitor reads a finite input word , and outputs if is a good prefix (the formula is satisfied), if is a bad prefix (the formula is violated), or if is undetermined. The monitor construction procedure first creates two NBAs for and . For each NBA, a state is marked by if the language of the NBA starting in is not empty. The two NBAs are then converted into two Nondeterministic Finite Automata (NFAs) whose accepting states are those marked by . The two NFAs are subsequently converted into two equivalent Deterministic Finite Automata (DFAs) and using a standard determinization, e.g., the power-set construction. The procedure finally obtains the deterministic FSM by computing and minimizing the product of the two DFAs. A state of the FSM outputs if it does not contain an accepting state of , if it does not contain an accepting state of , or otherwise. Note that each NBA is exponentially larger than the corresponding formula, and each DFA is exponentially larger than the corresponding NBA. Thus, this monitor construction procedure requires 2ExpSpace.
This construction procedure can be adapted to construct monitors for the -regular languages specified as NBAs and -regular expressions.
A three-valued monitor may contain , and states, which output , and respectively.
For example, in Figure 2, state
T is a state, whereas the remaining states are all states.
It has been shown that the three-valued monitor can be used to compute the truth value of an -language wrt. a finite word [BauLS11], which is the output of the corresponding monitor after reading this word.
Let be an -language, , , , , , be a three-valued monitor for , where is a finite set of states, is a finite alphabet, is a transition function, is an initial state, is an output alphabet and is an output function.
For any , .
The monitorability of an LTL formula can be determined by checking whether its monitor has a reachable ? state with no path leading to a or state [Bau10]. Such a ? state is reachable iff there exist ugly prefixes, thus iff is non-monitorable.
3 Four-valued monitorability
In this section, we propose a new notion of four-valued monitorability, to provide more informative answers to monitorability checking. As we promised, it can indicate whether only satisfaction, only violation, or both are active for a monitorable property. Two-valued monitorability cannot achieve this because its definition does not discriminate the finite words (i.e., in Definition 2) that can be extended to good or bad prefixes (which witness satisfaction or violation, respectively). Thus our idea is to discriminate accordingly these finite words by inspecting which types of prefixes they can be extended to.
Positive, negative and neutral prefixes
The traditional classification of prefixes (as the good, the bad and the ugly) is not satisfactory due to incompleteness, i.e., it does not include the finite words that are neither good nor bad but can be extended to good or bad prefixes. Thus we introduce the notions of positive, negative and neutral prefixes, in addition to good, bad and ugly prefixes, to complete the classification.
[Positive, negative and neutral prefixes] Let be an -language. A finite word is a
positive prefix for if it is not good, but some finite extension makes it good but never bad, i.e., , , and ,
negative prefix for if it is not bad, but some finite extension makes it bad but never good, i.e., , , and , or
neutral prefix for if some finite extension makes it good and some makes it bad, i.e., and .
We denote by , and the set of positive, negative and neutral prefixes for , respectively. It is easy to see that the three new sets of prefixes and the three traditional sets of good, bad and ugly prefixes are mutually disjoint. An interesting fact, as shown by the following theorem, is that the six sets of prefixes exactly constitute the complete set of finite words. Furthermore, the six types of prefixes directly correspond to the six-valued semantics (shown later). This completes the classification of prefixes. .
The traditional three-valued semantics can identify only good and bad prefixes with the truth values and respectively, whereas all the prefixes of the other four types are given the same value . To discriminate them, we further divide the value into four truth values: probably true , probably false , probably conclusive and inconclusive . An -language is evaluated wrt. a finite word, which is neither good nor bad, as probably true if the word can be extended to a good prefix but never a bad prefix, probably false if the word can be extended to a bad prefix but never a good prefix, probably conclusive if the word can be extended to good and bad prefixes, or inconclusive if neither.
[Six-valued semantics] The truth value of an -language wrt. a finite word , denoted by , is an element of , defined in Figure 3.
The six truth values in actually denote that the finite word is a good, bad, positive, negative, neutral or ugly prefix for , respectively. Thus the six-valued semantics can be used to evaluate whether a finite word can be extended to witness the satisfaction and violation of an -language: Satisfaction is possible iff the -language is evaluated wrt. the finite word as true, probably true or probably conclusive, while violation is possible iff the -language is evaluated wrt. the finite word as false, probably false or probably conclusive.
The definition of four-valued monitorability is built on the notion of four-valued -monitorability which is used to discriminate finite words by inspecting which types of prefixes they can be extended to. [Four-valued -monitorability] Let be an -language. is
weakly positively -monitorable for , if , s.t. is positively determined by . ( has a good extension.)
weakly negatively -monitorable for , if , s.t. is negatively determined by . ( has a bad extension.)
positively -monitorable if it is weakly positively, but not weakly negatively, -monitorable. ( has only good extensions, thus is a good or positive prefix.)
negatively -monitorable if it is weakly negatively, but not weakly positively, -monitorable. ( has only bad extensions, thus is a bad or negative prefix.)
neutrally -monitorable if it is both weakly positively and weakly negatively -monitorable. ( has both good and bad extensions, thus is a neutral prefix.)
not -monitorable if it is neither weakly positively nor weakly negatively -monitorable. ( has neither good nor bad extension, thus is an ugly prefix.)
In other words, the traditional notion of -monitorability is divided into two parts, i.e., weak positive and weak negative -monitorability. As a result, is -monitorable iff is positively, negatively or neutrally -monitorable.
[Four-valued monitorability] Let be an -language. is
positively monitorable if it is positively -monitorable for every .
negatively monitorable if it is negatively -monitorable for every .
neutrally monitorable if it is -monitorable for every , and is neutrally -monitorable for the empty word .
non-monitorable if it is not -monitorable for some .
In other words, the set of monitorable properties is divided into three classes, i.e., positively, negatively and neutrally monitorable properties. Note that the definition of neutral monitorability consists of two conditions, of which the first ensures that is monitorable while the second ensures that both of satisfaction and violation can be detected after some finite sequences of steps. We denote the four truth values (positively, negatively, neutrally and non-monitorable) by , , and respectively.
We can validate that four-valued monitorability indeed provides the informativeness we require, as described in Section 1, by showing the following theorem, that the truth values , , and indicate that only satisfaction, only violation, and both can be detected for a monitorable property after some finite sequences of steps, respectively. This theorem can be proved by using Definition 3, in which is substituted by the empty word . Let be an -language. If is
positively monitorable then and .
negatively monitorable then and .
neutrally monitorable then and .
Formula is positively monitorable, as any finite word can be extended to a good prefix (ended with ) but never a bad prefix. This means that only satisfaction, but not violation, of the property can be detected after some finite sequences of steps.
Formula is negatively monitorable, as any finite word can be extended to a bad prefix (ended with ) but never a good prefix. This means that only violation, but not satisfaction, of the property can be detected after some finite sequences of steps.
Formula is neutrally monitorable, as it is monitorable and (more generally, any finite word of the form ) can be extended to both a good prefix (ended with ) and a bad prefix (ended with ). This means that both of satisfaction and violation of the property can be detected after some finite sequences of steps.
Formula is non-monitorable, as any finite word can never be extended to a good or bad prefix, due to the infinite continuations and respectively. This means that neither satisfaction nor violation of the property can be detected.
4 Computing four-valued monitorability of -regular languages
In this section, we propose a procedure for computing the four-valued monitorability of -regular languages, based on the six-valued semantics.
The first step is a monitor construction procedure that transforms an LTL formula into a six-valued monitor, i.e., a deterministic FSM whose states are marked by outputs , , , , or . The monitor reads a finite input word , and outputs if is a good prefix (the formula is satisfied), if is a bad prefix (the formula is violated), if is a positive prefix (the formula can be satisfied, but never violated, after a finite continuation), if is a negative prefix (the formula can be violated, but never satisfied, after a finite continuation), if is a neutral prefix (the formula can be satisfied or violated after a finite continuation), or if is an ugly prefix (the formula can be neither satisfied nor violated).
The monitor construction procedure first constructs a three-valued monitor, using the traditional approach for deciding two-valued monitorability, which requires 2ExpSpace [BauLS11]. Then we refine its output function, assigning new outputs to states marked . Thus, our procedure traverses all the states in the monitor, and for each state, starts another nested traversal to check whether a state or a state is reachable. A state is assigned output if only states are reachable, if only states are reachable, if both and states are reachable, or if neither is reachable. This step requires polynomial time (as the three-valued monitor is the input). Thus, monitor construction requires 2ExpSpace; the same complexity as for two-valued monitorability. As we will see in Section 6, the small size of standard LTL patterns means that four-valued monitorability can be computed in very little time. Let us formalize the above monitor construction procedure (based on a three-valued monitor). Let be an -language, , , , , , be a three-valued monitor for . The corresponding six-valued monitor , , , , , is obtained by refining the output function of as shown in Figure 4.
For example, in Figure 2, states
N are all states under the three-valued semantics. After refining the output function, states
P2 are states, whereas state
N is a state under the six-valued semantics.
It is easy to prove the following lemma, which states that the six-valued monitor can be used to compute the truth value of an -language wrt. a finite word (by Definitions 3 and 2, Lemma 2 and Definition 4).
Let be an -language, , , , , , be a six-valued monitor for .
For any , .
As a property of the six-valued monitor, the following theorem shows that each state in a monitor can be reached by exactly one type of prefixes (by Lemma 4 and Definition 3). Let be an -language, , , , , , be a six-valued monitor for . For a state , equals , , , , or , iff it can be reached by good, bad, positive, negative, neutral or ugly prefixes, respectively.
The four-valued monitorability of an -language (Definition 3) can be determined by checking whether its monitor has specific reachable states. The monitorability of is , iff a state is reachable (the same as the two-valued case). The monitorability of is , iff neither nor states are reachable (thus neither nor states are reachable). The monitorability of is , iff neither nor states are reachable (thus neither nor states are reachable). The monitorability of is , iff no state is reachable, but a state is reachable (thus both and states are reachable). These rules can be formalized: Let be an -language, , , , , , be a six-valued monitor for . The monitorability of :
The above checking procedure can be done in linear time by traversing all the states of the six-valued monitor. However, note that this procedure is performed after constructing the six-valued monitor. Thus, when an -regular language is given in terms of an LTL formula, the four-valued monitorability of can be computed in 2ExpSpace.
Now consider other representations of -regular languages. If is given in terms of an NBA, we first explicitly complement the NBA, and the rest of the procedure stays the same. However, the complement operation also involves an exponential blowup. If is given in terms of an -regular expression, we first build an NBA for the expression, which can be done in polynomial time, and the rest of the procedure is the same as for NBA. Hence, independent of the concrete representation, four-valued monitorability of an -regular language can be computed in 2ExpSpace, by using the procedure based on monitor construction.
5 State-level four-valued monitorability
In this section, we apply four-valued monitorability at the state-level, to predict whether satisfaction and violation can be detected starting from a given state in a monitor. Recall that the notions of monitorability (cf. Definitions 2 and 3) are defined using the extensions to good and bad prefixes. However, good and bad prefixes are defined for an -language, not for a state. Thus such definitions cannot be directly applied at the state-level. Instead, we define state-level monitorability using the reachability of and states, which are equivalent notions to good and bad prefixes according to Theorem 4.
[State-level four-valued monitorability] Let be a six-valued monitor. A state is
positively monitorable if a state but no state is reachable from , for every state reachable from .
negatively monitorable if a state but no state is reachable from , for every state reachable from .
neutrally monitorable if a state or a state is reachable from , for every state reachable from , and both a state and a state are reachable from .
non-monitorable if neither states nor states are reachable from , for some state reachable from .
Note that the above notion of state-level monitorability is too strong to meet our requirements, because it places restrictions on all the states reachable from the considered state.
For example, in Figure 2, satisfaction can be detected starting from states
P2, but neither satisfaction nor violation can be detected starting from state
N. However, states
N are all non-monitorable as neither states nor states are reachable from state
N. That is, the above notion cannot discriminate state
N from states
P2. Thus we should use a weaker form of state-level monitorability as follows.
[State-level four-valued weak monitorability] Let be a six-valued monitor. A state is
weakly positively monitorable if a state but no state is reachable from .
weakly negatively monitorable if a state but no state is reachable from .
weakly neutrally monitorable if both a state and a state are reachable from .
weakly non-monitorable if neither states nor states are reachable from .
A state is weakly monitorable (with the two-valued notion), iff it is weakly positively, weakly negatively or weakly neutrally monitorable (with the four-valued notion).
For example, in Figure 2, states
T are all weakly positively monitorable as state
T is a state, while state
N is weakly non-monitorable. Thus state
N can now be discriminated from states
We can validate that state-level four-valued weak monitorability can indeed predict whether satisfaction and violation can be detected starting from a given state, as anticipated in Section 1, by showing the following theorem, that the truth values , , and indicate that only satisfaction, only violation, both and neither can be detected, respectively. This theorem can be proved by using Definition 5 and Theorem 4. Let be a six-valued monitor. Suppose a state can be reached from by reading , i.e., . If is
weakly then and .
weakly then and .
weakly then and .
weakly then and .
The four truth values can be used for a state-level optimization of monitoring algorithms:
If a state is weakly positively (resp. negatively) monitorable, then a monitor object can be safely removed when it enters this state, provided that only violation (resp. satisfaction) handlers are specified, as no handler can be triggered.
If a state is weakly neutrally monitorable, then a monitor object must be preserved if it is at this state as both satisfaction and violation can be detected after some continuations.
If a state is weakly non-monitorable, then a monitor object can be safely removed when it enters this state as no verdict can be detected after any continuation.
Besides, a monitor object can also be removed when it enters a state or a state, as any finite or infinite continuation yields the same verdict.
Let us consider the relationship between the language-level monitorability and the state-level weak monitorability. The following lemma shows that the monitorability of an -language depends on the weak monitorability of all the reachable states of its monitor. Let be an -language, , , , , , be a six-valued monitor for . is monitorable iff every reachable state of is weakly monitorable.
This result shows that if an -language is non-monitorable, then the state space of its monitor may consist of weakly monitorable states and weakly non-monitorable states. If a state is weakly monitorable then satisfaction or violation can be detected starting from this state. Otherwise, neither satisfaction nor violation can be detected.
Let us consider how one can compute state-level four-valued weak monitorability for each state in a six-valued monitor. Note that and states are weakly positively monitorable, and states are weakly negatively monitorable, states are weakly neutrally monitorable, and states are weakly non-monitorable. This correspondence shows that state-level weak monitorability can be computed from the output function of the monitor. We formalize this correspondence by introducing the following vtom operator. [Value-to-weak-monitorability] Let be the value-to-weak-monitorability operator that converts a truth value in into the corresponding result of weak monitorability in , defined as follows: , , and .
Let be an -language, , , , , , be a six-valued monitor for . The four-valued weak monitorability of equals .
6 Implementation and experimental results
We have developed a new tool, Monic, that implements the proposed procedure for computing four-valued monitorability of LTL formulas (cf. Section 4). Monic also supports deciding two-valued monitorability. We have evaluated its effectiveness using a set of LTL formulas, including formulas to (used in Section 1) and Dwyer et al.’s 97 LTL patterns [DwyAC99, BauLS11]. The evaluation was performed on an ordinary laptop, equipped with an Intel Core i7-6500U CPU (at 2.5GHz), 4GB RAM and Ubuntu Desktop (64-bit).
The result on formulas to shows that: is neutrally monitorable, is non-monitorable, is positively monitorable, is negatively monitorable, is neutrally monitorable, and is non-monitorable (but weakly monitorable). Thus, the violation of and the satisfaction of can never be detected, whereas both verdicts are active for and . Further, and can be discriminated by their different monitorability results.
We also ran Monic on Dwyer et al.’s specification patterns [DwyAC99, BauLS11], of which 97 are well-formed LTL formulas. The result shows that 55 formulas are monitorable and 42 are non-monitorable. For those monitorable ones, 6 are positively monitorable, 40 are negatively monitorable and 9 are neutrally monitorable. Our result disagrees with the two-valued result reported in [BauLS11] only on one formula. A manual inspection of its monitor shows that our result is correct (cf. Appendix A for details).
Finally, the above results for to and the 97 LTL patterns were computed in 0.03 and 0.07 seconds, with 16M and 20M memory consumed, respectively (all reported by GNU
To conclude, the results show that Monic can correctly report both two-valued and four-valued monitorability of typical formulas in very little time.
7 Related work
Monitorability is a principal foundational question in RV because it delineates which properties can be monitored at runtime. The classical results on monitorability have been established for -languages, especially for LTL [PnuZ06, BauLS11, Bau10]. Francalanza and Aceto et al. have studied monitorability for the Hennessy-Milner logic with recursion, both with a branching-time semantics [Fra16, FraAA17, FraAI17, AceAF18] and with a linear-time semantics [AceAF19a]. There exist some variants of monitorability as well. For example, monitorability has been considered over unreliable communication channels which may reorder or lose events [KauHF19]. However, all of the existing works only consider two-valued notions of monitorability at the language-level.
Monitorability has been studied in other contexts. For example, a topological viewpoint [DieL14] and the correspondence between monitorability and the classifications of properties (e.g., the safety-progress and safety-liveness classifications) [FalFM09, FalFM12, PelH19] have been established. A hierarchy of monitorability definitions (including monitorability and weak monitorability [ChenWW18]) has been defined wrt. the operational guarantees provided by monitors [AceAF19b].
A four-valued semantics for LTL [BauLS07, BauLS10] has been proposed to refine the three-valued semantics [BauLS11]. This divides the inconclusive truth value ? into two values: currently true and currently false, i.e., whether the finite sequence observed so far satisfies the property based on a finite semantics for LTL. Note that it provides more information on what has already been seen, whereas our six-valued semantics describes what verdicts can be detected in the future continuation.
We have proposed four-valued monitorability and the corresponding computation procedure for -regular languages. Then we applied the four-valued notion at the state-level. To our knowledge, this is the first study of multi-valued monitorability, inspired by practical requirements from RV. We believe that our work and implementation can be integrated into RV tools to provide information at the development stage and thus avoid the development of unnecessary handlers and the use of monitoring that cannot add value, enhance property debugging, and enable state-level optimizations of monitoring algorithms.
Appendix A Experimental results
Our result disagrees with the two-valued result reported in [BauLS11] only on the 6th LTL formula listed in the Appendix of [BauLS11]. More precisely, Monic reports negatively monitorable, whereas the result in [BauLS11] reports non-monitorable.
The formula is as follows (
! for ,
& for ,
| for ,
-> for ,
U for U,
<> for F,
 for G):
(("call" & <>"open") -> ((!"atfloor" & !"open") U ("open" | (("atfloor" & !"open") U ("open" | ((!"atfloor" & !"open") U ("open" | (("atfloor" & !"open") U ("open" | (!"atfloor" U "open"))))))))))
A manual inspection of its monitor (in Figure 5) shows that our result is correct. Indeed, state
F is a state, and states
N7 are all states that can reach the state
Appendix B Proof sketches
Let be an -language, , , , , , be a six-valued monitor for . The monitorability of :
() This implication follows by the converse of the above reasoning. ∎
Let be an -language, , , , , , be a six-valued monitor for . is monitorable iff every reachable state of is weakly monitorable.
() Suppose is monitorable. By Definition 2, for every , there exists , such that or . By Definition 3, this means . By Lemma 4, we have . Let . It follows . That is, a state or a state is reachable from , thus is weakly monitorable by Definition 5. It is easy to see denotes any reachable state, thus every reachable state of is weakly monitorable.
() This implication follows by the converse of the above reasoning. ∎
Let be an -language, , , , , , be a six-valued monitor for . The four-valued weak monitorability of equals .