Invariant Checking of NRA Transition Systems via Incremental Reduction to LRA with EUF

01/26/2018 ∙ by Alessandro Cimatti, et al. ∙ 0

Model checking invariant properties of designs, represented as transition systems, with non-linear real arithmetic (NRA), is an important though very hard problem. On the one hand NRA is a hard-to-solve theory; on the other hand most of the powerful model checking techniques lack support for NRA. In this paper, we present a counterexample-guided abstraction refinement (CEGAR) approach that leverages linearization techniques from differential calculus to enable the use of mature and efficient model checking algorithms for transition systems on linear real arithmetic (LRA) with uninterpreted functions (EUF). The results of an empirical evaluation confirm the validity and potential of this approach.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Invariant checking for infinite-state transition systems is a fundamental research area. Based on the recent improvements of SMT technologies, effective approaches have been developed for the case of transition systems with dynamics over Linear Real Arithmetic [9, 18, 21, 4]. However, many real-world industrial designs (e.g. aerospace, automotive) require modeling as transition systems over non-linear arithmetic (NRA). Although both problems are undecidable, proving properties of the NRA transition systems turns out to be much harder than the linear case, and has in fact received much less attention. Approaches based on BMC and k-induction [29, 15] are possible, so that non-linearity is handled at the SMT-level, by means of an SMT(NRA) solver (e.g. Z3 [13], nlSAT [20], Yices [14], SMT-RAT [1]). Their power is however limited. Consider the following simple transition system: initially, ; the transition relation is defined by . The property “it is always the case that

” is not k-inductive, not even for a very large value of k. Thus, the typical proving techniques that are based on k-induction using an SMT(NRA) solver will not be able to prove it. In principle, it is also possible to lift other approaches (e.g. interpolation, IC3 

[26, 9]) to handle non-linearities at the level of the solver. However, this requires the SMT(NRA) solver to carry out interpolation or quantifier elimination, and to proceed incrementally. These extra functions are usually not available, or they have a very high computational cost.

In this paper, we propose a completely different approach to tackle invariant checking for NRA transition systems. Basically, we work with an abstract version of the transition system, expressed over LRA with EUF, for which we have effective verification tools [9]. In the abstract space, nonlinear multiplication is modeled as an uninterpreted function. When spurious counter-examples are found, the abstraction is tightened by the incremental introduction of linear constraints, including tangent planes resulting from differential calculus, and monotonicity constraints.

We implemented the approach on top of the nuXmv model checker [7], leveraging the IC3 engine with Implicit Abstraction [9] for invariant checking of transition systems over LRA with EUF. We compared it, on a wide set of benchmarks, against multiple approaches working at NRA level, including BMC and k-induction using SMT(NRA), the recent interpolation-based iSAT3 engine [24], and the static abstraction approach proposed in [8]. The results demonstrate substantial superiority of our approach, that is able to solve the highest number of benchmarks.

The effectiveness of our approach is possibly explained with the following insights. On the one hand, in contrast to LRA, NRA is a hard-to-solve theory: in practice, most available complete solvers rely on CAD techniques [12], which require double exponential time in worst case. Thus, we try to avoid NRA reasoning, trading it for LRA and EUF reasoning. On the other hand, proving properties of practical NRA transition systems may not require the full power of non-linear solving. In fact, some systems are “mostly-linear” (i.e. non-linear constraints are associated to a very small part of the system), an example being the Transport Class Model (TCM) for aircraft simulation from the Simulink model library [19]. Furthermore, even NRA transition systems with significant non-linear dynamics may admit a piecewise-linear invariant of the transition system that is strong enough to prove the property.

Structure. In Sec. 2 we discuss the related work, and in Sec. 3 introduce some background. In Sec. 4 we discuss the approach in the setting of SMT(NRA). In Sec. 5 we present the verification algorithm for NRA transition systems. In Sec. 6 we describe the results of the experimental evaluation. In Sec. 7 we conclude and outline the directions for future research.

2 Related Work

There are not many tools that deal with NRA transition systems. The most relevant is the recently proposed iSAT3 [28], that uses an interpolation-based [23, 24] approach to prove invariants. In addition to NRA, it also supports trascendental functions and some form of differential equations. iSAT3 is built on an SMT solver based on numeric techniques (interval arithmetic), and is able to provide results that are accurate up to the specified precision. In fact, in addition to “safe” and “unsafe” answers, iSAT3 may return “maybe unsafe” when it finds an envelope of given precision that may (but is not guaranteed to) contain a counterexample. Another relevant tool is dReach [22], a bounded model checker implemented on top of the dReal [16] SMT solver, that adopts numerical techniques similar to iSAT3. dReach has an expressiveness similar to iSAT3, but being a bounded model checker it is unable to prove properties.

The work in [8] follows a reduction-based approach to check invariants of NRA transition systems. It over-approximates the non-linear terms with a coarse abstraction, encoding into LRA some weak properties of multiplication like identity and sign. Another reduction-based approach is presented in [25] in the context of program analysis. The idea is to find a (tighter) convex approximation of polynomials in form of polyhedra, thus obtaining a conservative linear transition system. The key differences of our approach with respect to [8, 25] are that we iteratively refine the abstraction, and we adopt a reduction to LRA+EUF. Furthermore, to the best of our knowledge, there is no available implementation of the approach [25] in a program analysis tool – it has been only shown to work on SMT problems.

The idea of approximating a univariate function (in particular the natural logarithm ) with tangent lines is used in [30]. Here we abstract a bivariate function (multiplication), and use tangent planes for the refinement. We also exploit other properties (e.g. monotonicity) to derive additional axioms. The idea of using tangent planes (spaces) has been explored in [27], limited to the case of SMT solving. Another key differences is that the tangent planes area used to under-approximate predicates, while we use them to refine the over-approximation of the multiplication function.

3 Background

3.0.1 Properties of the Multiplication Function.

(a)
(b) (top view)
(c) and tangent plane
(d) and tangent plane (top view)
Figure 5: Multiplication function and tangent plane.

Geometrically, the surface generated by the multiplication function is shown in Fig. (a)a and (b)b. This kind of surface is known in geometry as hyperbolic paraboloid. A hyperbolic paraboloid is a doubly-ruled surface, i.e. for every point on the surface, there are two distinct lines projected from the surface such that they pass through the point. In case of the multiplication surface, the projected lines basically lie on the surface.

Tangent Plane.

The tangent plane to a surface at a point of interest is a plane that “just touches” the surface at the point. The tangent planes can be used to linearly approximate the surface at the point of interest. An important property of the tangent plane to a hyperbolic paraboliod is that the two projected lines from the surface are also in the tangent plane, and they define how the plane cuts the surface (see Fig. (c)c and (d)d). The tangent plane to the multiplication function at point is calculated as follows:

where and are the first-order partial derivatives of w.r.t. and respectively, evaluated at . simplifies to:

(1)

3.0.2 Logic and Satisfiability.

We assume the standard first-order quantifier-free logical setting and standard notions of theory, model, satisfiability, and logical consequence. If is a formula, we denote with the set of its variables, and with the set of its atoms. We write to denote that . If and are two variables, we denote with the formula obtained by replacing all the occurrences of in with . We extend this notation to ordered sequences of variables in the natural way. If is a model and is a variable, we write to denote the value of in , and we extend this notation to terms in the usual way. If is a set of variables, we denote with the set obtained by replacing each element with , and with the set obtained by replacing with . If is a set of formulas, we write to denote the formula obtained by taking the conjunction of all its elements. If is unsatisfiable (modulo some theory ), an unsatisfiable core is a set such that is still unsatisfiable.

3.0.3 Symbolic Transition Systems.

A symbolic transition system is a tuple where is a finite set of (state) variables, is a formula denoting the initial states of the system, and is a formula expressing its transition relation. A state of is an assignment to the variables . A path (execution trace) of length (possibly infinite) for is a sequence of states such that and for all . We call an unrolling of of length the formula

Let be a formula whose assignments represent a property (good states) over the state variables . The invariant verification problem, denoted with , is the problem of checking if for all the finite paths of , for all , , . Its dual formulation in terms of reachability of is the problem of finding a path of such that . represents the “good” states, while represents the “bad” states.

4 Solving SMT(NRA) via SMT(LRA+EUF)

4.0.1 Top-level Algorithm.

The main idea of this paper is that of solving an SMT formula containing non-linear polynomial constraints (i.e., expressed in the NRA theory) by overapproximating it with a formula over the combined theory of linear arithmetic and uninterpreted functions (LRA+EUF). Our main SMT solving procedure follows a classic abstraction refinement loop, in which at each iteration the current overapproximation of the input SMT formula is refined by adding new constraints that rule out one (or possibly more) spurious solutions, until one of the following occurs: (i) the SMT formula becomes unsatisfiable in the LRA+EUF theory; or (ii) the LRA+EUF model for the current overapproximation can be lifted to an NRA model for the original SMT formula; or (iii) the resource budget (e.g. time, memory, number of iterations) is exhausted.

The pseudocode for the top-level algorithm is shown in Fig. 6. We provide more details about its main components in the rest of this section.

Figure 6: Solving SMT(NRA) via abstraction to SMT(LRA+EUF).

4.0.2 Initial Abstraction.

The function initial-abstraction takes as input an SMT(NRA) formula and returns an overapproximation of it in the LRA+EUF theory.

First, each multiplication expression between two variables111To simplify the presentation, we assume (here and in the rest of the paper) that all multiplications in are either between two variables or between one constant and one variable. occurring in is replaced by , where is a binary uninterpreted function returning a real. We remark that this happens only for non-linear multiplications: expressions like or in which is a constant are not rewritten.

Then, some simple axioms about multiplication are added to via static learning. For each , we add the following axioms:

Commutativity:
Sign:
Zero:

4.0.3 Abstraction Refinement.

If the SMT check on the LRA+EUF abstraction returns false (line LABEL:code:UF-LRA-check of Fig. 6), we can conclude that the input formula is unsatisfiable. In this case, contains all the lemmas (discussed later in this section) that were added in the earlier refinements (line LABEL:code:refine of Fig. 6).

Otherwise, we have to check whether the model found for is also a model for the original NRA formula . Let be the set of all terms occurring in . In its simplest version, the function get-NRA-model checks whether, for all in , . If this is the case, then is also a model for the original formula, and get-NRA-model returns true. (We present more sophisticated versions of get-NRA-model below.) Otherwise, let be the set of all terms whose value in is different from . The function refine generates a set of axioms such that there exists at least one element of such that the formula has no model that agrees with on the values of and (i.e. such that , and ). Intuitively, the axioms block the bad model values for , making the abstraction more precise by restricting the set of spurious solutions.

In our current implementation, two kinds of lemmas are generated during refinement: tangent lemmas and monotonicity lemmas.

Tangent Lemmas.

We use the model values , and and (1) to generate tangent plane lemmas for :

(2)

where we can choose and as:

(3)
(4)
(5)

Basically the equalities in the tangent lemma are providing multiplication lines that enforce the correct value of when or . Moreover, the inequalities of the tangent lemma are providing bounds for when and are not on the multiplication lines.

Monotonicity Lemmas.

Let and be two terms in , such that , , and . Then, we add the monotonicity lemma

(6)

where stands for .

4.0.4 Finding Models.

It is easy to see that our algorithm is expected to perform much better for unsatisfiable instances than for satisfiable ones. The algorithm can return true (meaning that the formula is satisfiable) only if the LRA+EUF solver “guesses” a model that is consistent with all the nonlinear multiplications. In an infinite and dense domain like the reals, the chances that this will happen are close to zero in general.

Moreover, our approach is inherently limited, because it can only find models over the rationals. If the input formula is satisfiable, but all its models contain some irrational values, then our algorithm will always abort (or never terminate, if there is no resource budget set). In practice, it is very likely that the same will happen even for formulas admitting a rational solution.

One possibility for addressing this limitation would be to couple our procedure with a complete solver for NRA, to be used for detecting satisfiable cases, in order to implement a more effective version of get-NRA-model . One such possibility is shown in Fig. 8, where we extract the truth assignment induced by the LRA+EUF model on the atoms of :

(7)

We concretize it by replacing each in with , and invoke the complete NRA theory solver on the resulting conjunction of NRA-literals , to check whether it contains at least one solution. Although in general the problem is expected to be simpler than the original input formula because the Boolean structure of is disregarded, invoking a complete NRA theory solver at each loop iteration of SMT-NRA-check-abstract-ext could be very expensive. Moreover, this would still require a complete NRA theory solver, which might not always be available.

 

Figure 7: A complete procedure using an NRA solver.
Figure 8: An incomplete procedure using an SMT(LRA+EUF) solver.

As an alternative, we propose the procedure outlined in Fig. 8, where we extract the truth assignment induced by the LRA+EUF model on the atoms of , and we conjoin to it the multiplication lines:

(8)

being the usual set of all terms occurring in .

The main idea is to build an LRA+EUF underapproximation of the NRA formula of Fig. 8, in which all multiplications are forced to be linear. Compared to the previous solution, this has the advantage of requiring a complete SMT(LRA+EUF) solver rather than a (much more expensive) complete NRA solver. Moreover, given the simplicity of the Boolean structure of the underapproximated formula, the check should in general be very cheap. The drawback is that this is (clearly) still an incomplete procedure. However, in our experiments (for which we refer to Sec. 6) we have found it to be surprisingly effective for many problems.

Unlike with the basic implementation of get-NRA-model which considers only one single candidate model at a time, the implementations in Fig.8 and Fig. 8 consider an infinite amount of them, drastically increasing the chances of finding a model.

Correctness and progress.

We notice that the procedure in Fig. 6 is correct. In fact, it returns false only if is NRA-unsatisfiable because by construction is an over-approximation of , and all axioms in are valid in any theory interpreting as . Also, it returns true only if is NRA-satisfiable:

  • if get-NRA-model is based only on evaluation, then by construction is an LRA+EUF-model for s.t. each equals in , so that is also a model for ;

  • if get-NRA-model is as in Fig. 8, then is an NRA-model of a conjunction of literals which tautologically entails , so that is a model for ;

  • if get-NRA-model is as in Fig. 8, then is an LRA+EUF-model of a conjunction of literals which tautologically entails and it is s.t. each equals in , so that is a also model for .

We also notice that the progress of the procedure in Fig. 6 is guaranteed by the refinement step, which rules out significant parts of the search space at every loop by means of the added lemmas.

4.0.5 Important Heuristics for Refinement.

The description of refine provided above leaves some flexibility in deciding what axioms to add (and how many of them) at each iteration. It is possible to conceive strategies with an increasing degree of eagerness, from very lazy (e.g. adding only a single axiom per iteration) to more aggressive ones. In our current implementation, we eagerly add all the axioms (2)–(6) that are violated by the current abstract solution , leaving the investigation of alternative strategies as future work. However, we found the following two strategies to be crucial for performance.

Tangent lemma frontiers.

The tangent lemmas of (2) for a given point are based on the fact that the multiplication function is a hyperbolic paraboloid surface, and a tangent plane to such surface cuts the surface into four regions such that in two of the regions the tangent plane is above the surface, whereas in the other two regions the tangent plane is below the surface (see Fig. 5). Each instantiation of (2) for a given point, therefore, can only provide either a lower or an upper bound for a given region. In some cases, this might lead to an infinite refinement loop in which at each iteration the “wrong” bound is refined. In order to address the problem, we use the following strategy. For each in the input formula, we maintain a frontier with the invariant that whenever is in the interval or is in the interval , then has both an upper and a lower bound. Initially, the frontiers are set to . Whenever a lemma (2) for is instantiated on a point , we generate further instantiations of (2) and update the frontier as follows:

case and :

instantiate (2) on and on , and set the frontier to ;

case and :

instantiate (2) on and on , and set the frontier to ;

case and :

instantiate (2) on and on , and set the frontier to ;

case and :

instantiate (2) on and on , and set the frontier to .

Fig. 13 shows a graphical illustration of the strategy.

(a) current frontier
(b) new point
(c) instantiation of (2) on
(d) additional instantiations and updated frontier
Figure 13: Illustration of the tangent lemma frontier strategy.

Tangent lemma rounding.

The instantiation of a tangent lemma at the point has the side-effect of adding the rational constants , and to the formula that is solved by the LRA+EUF solver. If such values have large numerators and/or denominators, they might be a source of a significant slow-down for the LRA solver (which works on exact, arbitrary-precision rational arithmetic). We address this issue by observing that, in order to block a bad model such that , it is sufficient to add one of the two equalities of (2); therefore, instead of instantiating a tangent lemma at , we can instantiate it at either or at , for any value of . In practice, if (resp. ) is a rational constant with a very large numerator or denominator, instead of instantiating a tangent lemma at , we instantiate two tangent lemmas at and .

5 From Satisfiability to Verification

We now move from satisfiability checking to verification.

5.0.1 Overview.

In principle, the solver described in the previous section could be integrated as a “black box” in any off-the-shelf SMT-based verification algorithm, such as BMC, k-induction, or one of the many extensions of IC3 to the SMT case (e.g. [9, 18, 21, 4]). In practice, however, such black-box integration would hardly be effective, especially in the case of state-of-the-art algorithms like IC3. IC3 requires a very incremental interaction with the underlying SMT engine, which is asked to solve a large number of relatively-cheap queries. The procedure of Sec. 4, however, can be very expensive, especially for satisfiable queries, which are very common in an IC3-like algorithm.222In fact, as already discussed in Sec. 4, the procedure is biased towards unsatisfiable instances, and might easily diverge on satisfiable ones. Moreover, some of the IC3 extensions mentioned above require the ability of performing (approximated) quantifier eliminations, a functionality not provided by the algorithm of Fig. 6.

We propose therefore a white-box integration, in which we lift the abstraction refinement approach of Sec. 4 at the transition system level. We generate an abstract LRA+EUF version of the input NRA transition system, which is then checked with the IC3-based procedure of [9]. In case a counterexample is produced, we use the SMT-NRA-check-abstract-ext algorithm of Fig. 6 to check whether it is spurious. If so, the axioms generated by SMT-NRA-check-abstract-ext are then used to refine the abstraction of the transition system. The pseudo-code of this algorithm is reported in Fig. 14. Similarly to the satisfiability checking case, the initial-abstraction function replaces every non-linear multiplication in the input transition system and property with a term, and adds some simple axioms about the behaviour of multiplication to the initial-state and transition-relation formulas of the transition system (see Sec. 4.0.2). In the rest of this section, we describe the abstraction refinement algorithm in more detail.

Figure 14: Verification of NRA transition systems via abstraction to LRA+EUF.

5.0.2 Counterexample Checking and Refinement.

Figure 15: Refinement of the LRA+EUF transition system.

When IC3-LRA+EUF-prove returns a counterexample trace for the abstract system , we use SMT-NRA-check-abstract-ext to check for its spuriousness. The function get-cex-formula builds a formula to feed to SMT-NRA-check-abstract-ext , whose unsatisfiability implies that is spurious. The formula is built by unrolling the transition relation of , and optionally adding constraints that restrict the allowed transitions to be compatible with the states in

. Various heuristics are possible, trading generality for complexity:

could be fully constrained by the states in (thus checking only one abstract counterexample path per iteration); it could be only partially constrained (e.g. by considering only the Boolean variables and/or the state variables occurring only in linear constraints); or it could be left unconstrained, considering only the length of the abstract counterexample. In our current implementation (see Sec. 6), we use the last option, i.e. we only consider the length of to build a BMC formula that checks for any counterexample of the given length, leaving the investigation of alternative strategies to future work.

If SMT-NRA-check-abstract-ext returns true, the property is violated. In this case, we can use the model found by SMT-NRA-check-abstract-ext to build a counterexample trace for the input system and property.

If SMT-NRA-check-abstract-ext returns false, we use the axioms produced during search to refine the transition system , using the procedure shown in Fig. 15. Essentially, refine-transition-system translates back the axioms from their unrolled version (i.e. on variables ) to their “single step” version (on variables and ), adding each of them either to the initial-states formula or to the transition relation formula. In case an axiom spans more than a single transition step (lines LABEL:code:vmt-refine-multi-step-beginLABEL:code:vmt-refine-multi-step-end of Fig. 15), we arbitrarily choose to map the variables with the lowest index as current state variables , and all the others as next-state variables . Notice that this might cause some refinement failure, as discussed in the next paragraph.

Reducing the number of axioms to add.

In general, not all the axioms generated during a call to SMT-NRA-check-abstract-ext are needed to successfully block a counterexample, especially if eager strategies like those described in Sec. 4 are used. In the long run, having a large number of redundant axioms can be quite harmful for performance. In order to mitigate this problem, we apply a filtering strategy (based on unsatisfiable cores) to the set of axioms, before adding them to the transition system. Instead of adding and directly to , we invoke the function shown in Fig. 16. Note that due to the flattening of multi-step axioms described above (lines LABEL:code:vmt-refine-multi-step-beginLABEL:code:vmt-refine-multi-step-end of Fig. 15), the refinement might fail. In this case, our current implementation simply aborts the execution.333We remark however that so far we have never observed this behaviour during our experiments.

Figure 16: Reducing the axioms needed for refinement.

6 Experimental Analysis

6.0.1 Implementation and comparisons.

We have implemented a prototype of the IC3-NRA-prove procedure using the IC3 engine of nuXmv [7] for IC3-LRA+EUF-prove . The code is written in Python, using the PySMT library [17]. Our implementation, benchmarks, and experimental data are available at https://es-static.fbk.eu/people/griggio/papers/tacas17-ic3-nra.tar.gz. We have used the following tools for our evaluation.

nuXmv-LRA-static: we apply the upfront abstraction of NRA to LRA proposed in [8], running the IC3 engine of nuXmv on the resulting transition system.

NRA-BMC-{z3, dReal } and NRA-K-induction-{z3, dReal }: we have implemented the BMC [3] and k-induction [29] algorithms in Python (using PySMT), using either z3 (NRA) or dReal (NRA) as back-end SMT solver.

iSAT3[1e-1] and iSAT3[1e-9]: we have used the latest version of the iSAT3 solver [24], which combines an SMT solver integrating CDCL and interval constraint propagation techniques with an interpolation-based abstaction/refinement algorithm for verification. iSAT3 supports both transition systems and software programs encoded as control flow graphs. Similarly to dReal, iSAT3 may return a “maybe unsafe” answer and provide a candidate solution identifying the upper and lower bounds on the variables. In the experiments, iSAT3[1e-1] is the configuration suggested by the iSAT3 authors 444 -I --use-craig-interpolation --use-cegar --cegar-abstraction-inductive --interpolant-rules-mcmillan --interpolant-a-biased --interpolation-offset --interpolant-offset 2 and iSAT3[1e-9] is the same except that the minimum splitting width (msw) parameter is set to . We have used a smaller value for the msw to get more precise answers, i.e. “safe” or “unsafe”, as suggested in the iSAT3 user manual.

6.0.2 Benchmarks.

We have collected a total of 114 NRA benchmarks from various sources.

Handcrafted. This set contains 14 hand-written instances, 13 safe and 1 unsafe.

HyComp. The second set contains 7 benchmarks (3 safe, 4 unsafe) which are taken from [11] and converted to NRA transition systems using HyComp [10].

HYST. This is the biggest set, consisting of 65 benchmarks. These are generated from the Hybrid examples that come with the HYST [2] distribution, by approximating the continuous time by sampling at a fixed time interval. This process is done automatically using an extended version of HYST. Since the generated benchmarks are approximations, we do not know their safety status. The benchmarks contain mostly non-linear behaviour.

iSAT3 and iSAT3-CFG. The 11 benchmarks in this set (7 safe, 4 unsafe) are taken from [24] and the iSAT3 examples available online.

nuXmv. In this set, we have 2 safe benchmarks which we collected from the nuXmv users’ mailing list. These benchmarks have complex boolean structure.

SAS13. These 13 benchmarks are generated from the C programs used in [5], but interpreted over NRA instead of the theory of IEEE floating-point numbers. This makes some of the instances unsafe.

TCM. We have generated 2 safe benchmarks from the Simulink models (taken from the case study [6]) by first generating the C code using the Embedded Coder555https://www.mathworks.com/products/embedded-coder/ and then encoding the program into a symbolic transition system.

6.0.3 Results.

Figure 17: Plots of SAFE and UNSAFE results.

Total

Handcrafted 

HyComp 

HYST 

ISAT3 

ISAT3-CFG

nuXmv

SAS13

TCM

(114) (14) (7) (65) (1) (10) (2) (13) (2)
IC3-NRA-prove 60/15 9/1 3/0 33/7 0/0 6/2 2/0 5/5 2/0
iSAT3[1e-1] 48/2(47) 2/0(8) 0/0(3) 34/2(23) 0/0 6/0(4) 0/0 4/0(9) 2/0
iSAT3[1e-9] 47/2(19) 2/0(3) 0/0(2) 32/2(3) 0/0 6/0(3) 0/0 5/0(8) 2/0
NRA-K-induction-Z3 22/25 2/1 0/2 12/15 0/0 6/2 0/0 0/5 2/0
nuXmv-LRA-static 37/0 4/0 1/0 19/0 0/0 4/0 2/0 5/0 2/0
NRA-BMC-Z3 0/26 0/1 0/2 0/15 0/0 0/3 0/0 0/5 0/0
NRA-K-induction-DREAL 16/0(32) 2/0(4) 0/0(2) 9/0(19) 0/0 5/0(2) 0/0 0/0(5) 0/0
NRA-BMC-DREAL 0/0(39) 0/0(8) 0/0(2) 0/0(19) 0/0 0/0(3) 0/0 0/0(7) 0/0
virtual-best 66/26 9/1 3/2 38/15 0/0 7/3 2/0 5/5 2/0

Each column shows a benchmark family, and each entry gives the number of safe/unsafe instances found. For tools working over interval arithmetic, the number of “maybe unsafe” is reported in parentheses.

Table 1: Summary of experimental results.
# Solved # Uniquely Solved Difference wrt. IC3-NRA-prove Gained Lost
IC3-NRA-prove 60/15 9/0 - - -
nuXmv-LRA-static 37/0 0/0 -38 1/0 24/15
iSAT3[1e-1] 48/2(47) 4/0 -25 4/0 16/13
iSAT3[1e-9] 47/2(19) -26 3/0 16/13
NRA-K-induction-Z3 22/25 0/11 -28 2/11 40/1
NRA-BMC-Z3 0/26 -49 0/11 60/0
NRA-K-induction-DREAL 16/2(32) 0/0 -59 2/0 46/15
NRA-BMC-DREAL 0/0(39) -75 0/0 60/15
virtual-best 66/26 - 17 6/11 0
Table 2: Comparitive summary of total solved benchmarks.

We ran our experiments on a cluster of machines with 2.67GHz Xeon X5650 CPUs and 96GB of RAM, running Scientific Linux 6.7. We used 6GB memory limit and 3600 seconds CPU timeout.

The results are summarized in Tables 1 and 2 and in Fig. 17. The plots show the time to solve an instance on the x-axis and the total number of solved instances on the y-axis. Table 1 reports a summary of the solved instances by family, whereas Table 2 shows a comparitive analysis by reporting for each tool the number of uniquely solved instances and the difference of solved instances w.r.t IC3-NRA-prove. We can make the following observations from the experimental results:

  • IC3-NRA-prove is the best performer overall, and it significantly outperforms all the other approaches on safe instances (where it can solve 9 problems that are out of reach for all the other tools). Interestingly, despite its simplicity, our model finding approach (as outlined in §4) is surprisingly effective, allowing IC3-NRA-prove to find 15 counterexample traces.

  • The simple abstraction proposed in [8] is quite effective for many families, allowing nuXmv-LRA-static to verify more properties than the approaches based on K-induction with an NRA solver. However, IC3-NRA-prove results in a clear and very significant improvement, solving more than twice as many instances than nuXmv-LRA-static (and losing only 1).

  • None of the other tools (with the exception of nuXmv-LRA-static) is able to solve any safe benchmark in the HyComp and nuXmv families. These benchmarks have a non-trivial Boolean structure and a significant linear component. Both IC3-NRA-prove and nuXmv-LRA-static are able to fully exploit the effectiveness of the underlying IC3 engine of nuXmv, outperforming the competitors. However, IC3-NRA-prove is very competitive also on the HYST family, whose instances are mostly non-linear and have very little Boolean structure.

  • Increasing the default precision of iSAT3

    significantly reduces the number of “maybe unsafe” answers, but it doesn’t seem to help in solving more benchmarks. In fact, we remark that even with the increased precision iSAT3[1e-9] classifies 2 safe instances as “maybe unsafe” (whereas in the default configuration, 6 safe instances are classified as “maybe unsafe”).

7 Conclusions and Future Work

We presented a novel abstraction-refinement approach to the verification of transition systems with nonlinear dynamics expressed in the NRA theory. We abstract non-linear multiplication as an uninterpreted function, leveraging efficient invariant checkers for transition systems over LRA and EUF to solve the problem in the abstract space. In case of spurious counterexample, the abstraction of multiplication is incrementally refined by introducing suitable axioms, based on the idea of tangent planes. An extensive experimental evaluation demonstrates that the proposed approach is significantly more effective than approaches directly based on SMT(NRA) solving.

This work opens up several important directions. First, we are going to improve the implementation, by integrating all the steps within the nuXmv [7] model checker, and to perform a thorough analysis of the various heuristic choices. Second, we will investigate the potential of the approach for SMT, both for other theories (e.g. NIA) and for extended functionalities (e.g. interpolation). We will also extend the scope of the approach to deal with transcendental functions, look-up tables, and partially axiomatized functions (e.g. gain functions known to be monotonic and of restricted co-domain).

Finally, we are going to investigate the generalization of the approach from transition systems to continuous-time hybrid systems with nonlinear characteristic functions.

Acknowledgement.

We greatly thank the iSAT3 team for providing the latest iSAT3 executable and iSAT3-CFG benchmarks. We also thank James Davenport for the fruitful discussions on CAD techniques and finding solutions in NRA.

References

  • [1] Erika Ábrahám, Florian Corzilius, Ulrich Loup, and Thomas Sturm. A lazy SMT-solver for a non-linear subset of real algebra. In Dagstuhl Seminar Proceedings. Schloss Dagstuhl-Leibniz-Zentrum f Ar Informatik, 2010.
  • [2] Stanley Bak, Sergiy Bogomolov, and Taylor T Johnson. HYST: a source transformation and translation tool for hybrid automaton models. In Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control, pages 128–133. ACM, 2015.
  • [3] Armin Biere, Alessandro Cimatti, Edmund M. Clarke, and Yunshan Zhu. Symbolic Model Checking Without BDDs. In TACAS, pages 193–207, London, UK, 1999. Springer-Verlag.
  • [4] Johannes Birgmeier, Aaron R. Bradley, and Georg Weissenbacher. Counterexample to induction-guided abstraction-refinement (CTIGAR). In CAV, volume 8559 of LNCS, pages 831–848. Springer, 2014.
  • [5] Martin Brain, Vijay D’Silva, Alberto Griggio, Leopold Haller, and Daniel Kroening. Interpolation-based verification of floating-point programs with abstract CDCL. In International Static Analysis Symposium, pages 412–432. Springer, 2013.
  • [6] Guillaume Brat, David Bushnell, Misty Davies, Dimitra Giannakopoulou, Falk Howar, and Temesghen Kahsai. Verifying the safety of a flight-critical system. In International Symposium on Formal Methods, pages 308–324. Springer, 2015.
  • [7] Roberto Cavada, Alessandro Cimatti, Michele Dorigatti, Alberto Griggio, Alessandro Mariotti, Andrea Micheli, Sergio Mover, Marco Roveri, and Stefano Tonetta. The nuXmv symbolic model checker. In CAV. Springer, 2014.
  • [8] Adrien Champion, Arie Gurfinkel, Temesghen Kahsai, and Cesare Tinelli. Cocospec: A mode-aware contract language for reactive systems. In Software Engineering and Formal Methods - 14th International Conference, SEFM 2016, Held as Part of STAF 2016, Vienna, Austria, July 4-8, 2016, Proceedings, pages 347–366, 2016.
  • [9] Alessandro Cimatti, Alberto Griggio, Sergio Mover, and Stefano Tonetta. IC3 modulo theories via implicit predicate abstraction. In TACAS, 2014.
  • [10] Alessandro Cimatti, Alberto Griggio, Sergio Mover, and Stefano Tonetta. HyComp: An SMT-based model checker for hybrid systems. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 52–67. Springer, 2015.
  • [11] Alessandro Cimatti, Sergio Mover, and Stefano Tonetta. A quantifier-free SMT encoding of non-linear hybrid automata. In Formal Methods in Computer-Aided Design (FMCAD), 2012, pages 187–195. IEEE, 2012.
  • [12] George E. Collins. Quantifier Elimination for Real Closed Fields by Cylindrical Algebraic Decomposition–preliminary Report. SIGSAM Bull., 8(3):80–90, August 1974.
  • [13] Leonardo De Moura and Nikolaj Bjørner. Z3: An efficient SMT solver. In International conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 337–340. Springer, 2008.
  • [14] Bruno Dutertre. Yices 2.2. In Armin Biere and Roderick Bloem, editors, Computer-Aided Verification (CAV’2014), volume 8559 of Lecture Notes in Computer Science, pages 737–744. Springer, July 2014.
  • [15] Niklas Eén and Niklas Sörensson. Temporal induction by incremental SAT solving. Electronic Notes in Theoretical Computer Science, 89(4):543–560, 2003.
  • [16] Sicun Gao, Soonho Kong, and Edmund M Clarke. dReal: An SMT solver for nonlinear theories over the reals. In International Conference on Automated Deduction, pages 208–214. Springer, 2013.
  • [17] Marco Gario and Andrea Micheli. pysmt: a solver-agnostic library for fast prototyping of SMT-based algorithms. In Proceedings of the 13th International Workshop on Satisfiability Modulo Theories (SMT), pages 373–384, 2015.
  • [18] Krystof Hoder and Nikolaj Bjørner. Generalized Property Directed Reachability. In SAT, volume 7317 of LNCS, pages 157–171. Springer, 2012.
  • [19] Richard M Hueschen. Development of the Transport Class Model (TCM) aircraft simulation from a sub-scale Generic Transport Model (GTM) simulation. Technical report, NASA Langley Research Center, 2011.
  • [20] Dejan Jovanović and Leonardo De Moura. Solving non-linear arithmetic. In

    International Joint Conference on Automated Reasoning

    , pages 339–354. Springer, 2012.
  • [21] Anvesh Komuravelli, Arie Gurfinkel, and Sagar Chaki. SMT-based model checking for recursive programs. Formal Methods in System Design, 48(3):175–205, 2016.
  • [22] Soonho Kong, Sicun Gao, Wei Chen, and Edmund Clarke. dreach: -reachability analysis for hybrid systems. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 200–205. Springer, 2015.
  • [23] Stefan Kupferschmid and Bernd Becker. Craig interpolation in the presence of non-linear constraints. In International Conference on Formal Modeling and Analysis of Timed Systems, pages 240–255. Springer, 2011.
  • [24] Ahmed Mahdi, Karsten Scheibler, Felix Neubauer, Martin Franzle, and Bernd Becker. Advancing software model checking beyond linear arithmetic theories. In Haifa Verification Conference, 2016. to appear, 2016.
  • [25] Alexandre Maréchal, Alexis Fouilhé, Tim King, David Monniaux, and Michael Périn. Polyhedral Approximation of Multivariate Polynomials Using Handelman’s Theorem. In International Conference on Verification, Model Checking, and Abstract Interpretation, pages 166–184. Springer, 2016.
  • [26] Kenneth L McMillan. Interpolation and SAT-based model checking. In Computer Aided Verification, pages 1–13. Springer, 2003.
  • [27] Pierluigi Nuzzo, Alberto Puggelli, Sanjit A Seshia, and Alberto Sangiovanni-Vincentelli. CalCS: SMT solving for non-linear convex constraints. In Proceedings of the 2010 Conference on Formal Methods in Computer-Aided Design, pages 71–80. FMCAD Inc, 2010.
  • [28] Karsten Scheibler, Stefan Kupferschmid, and Bernd Becker. Recent Improvements in the SMT Solver iSAT. MBMV, 13:231–241, 2013.
  • [29] Mary Sheeran, Satnam Singh, and Gunnar Stålmarck. Checking safety properties using induction and a SAT-solver. In Formal Methods in Computer-Aided Design, pages 127–144. Springer, 2000.
  • [30] Ashish Tiwari. Time-Aware Abstractions in HybridSal. In International Conference on Computer Aided Verification, pages 504–510. Springer, 2015.