Improving Convergence Rate Of IC3

09/03/2018 ∙ by Eugene Goldberg, et al. ∙ 0

IC3, a well-known model checker, proves a property of a state system ξ by building a sequence of formulas F_0,...,F_k. Formula F_i, 0 ≤ i ≤ k over-approximates the set of states reachable in at most i transitions. The basic algorithm of IC3 cannot guarantee that the value of k never exceeds the reachability diameter of ξ. We describe an algorithm called IC4 that gives such a guarantee. (IC4 stands for "IC3 + Improved Convergence"). One can also argue that the average convergence rate of IC4 is better than for IC3. This improvement is achieved at a cost that, in particular, involves generation of a small set of reachable states. We present the theory behind IC4 and describe techniques for improving its efficiency. Since we do not provide any experimental data yet, the comparison of performance of IC3 and IC4 is an open question. At the same time, we describe a few important applications that are unique to IC4 due to computation of reachable states.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

IC3 is a model checker [2] that has become very popular due to its high scalability. Let be a state transition system and be a safety property of . IC3 builds a sequence of formulas where over-approximates the set of states reachable from an initial state of in at most transitions. Property is proved when becomes an inductive invariant of for some .

One of the reasons for high performance of IC3 is that the value of above is typically much smaller than (i.e. the reachability diameter of ). So, on average, IC3 converges to an inductive invariant much faster than an RA-tool (where RA stands for “reachability analysis”). Interestingly, the worst case behavior of an RA-tool and IC3 is quite different from their average behavior. Namely, IC3 cannot guarantee that never exceeds . We introduce a modification of IC3 called IC4 that fixes the problem above. (IC4 stands for “IC3 + Improved Convergence”). On one hand, IC4 has the same worst case behavior as an RA-tool. On the other hand, the average convergence rate of IC4 is arguably better than that of IC3 as well.

The main difference between IC4 and IC3 is as follows. IC3 checks if formula is an inductive invariant by “pushing” the clauses of to . If every clause of can be pushed to , the former is an inductive invariant. Otherwise, there is at least one clause that cannot be pushed to . In this case, IC3 moves on re-trying to push to when new clauses are added to . In contrast to IC3, IC4 applies extra effort to push to . Namely, it derives new inductive clauses to exclude states that prevent from being pushed to . This extra effort results either in successfully pushing to or in proving that is “unpushable”.

The proof of unpushability consists of finding a reachable state that satisfies formula and falsifies clause . The existence of means that cannot be turned into an inductive invariant by adding more clauses. Thus, semantically, the difference between IC4 and IC3 is that the former starts building a new over-approximation only after it proved that adding one more time frame is mandatory. Operationally, IC4 and IC3 are different in that IC4 generates a small set of reachable states.

At the time of writing the first version of the paper we were not aware of QUIP, a version of IC3 published at [1]. We fix this omission and describe the relation between IC4 and QUIP in Subsection VII-A. QUIP more aggressively than the basic IC3 pushes clauses to future time frames and generates reachable states as a proof that a clause cannot be pushed. However, no relation of QUIP’s good performance with improvement of its convergence rate has been established either theoretically or experimentally.

The contribution of this paper is as follows. First, we show the reason why IC3 has a poor upper bound on the convergence rate (Section III). Second, we formulate a new version of IC3 called IC4 (Section IV) that is meant for fixing this problem. In particular, we show that IC4 indeed has a better upper bound than IC3 (Section V

). We also give an estimate of the number of reachable states

IC4 has to generate (Section VI). Third, we discuss arguments in favor of IC4 (Section VII).

Ii A Brief Overview Of Ic3

Let and be formulas111We assume that all formulas are propositional and are represented in CNF (conjunctive normal form) specifying the initial states and transition relation of a transition system respectively. Let be a formula specifying a safety property of . IC3 proves by building a set of formulas . Here formula , depends on the set of state variables of -th time frame (denoted as ) and over-approximates the set of states222A state is an assignment to the set of state variables. reachable in at most transitions. That is every state reachable in at most transitions is an -state 333Given a formula , a state is said to be an -state if ..

IC3 builds formula as follows. Formula is always equal to . Every formula , is originally set to . (So is always true because the only modification applied to is adding clauses.) Then IC3 tries to exclude every -state that is a predecessor of a bad state444Given a property , a -state is called a bad state. i.e. a state that breaks . Here is a short for and , as usual, means that depends on next-state variables i.e. those of . Exclusion of is done by derivation of a so-called inductive clause falsified by . Adding to excludes from consideration. (If cannot be excluded, IC3 generates a counterexample.)

One of the properties of formulas maintained by IC3 is . To guarantee this, IC3 maintains two stronger properties of : a) and b) implies that . That is the set of clauses of contains all the clauses of and the fact that contains at least one clause that is not in means that and are logically inequivalent. Since every formula implies , one cannot have more than - different formulas . That is if the value of exceeds -, there should be two formulas , , such that . This means that is an inductive invariant and property holds.

Iii Convergence Rate Of Ic3 And Clause Pushing

We will refer to the number of time frames one has to unroll before proving property as the convergence rate. We will refer to the latter as . As we mentioned in Section II, an upper bound on of the basic version of IC3 formulated in [2] is -. Importantly, the value of - can be much larger than (i.e. the reachability diameter of ). Of course, on average, of IC3 is much smaller than , let alone -. However, as we argue below, a poor upper bound on is actually a symptom of a problem.

Recall that formula specifies an over-approximation of the set of states reachable in at most transitions. So, it cannot exclude a state reachable in transitions where . (That is such a state cannot falsify .) On the other hand, may exclude states reachable in at least transitions or more.

Suppose IC3 just finished constructing formula . At this point holds i.e. no bad state can be reached from an -state in one transition. After constructing , IC3 invokes a procedure for pushing clauses from to . In particular, this procedure checks for every clause of if implication holds. We will refer to this implication as the pushing condition. If the pushing condition holds for clause , it can be pushed from to . If the pushing condition holds for every clause555In reality, since both and contain the clauses of , only the inductive clauses of added to strengthen are checked for the pushing condition. of , then and is an inductive invariant.

Suppose that the pushing condition does not hold for a clause of . Below, we describe two different reasons for the pushing condition to be broken. IC3 does not try to identify which of the reasons takes place. This feature of IC3 is the cause of its poor upper bound on . Moreover, intuitively, this feature should affect the average value of as well.

The first reason for breaking the pushing condition is that clause excludes a state that is reachable in -th time frame from an initial state. In this case, formula cannot be turned into an inductive invariant by adding more clauses. In particular, the broken pushing condition cannot be fixed for . The second reason for breaking the pushing condition is that clause excludes a state that is unreachable in -th time frame from an initial state. In this case, every -state that is a predecessor of can be excluded by deriving a clause falsified by . So in this case, the broken pushing condition can be fixed. In particular, by fixing broken pushing conditions for one may turn the latter into an inductive invariant.

Iv Introducing Ic4

Iv-a A high-level view of IC4

We will refer the version of IC3 with a better convergence rate described in this paper as IC4. The main difference between IC3 and IC4 is that the latter makes an extra effort in pushing clauses to later time frames. This new feature of IC4 is implemented in a procedure called NewPush (see Figure 1). It is invoked after IC4 has built where the predecessors of bad states are excluded i.e. as soon as holds. For every clause of , NewPush checks the pushing condition (see Section III). If this condition is broken, NewPush tries to fix it or proves that it cannot be fixed and hence is “unpushable”.

Depending on the clause-pushing effort, one can identify three different versions of IC4

: minimal, maximal and heuristic. The

minimal IC4 stops fixing pushing conditions as soon as NewPush finds a clause of that cannot be pushed. After that the minimal IC4 switches into the “IC3 mode” where the pushing conditions are not fixed for the remaining clauses of . The maximal IC4 tries to fix the pushing condition for every inductive clause of . That is if a clause cannot be pushed to , the maximal IC4 tries to fix the pushing condition (regardless of how many unpushable clauses of has been already identified). Moreover, if an inductive clause is added to , , the maximal IC4 try to fix the pushing condition for if it cannot be immediately pushed to .

A heuristic IC4 uses a heuristic to stay between minimal and maximal IC4 in terms of the clause-pushing effort. In this paper, we describe the minimal IC4 unless otherwise stated. So, when we just say IC4 we mean the minimal version of it.

// = ;
//
{
1 ;
2
3 while () {
4 ;
5 foreach {
6 if continue;
7 := ;
8 if () {
9
10 continue; }
11 ;
12 if () return();
13 }}
14 return(); }
Fig. 1: The procedure

Iv-B Description of NewPush

The pseudo-code of NewPush is given in Fig. 1. At this point IC4 has finished generation of . In particular, no bad state can be reached from an -state in one transition. NewPush tries to push every inductive clause of to . If a clause is unpushable, NewPush returns and a trace leading to a state falsified by clause . Trace proves the unpushability of and hence the fact that cannot be turned into an inductive invariant by adding more clauses. If every clause of can be pushed to , then is an inductive invariant and NewPush returns (nilnil) instead of clause and trace .

NewPush consists of two nested loops. A new iteration of the outer loop (lines 3-13) starts if variable NewClauses equals true. The value of this variable is set in the inner loop (lines 5-13) depending on whether new clauses are added to . In every iteration of the inner loop, NewPush checks the pushing condition (line 7) for an inductive clause of that is not in . If it holds, then is pushed to .

If the pushing condition fails, an -state is generated that falsifies clause . Then NewPush tries to check if is reachable exactly as IC3 does this when looking for a counterexample. The only difference is that is a good state666Recall that at this point of the algorithm, no bad state can be reached from an -state in one transition.. As we mentioned above, if is reachable by a trace , NewPush terminates returning and . Otherwise, it sets variable NewClauses to true and starts a new iteration of the inner loop.

V Better Convergence Rate of Ic4

As we mentioned in Section II, an upper bound on is -. Below, we show that using procedure NewPush described in Section IV brings the upper bound on for IC4 down to . (Note that if property holds, .)

Let be a formula for which NewPush is called when . At this point holds. Let be a state breaking the pushing condition for a clause of . That is falsifies (and hence it is not an -state) but is reachable from an -state in one transition.

Recall that is an over-approximation of the set of states that can be reached in at most -transitions. Since falsifies , reaching it from an initial state of requires at least transitions. However, this is impossible since and hence state is unreachable. This means that every -state that is a predecessor of can be excluded by an inductive clause added to . So eventually, NewPush will fix the pushing condition for . After fixing all broken pushing conditions for clauses of , NewPush will turn into an inductive invariant.

Vi Number Of Reachable States To Generate

The number of generated reachable states depends on which of the three versions of IC4 is considered (see Subsection IV-A). Let denote the maximal number of time frames unfolded by IC4. In the case of the minimal IC4, the upper bound on the number of reachable states for proving property is equal777For every formula , , IC4 generates one reachable state falsifying a clause of . To reach , one needs to generate a trace of states. So the number of reachable states generated for is equal to . The total number of reachable states is equal to . to . For the maximal IC4, the upper bound is where and is the subset of consisting of unpushable clauses. Indeed, an inductive clause is proved unpushable only once. This proof consists of a trace to a state falsified by . The length of this trace is equal to and hence bounded by . The upper bound for the maximal IC4 above is loose because one assumes that

  • the length of every trace proving unpushability equals

  • two (or more) clauses cannot be proved unpushable by the same reachable state.

Re-using reachable states can dramatically reduce the total number of reachable states one needs to generate. For instance, for the minimal IC4, this number can drop as low as . For the maximal IC4, the total number of reachable states can go as low as where is the total number of reachable states generated to prove the unpushability of clauses of .

Vii A Few Arguments In Favor Of Ic4

In this section we give some arguments in favor of IC4. The main argument is given in Subsection VII-A where we relate IC4 with a model checker called QUIP. The latter was introduced888As we mentioned in the introduction, at the time of writing the first version of our paper we were not aware of QUIP. in [1] in 2015. In Subsections VII-B and VII-C we describe potential advantages of IC4 that exploit generation of reachable states and are not discussed in [1].

Vii-a IC4 and QUIP

As we mentioned in the introduction, QUIP makes an extra effort to push clauses to future time frames. To show that a clause cannot be pushed, QUIP generates a reachable state. Although the premise of QUIP is that the strategy above may lead to a faster generation of an inductive invariant, this claim has not been justified theoretically. The advantage of QUIP over IC3 is shown in [1] in terms of better run times and a greater number of solved problems. So, no direct experimental data is provided on whether QUIP has a better convergence rate than IC3. (As mentioned in [1] and in the first version of our paper, having at one’s disposal reachable states facilitates construction of better inductive clauses999By avoiding the exclusion of known reachable states, one increases the chance for an inductive clause to be a part of an inductive invariant.. So one cannot totally discard the possibility that the performance of QUIP is mainly influenced by this “side effect”.) Nevertheless, great experimental results of QUIP is an encouraging sign.

Vii-B Multi-property verification

Suppose one has to verify multiple properties of the same design. That is one needs to check a set of safety properties of the same transition system . In [5], we used IC3 to make a case for separate verification where properties are checked individually101010Alternatively, one can verify jointly by checking if property holds.. Separate verification benefits a lot from re-using inductive invariants of already proved properties when proving a new property. IC4 adds one more important piece of information to share: reachable states. The fact that a state is reachable in transitions is property-independent. So it can be used to derive better inductive clauses (see the previous subsection) for over-approximations , when proving a new property.

Vii-C Test generation

Formal verification of some properties of state system does not guarantee that the latter is correct111111Moreover, can be incorrect even if a supposedly complete set of properties is proved true [4, 3]. For instance, the designer may “misdefine” a property and so instead of verifying the right property (that does not hold) a formal tool checks a weaker property (that holds). . In this case, testing is employed to get more confidence in correctness of . Traces generated by IC4 can be used as tests in two scenarios. First, one can check that reachable states found by IC4 satisfy the properties that formal verification tools failed to prove. Second, one can just inspect the states visited by and the outputs produced in those states to check if they satisfy some (formal or informal) criteria of correctness.

References

  • [1] A.Ivrii and A.Gurfinkel. Pushing to the top. FMCAD-15, pages 65–72, 2015.
  • [2] A. R. Bradley. Sat-based model checking without unrolling. In VMCAI, pages 70–87, 2011.
  • [3] E. Goldberg. Complete test sets and their approximations. In FMCAD-18. To be published.
  • [4] E. Goldberg. Complete test sets and their approximations. Technical Report arXiv:1808.05750 [cs.LO], 2018.
  • [5] E. Goldberg, M. Güdemann, D. Kroening, and R. Mukherjee. Efficient verification of multi-property designs (the benefit of wrong assumptions). In DATE ’18, pages 43–48, Dresden, Germany, 2018.