Deciding and Interpolating Algebraic Data Types by Reduction (Technical Report)

01/08/2018
by   Hossein Hojjat, et al.
0

Recursive algebraic data types (term algebras, ADTs) are one of the most well-studied theories in logic, and find application in contexts including functional programming, modelling languages, proof assistants, and verification. At this point, several state-of-the-art theorem provers and SMT solvers include tailor-made decision procedures for ADTs, and version 2.6 of the SMT-LIB standard includes support for ADTs. We study an extremely simple approach to decide satisfiability of ADT constraints, the reduction of ADT constraints to equisatisfiable constraints over uninterpreted functions (EUF) and linear integer arithmetic (LIA). We show that the reduction approach gives rise to both decision and Craig interpolation procedures in (extensions of) ADTs.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

02/07/2012

Optimization in SMT with LA(Q) Cost Functions

In the contexts of automated reasoning and formal verification, importan...
08/26/2019

CLS-SMT: Bringing Together Combinatory Logic Synthesis and Satisfiability Modulo Theories

We introduce an approach that aims to combine the usage of satisfiabilit...
04/09/2020

Politeness for the Theory of Algebraic Datatypes

Algebraic datatypes, and among them lists and trees, have attracted a lo...
03/17/2018

Meta-F*: Metaprogramming and Tactics in an Effectful Program Verifier

Verification tools for effectful programming languages often rely on aut...
10/12/2021

Toward SMT-Based Refinement Types in Agda

Dependent types offer great versatility and power, but developing proofs...
04/26/2020

Monadic Decomposition in Integer Linear Arithmetic (Technical Report)

Monadic decomposability is a notion of variable independence, which asks...
10/10/2018

Smtlink 2.0

Smtlink is an extension of ACL2 with Satisfiability Modulo Theories (SMT...

I Introduction

Recursive algebraic data types (ADTs) with absolutely free constructors are increasingly supported by SMT solvers, and find application in a variety of areas, including functional programming, modelling languages, proof assistants, and verification. In solvers, ADTs are usually implemented as native theory solvers [12, 2, 17, 13] that apply congruence closure (upward closure), syntactic unification (downward closure), cycle detection (occurs-check), and in additional handle selectors and testers in order to decide satisfiability of quantifier-free ADT formulas.

In this paper, we study a simple alternative approach to ADT reasoning, based on the reduction of ADT formulas to equisatisfiable formulas over uninterpreted functions and linear integer arithmetic (EUF+LIA). Our approach is partly inspired, besides by eager SMT in general, by the reduction approach from [10], in which quantifier-free formulas are mapped to simpler theories for the purpose of checking satisfiability and computing interpolants. For instance, as shown in [10], the theory of sets with finite cardinality constraints can be reduced to the theory of equality with uninterpreted functions (EUF). Like in [10], the target theories of our ADT reduction are EUF and linear arithmetic. Unlike [10], we are able to completely avoid universal quantifiers in the process of reduction, but the reduction depends on the introduction of further uninterpreted functions (which create some additional work in interpolation, see Section V).

The main idea of reduction is to augment an ADT formula with additional literals that ensure that constructors, selectors, and testers are interpreted consistently, and that constructors are free. EUF takes care of upward and downward closure, while cycle detection and constructor testers are handled by LIA constraints. The reduction can be implemented with little effort, and is widely applicable since EUF and LIA are supported by virtually all SMT solvers, and increasingly also by other theorem provers. Reduction to EUF+LIA has a few further advantages, in particular it is possible to reuse existing, highly optimised EUF+LIA simplifiers in solvers, and to compute interpolants using EUF+LIA interpolation procedures.

The contributions of the paper are (i) definition and correctness proof of the reduction from ADTs to EUF+LIA; (ii) discussion of Craig interpolation for ADTs; (iii) extension to ADTs with size constraints, and an effective characterisation of the ADTs for which the resulting procedure is complete. The procedures discussed in the paper have been implemented in the Princess theorem prover [14].111http://www.philipp.ruemmer.org/princess.shtml

I-a Related Work

ADT Solving

While ADTs have only recently been standardised in the SMT-LIB, some solvers (including STeP [11], CVC3 [3], CVC4 [1], and Z3 [7]) have for a while supported ADTs through native decision procedures extending the congruence closure algorithm [12, 2, 13]. Native solvers offer excellent performance, but also require significant implementation effort. The listed solvers do not support Craig interpolation or formulas with size constraints.

Satisfiability of ADT formulas can also be checked by introducing explicit axioms about the constructors and selectors. Since ADTs form a local theory [16], the set of required instances of the axioms can effectively be computed, and a decision procedure for ADT satisfiability is obtained.

Our reduction-based approach sits in between native solvers and methods based on explicit axioms. Like with explicit axioms, our method leaves most of the heavy work to other theory solvers (EUF and LIA), and is therefore easy to implement. The reduction approach is structure-preserving, however, which makes us believe that it can utilise existing contextual simplifiers (pre-processors or in-processors) more effectively than approaches based on axioms; it also directly gives rise to an interpolation procedure.

ADT Interpolation

It has been observed in [10] that the theory of ADTs has the interpolation property; this result directly follows from admissibility of quantifier elimination in ADTs [12]. To the best of our knowledge, our ADT solver implemented in Princess is the first proof-based interpolation procedure for ADTs.

ADTs with Size Constraints

Our approach for handling ADT formulas with size constraints is inspired by the more general unfolding-based decision procedure for ADTs with abstractions (i.e., catamorphisms) in [17]. The algorithm in [17] is complete for sufficiently surjective abstraction functions, which includes the size function on binary trees, but not the size function on ADTs in general. We augment the setting from [17] by giving a necessary and sufficient criterion for sufficient surjectivity of the size function, and thus for completeness of the overall procedure.

ADTs with size constraints can also be represented in the local theory framework [16], again by introducing the necessary instances of explicit axioms.

A further decision procedure for ADTs with size constraints, based on the concept of length constraint completion, has been described in [19]. Our method uses the simple approach of unfolding in order to add size constraints to the overall reduction-based procedure; it is at this point unclear whether length constraint completion could be combined with the reduction approach as well.

Ii Preliminaries

We formulate our approach in the setting of multi-sorted first-order logic. The signature  of an ADT is defined by a sequence  of sorts and a sequence  of constructors. The type  of an -ary constructor is an -tuple , normally written in the form . Zero-ary constructors are also called constants. By slight abuse of notation, we also write if the result type of is , i.e., if for some .

In addition to constructors, formulas over ADTs can be formulated in terms of variables  (with some type ); selectors , which extract the argument of an -term; and testers , which determine whether a term is an -term. The syntactic categories of terms  and formulas  are defined by the following rules:

Variables
Constructors
Selectors
Testers
Equality
Boolean operators

Well-typed terms and formulas are defined as expected, assuming that selectors have type whenever , and testers  expect an argument of type  if . In the whole paper, we assume that considered expressions are well-typed.

Example 1 (Lists)

We show examples in the concrete syntax used in our implementation.

1\sorts {
2  Colour { red; green; blue; };
3  CList { nil; cons(Colour head, CList tail); };
4}

Given variables  of sort CList and of sort Colour, a formula over this data type is:222In all examples, the link will take you to the web interface of our SMT solver Princess and directly load the given constraint.

1                                                                                                                                                                x.is_cons & y != blue &
2  (x.head = red | x = cons(y, nil))

corresponding to the abstract syntax formula

Assigning satisfies the formula. ∎

A constructor term is a ground term  that only consists of constructors (i.e., does not contain selectors or variables). We denote the set of all constructor terms (for some fixed ADT signature) by , and the set of all constructor terms of type  by . An ADT is well-defined if is non-empty for all sorts , and we will henceforth only consider well-defined ADTs.

Semantics is defined in terms of structures  over the universe  of constructor terms, i.e., constructors are absolutely free. Selectors  in particular are mapped to total set-theoretic functions satisfying .

Iii A Verification Example

As a high-level example, we outline how a simple program operating on the ADT from Example 1 can be verified using our procedures. We represent the program in the form of (constrained) Horn clauses, following the approach taken in several recent verification systems [8, 15]

. The result resembles a classical logic program implementing the concatenation of two lists;

C(x, y, r) expresses that r is the result of concatenating lists x, y:

1\forall CList y;                         // (C1)
2  C(nil, y, y)
3\forall CList x, y, r; \forall Colour c; // (C2)
4  (C(x, y, r) -> C(cons(c, x), y, cons(c, r)))

As a first property of the program, we can observe that the head of a non-empty result list r has to be the head of one of the arguments x, y:

1\forall CList x, y, r; (                 // (P1)
2  r != nil & C(x, y, r) ->
3  (r.head = x.head | r.head = y.head))

To verify this property, it is enough to find a model of the (constrained) Horn clauses (C1), (C2), (P1), i.e., an interpretation of the predicate C that satisfies all three formulas. The predicate C can then be considered as a post-condition (or inductive invariant) that is sufficient to show property (P1). One solution of (C1), (C2), (P1) is to interpret C(x, y, r) as

1                                                                                                                                                              C(CList x, CList y, CList r) {
2  r = y | r.head = x.head
3}

which can indeed be observed to satisfy all three clauses. The decision procedure for ADTs defined in the next section can easilycheck correctness of this model mechanically, after inlining the definition of C, and skolemising away quantifiers.

To find models of clauses like (C1), (C2), (P1) automatically, the principle of Craig interpolation can be applied to derivation trees of the clauses, an approach that has been implemented in several model checkers [8, 15]. To support ADTs, which are currently beyond the scope of most model checkers, in Section V we explain how our decision procedure can be extended to Craig interpolation.

Consider now additional clauses computing the list length:

1L(nil, 0)                                // (C3)
2\forall CList x;                         // (C4)
3  \forall Colour c; \forall int n;
4    (L(x, n) -> L(cons(c, x), n+1))

We can combine the two programs to state a second property relating concatenation and list length. Concatenating two lists yields a list whose length is the sum of the individual list lengths:

1\forall CList x, y, r;                   // (P2)
2  \forall int nx, ny, nr; (
3    C(x, y, r) & L(x, nx) & L(y, ny) & L(r, nr)
4    -> nr = nx + ny)

To verify this property, as before by showing the existence of a model of (C1), (C2), (C3), (C4), (P2), we need a slightly extended logic providing also an operator for the size of ADT terms (Section VI). ADT constraints without size operator are not sufficiently expressive to formulate any model. The size of a term  is the number of constructor occurrences in . A model of (C1), (C2), (C3), (C4), (P2), interpreting both the predicate C and L, is then

1                                                                                                                                                              C(CList x, CList y, CList r) {
2  \size(x) + \size(y) = \size(r) + 1
3};
4L(CList x, int n) {
5  \size(x) = 2*n + 1
6};

Note that the \size operator also counts the nil symbol, as well as the colour constructors red, green, blue, leading to the stated relationship between the size and the length of a list. The correctness of the model can be checked using the procedure we define in Section VI.

Reduction rules for constructors , selectors , testers , and equations between variables:

(1) (2) (3) (4) (5) (6)
The following abbreviations are used, for each constructor and each sort :

TABLE I: Rules for reduction of ADTs to EUF+LIA

Iv Checking ADT Satisfiability by Reduction

We now define our reduction from ADTs to EUF+LIA. Suppose is an ADT formula as defined in Section II. For sake of presentation, we assume that has been brought into a flat form upfront. A formula  is flat if function symbols (in our case, constructors and selectors) only occur in equations of the form  (where are variables, though not necessarily pairwise distinct), and only in positive positions. Flatness can be established at the cost of introducing a linear number of additional variables.

Example 2

The formula in Example 1 can be flattened by introducing variables , and :

1                                                                                                                                                                x.is_cons & blue = t1 & y != t1 &
2  ((red = t2 & x.head = t2) |
3   (nil = t3 & cons(y, t3) = x))

Notation

We need some further notation before we can formally define the reduction. As before, we assume that sorts and constructors have been fixed. For each sort , we define to be the number of constructors of :

Similarly, each constructor  with is given a unique index  as identifier within its sort :

For each sort , we furthermore need to know the cardinality  of the term domain . The cardinality can be derived by computing the strongly connected components of the dependency graph induced by the constructors (the graph with sorts as nodes, and edges whenever there is a constructor with a -sorted argument and result sort ). We write for infinite domains.

Iv-a Definition of the Reduction

Suppose is a flat formula in negation normal form (NNF) over an ADT as defined in Section II. To translate to an EUF+LIA formula , we introduce a new set of function symbols ranging over integers: for each constructor  a new function  with the same arity ; for each selector  a unary function ; for each sort  a function symbol  to encode testers, and a function  to ensure acyclicity of terms. Further, for each variable  occurring in , we introduce an integer-valued variant .

The actual reduction is defined through the rewriting rules in the upper half of Table I. Since the reduction works differently for positive and negative occurrences of literals, we assume that rules are only applied in positive positions, and handle negation explicitly in the rules (and assume that is in negation normal form). Rule (1) augments every occurrence of a constructor symbol with corresponding statements about selectors (ensuring that both are inverses of each other); about the index of the constructor (ensuring that different constructors of the same sort produce distinct values); and about the depth of the constructed term (ensuring that no term can occur as sub-term of itself). Essentially the same translation is done for testers by rule (3), introducing fresh constructor arguments through an existential quantifier. Rule (2) augments each occurrence of a selector with a disjunction stating that the considered term was actually created using one of the constructors of the sort; this is necessary in general since selectors  can be applied to terms constructed using constructors other that (an optimisation is discussed in Section IV-C). Rule (4) asserts that the constructor of a term is different from , and (5), (6) translate equations by simply renaming variables.

Suppose is the result of exhaustively applying the rules at positive positions in , and are all variables occurring in , then the reduct of is defined as .

Example 3

In the encoded version of the formula from Example 2, all variables and functions range over integers; for readability, we keep the names of all variables. New variables  are introduced to eliminate the quantifiers of expressions through Skolemisation:

1                                                                                                                                                                // encoding of x.is_cons
2  cons(s1, s2) = x & ctorId_CList(x) = 1 &
3  head(x) = s1 & tail(x) = s2 & 0<=s1 & s1<3 &
4  depth_CList(x) > depth_Colour(s1) &
5  depth_CList(x) > depth_CList(s2) &
6  // encoding of blue = t1
7  blue = t1 & ctorId_Colour(t1) = 2 &
8  // encoding of y != t1 (unchanged)
9  y != t1 &
10    // encoding of red = t2
11  ((red = t2 & ctorId_Colour(t2) = 0 &
12    // encoding of x.head = t2
13    head(x) = t2 & (
14      // case x.is_nil
15      (nil = x & ctorId_CList(x) = 0) |
16      // case x.is_cons
17      (cons(s3, s4) = x & ctorId_CList(x) = 1 &
18       head(x) = s3 & tail(x) = s4 &
19       0 <= s3 & s3 < 3 &
20       depth_CList(x) > depth_Colour(s3) &
21       depth_CList(x) > depth_CList(s4)))) |
22    // encoding of nil = t3
23   (nil = t3 & ctorId_CList(t3) = 0 &
24    // encoding of cons(y, t3) = x
25    cons(y, t3) = x & ctorId_CList(x) = 1 &
26    head(x) = y & tail(x) = t3 &
27    depth_CList(x) > depth_Colour(y) &
28    depth_CList(x) > depth_CList(t3))) &
29  // range constraints for x, y, t1, t2, t3
30  // (some of which are just "true")
31  0<=y & y<3 & 0<=t1 & t1<3 & 0<=t2 & t2<3

It should be noted that it is not necessary to assume positiveness of the functions, since the functions are only used to ensure acyclicity of terms by comparing the depth of a term with the depths of its direct sub-terms. In general, although the formula makes use of integer arithmetic, only very simple arithmetic constraints are needed. Up to slight syntactic modifications, all constraints fall into the Unit-Two-Variable-Per-Inequality fragment UTVPI [9, 6], i.e., only inequalities with up to two variables and unit coefficients are needed. The constraints can therefore be both solved and interpolated efficiently (of course, presence of Boolean structure or negation still implies NP-hardness).

Iv-B Correctness of Reduction

Theorem 1

The reduct  of a flat ADT formula  in NNF is satisfiable (over EUF+LIA) if and only if is satisfiable (over an ADT).

Proof:

Since reduction preserves the Boolean structure of a formula, and the reduction rules are agnostic of the position at which they are applied, it is enough to prove the theorem for flat conjunctions of literals (i.e., formulas in negation normal form that do not contain disjunctions).

” (easy direction) Suppose is satisfiable, with structure  and variable assignment . We construct a family  of injective functions as embedding of the domains  into . For such that is infinite, can be any bijection ; if is finite, we choose to be a bijection . Let . To satisfy , choose variable assignment , and the interpretation  of constructors and selectors over that is induced by . Define to be the depth of the constructor term , and as the index  of the head symbol  of (and arbitrary if is undefined).

” Suppose is satisfiable, with structure  and assignment . We construct a set  of relevant integer indices  and corresponding sorts  in the model, and a mapping  (with for each ) that can be used to define a variable assignment  to satisfy . The main difficulty is to ensure that is injective, since otherwise disequalities in might be violated.

We set , where is the set of pairs  for variables  for which contains a constructor literal , a selector literal , or a (possibly negated) tester . The encoding ensures that head symbols and children of terms represented by elements of are defined by the functions and the selectors; for , define therefore if , with , and for .

Let contain all pairs  in tuples , for any ; as well as pairs  for any further variable  in .

We inductively define a sequence  of partial functions :

  1. let ;

  2. for , if there is such that , the function  is defined for each pair  (for , but is not defined for , then let .

  3. for , if case 2) does not apply, pick any pair  for which is not defined, and any constructor term  that does not occur in the range of yet; choose and such that the depth of becomes minimal. Let .

Importantly, the final function  is defined for all elements of , and no two elements of are mapped to the same term. To see that is defined for all elements of , observe that the use of functions in the encoding ensures that the function is acyclic, i.e., no term can ever be required to contain itself as a sub-term. To see that is injective, observe that by definition the choice of in 3) cannot violate injectivity in . Different iterations of 2) cannot construct a term twice, due to the presence of constructor literals  in that are consistently interpreted. Finally, the fact that case 2) is always preferred over 3) implies that the term has to contain the most recently chosen term  from case 3) (if there is any) as a sub-term; this implies that is deeper than all terms  previously chosen in case 3), and therefore different from all of them.

It is then possible to choose the variable assignment for each variable  in .

Iv-C Two Optimisations

The reduction, as presented so far, can be improved in a number of ways. A first optimisation concerns the way selectors are translated to EUF+LIA, rule (2). It can be observed that the disjunction of literals introduced by rule (2) is in most cases unnecessary, and usually the rule can be simplified to

(2’)

This simplification is possible whenever rule (2) is applied to guarded selector literals, i.e., whenever occurs in conjunction with a (positive or negative) test , or in conjunction with a constructor literal  (in both cases, regardless of whether ).

Example 4

The effect of this redundancy can be seen in Example 3: given lines 1–5, the disjunction in 14–21 can be simplified to s3= s1 & s4 = s2, and can be removed entirely since s3 and s4 do not occur elsewhere in the formula. ∎

Example 5

The full rule (2) is necessary for the following formula over the ADT in Example 1:

1                                                                                                                                                                x = cons(x.head, x.tail) <-> x.is_cons

This is because x.head and x.tail occur unguarded. ∎

As a second optimisation, the treatment of sorts with finite domain can be improved, in particular for sorts that are enumerations (i.e., sorts with only nullary constructors). The full EUF encoding is overkill for enumerations, since instead we can map each constructor  directly to the index :

(1’)

Similarly, testers in enumerations reduce to simple arithmetic comparisons.

Iv-D Size Increase Caused by the Reduction

The reduction rules replace every literal in a formula  with an expression that is linear in the size  of the considered ADT, so that . If the ADT is considered as fixed, the reduction is linear.

As an experimental evaluation of the size increase, we applied the procedure (including the optimisations from the previous section) to the 8000 randomly generated ADT benchmarks from [2] (4422 of the benchmarks are unsat). The benchmarks themselves are not very challenging, with the most complicated one solved in around 1 s, and the average solving time of 43 ms dominated by parsing, pre-processing, etc. The average problem sizes, counted as the number of sub-expressions of each formula, were:

After parsing After reduction After red. & simpl.
76 337 34

This means that reduction led to an increase in size by a factor of 4.5, but this increase was more than offset by subsequent simplification (using the standard EUF+LIA simplifier implemented in Princess). Analysing further, it turned out that reduction followed by simplification was extremely effective on the unsatisfiable benchmarks: of the 4422 unsatisfiable problems, 4334 could directly be simplified to . The average size of the remaining 3666 problems, after reduction and simplification, was 74, incidentally the same as the average initial size of all benchmarks.

An experimental comparison of our solver with other SMT solvers, on a larger set of benchmarks, is ongoing.

V Craig Interpolation in (Extensions of) ADTs

Since quantifier-free Craig interpolation in EUF+LIA is well understood (e.g., [4, 5, 6]), the reduction approach can also be leveraged for interpolation. Given an unsatisfiable conjunction , the problem of (reverse) interpolation is to find a formula  such that , , and all variables in are common to and . If are ADT formulas, it is natural to approach interpolation by first computing an EUF+LIA interpolant  for the reduced conjunction .

Example 6

An interpolation problem over the list ADT from Example 1 is:

1                                                                                                                                                                \part[left]  (x.is_cons & x.tail = z &
2                z.is_cons & x.head != z.head)
3& \part[right] (x = cons(c, cons(c, y)))

The only common variable of the two formulas is x, and a solution of the interpolation problem is the disequality x.head != x.tail.head. Note that this formula is a correct interpolant even though the selectors are unguarded. ∎

To translate  back to an ADT interpolant , three main points have to be addressed. First, since all ADT sorts are translated to integers, the formula  might contain arithmetic operations on ADT terms that cannot easily be mapped back to the ADT world. This turns out to be a non-issue for infinite ADT sorts, since reduction does not make use of arithmetic operations for terms over infinite sorts (indeed, equivalently every infinite ADT sort  could be mapped to a fresh uninterpreted sort ). The situation is different for finite sorts, where predicates  from Table I represent cardinality constraints that can contribute to unsatisfiability of a formula. One solution is the optimisation discussed in Section IV-C: by defining a fixed mapping of terms in finite domains to integers, translation of interpolants back to ADT formulas is significantly simplified.333In our implementation, such fixed mapping is currently only done for enumerations, not for other finite ADT sorts.

Second, the functions  introduced by the reduction are not valid ADT operations, and have to be translated back to testers (which can be done quite easily).

Third, interpolants might also mention operations, which have no direct correspondence in the original ADTs theory. Instead of devising ways how to eliminate such operations, we decide to embrace them instead as a useful extension of ADTs, and adapt our reduction method accordingly. Since depth is but one measure that can be used to ensure acyclicity, the next sections therefore discuss how we can reason about ADTs with size constraints.

Vi Solving ADTs with Size Constraints

We now consider ADT formulas extended with constraints about term size. The size  of a term  is the number of constructor occurrences in . The resulting formal language is an extension of the language defined in Section II:

Size constraints

where is any Presburger formula about the size of ADT terms .

Example 7

Consider the ADT in Example 1, and a variable of sort CList. The formula

1                                                                                                                                                                \size(x) = 3 & x.head = blue

has the satisfying assignment , and this assignment is unique. In contrast, the formula

1                                                                                                                                                                \size(x) % 2 = 

is unsatisfiable, since the size of any list term is odd (term size does not exactly coincide with the length of a list). ∎

To extend our reduction approach to formulas with size constraints, there are two main issues that have to be addressed: (i) constructor terms might not exist for all sizes , and (ii) even if terms of some size  exist, there might be too few of them to satisfy a formula.

Example 8

Consider the ADT of positive natural numbers:

1\sorts {
2  Nat { one; succ(Nat pred); };
3}

For every size  there is exactly one constructor term  with . This implies unsat. of the formula

1                                                                                                                                                                \size(x) = 3 & \size(y) = 3 & x != y

Vi-a Reduction and Incremental Unfolding

Additional reduction rule for size expressions, assuming is a variable of sort , and a variable of sort :

(7)
Compared to Table I, for each constructor the abbreviation is replaced with :

TABLE II: Additional rules for reduction of ADTs with size constraints to EUF+LIA

We address issue (i) noted above by reasoning globally about possible term sizes within an ADT. For an ADT sort , we define to be the size image of the term set , i.e., the set of term sizes in . The size image turns out to be a special case of the Parikh image of a context-free language, since an ADT can be interpreted as a context-free grammar over a singleton alphabet (by considering every sort as a non-terminal symbol, and mapping every constructor to the unique letter in the singleton alphabet). This implies that is semi-linear, and that a representation of the set in the form of an existential Presburger formula can be derived from the set of constructors in linear time [18].

Table II shows how the reduction from Section IV-A (and Table I) is augmented to deal with size constraints. Instead of the functions, for each sort  a function  representing term size is introduced, and the constraints are changed accordingly; and an additional reduction rule (7) is introduced to handle equations  with the size operation. Rule (7) adds constraints  to ensure that only genuine term sizes are considered, assuming implicitly that the size image  is represented as a Presburger formula.

The resulting modified reduction approach is sound for checking unsatisfiability of ADT formulas:

Lemma 1

If the reduct  of a flat ADT formula  in NNF with size constraints is unsatisfiable, then is unsatisfiable.

Reduction does not directly give rise to a decision procedure for ADT constraints with size constraints, in contrast to the situation without size. This is because reduction does not precisely translate the number of terms for each size  (issue (ii) from above). We can observe that the reduct  of the formula  in Example 8 is satisfiable, while is unsatisfiable, showing that reduction alone is not sound for satisfiability (unsurprisingly).

Different approaches exist to establish soundness also for satisfiability, in particular the extraction of length constraint completion formulas [19] that precisely define term sizes with sufficiently many distinct terms. We follow the approach of incrementally unfolding (aka. unrolling) from [17], which is quite flexible, and complete in many relevant cases.

Let again be a (flat and NNF) ADT formula with size constraints. We construct unfolding sequences  by setting , and for each deriving by unfolding one ADT variable  that occurs in . If are all constructors of the considered ADT, we set

with fresh sorted argument variables .

In practice, unfolding will usually happen incrementally: the next variable to unfold is selected based on a model of the previous partial unfolding , until enough terms have been constructed to obtain a genuine model of , or unsatisfiability is detected.

Lemma 2

Let be an unfolding sequence for , and for each let be the set of variables unfolded in (i.e., , and if was derived by unfolding in ). Then for any :

  1. if is unsatisfiable (over EUF+LIA) then is unsatisfiable (over ADTs with size);

  2. if is satisfied by a model  and assignment , such that for every ADT variable  in there is a variable  with and , then is satisfiable (over ADTs with size).

Proof:

1) follows directly from Lemma 1.

2) Models over EUF+LIA can be translated to ADT models like in the proof of Theorem 1”. It can be noted that case 3) in the proof never applies due to the assumption that all variables are mapped to unfolded terms.

Example 9

In Example 8, unsatisfiability is detected after unfolding and three times each. ∎

As the next example shows, however, unfolding is not always enough to show unsatisfiability of a formula. The next sections will therefore formulate a sufficient and necessary criterion for termination of unfolding.

Example 10

With the ADT from Example 8, the formula

1                                                                                                                                                                \size(x) = \size(y) & x != y

is unsatisfiable, but cannot be shown to be unsatisfiable with a finite number of unfolding steps. ∎

Vi-B Completeness and Incompleteness of Unfolding

We give a precise characterisation of the ADTs for which unfolding will allow us to eventually detect (un)satisfiable of a formula, and therefore gives rise to a decision procedure. As identified in[17], the essential property of an ADT (resp., of the considered catamorphism, which in our case is the size function) is sufficient surjectivity, implying that ADTs are sufficiently populated to satisfy disequalities in a formula: the number of terms of size  grows unboundedly when tends to infinity. We write for the set of constructor terms of ADT sort  and size , i.e., .

Definition 1

An ADT sort  is expanding if for every natural number  there is a bound  such that for every  either or . An ADT is expanding if each of its sorts  is expanding.

Example 11

An example of an ADT that is not expanding are the natural numbers (Example 8): for every size  there is exactly one constructor term  with . ∎

CList

Colour

nil

green

red

blue

cons
Fig. 1: Dependency graph for the list ADT from Example 1
Lemma 3

Systematic unfolding terminates (i.e., in every unfolding sequence  in which every variable is eventually unfolded, eventually one of the cases of Lemma 2 applies) for all formulas  if and only if the considered ADT is expanding.

Proof:

” Example 10 generalises to arbitrary non-expanding ADTs: for every non-expanding sort  there is a constant  and an infinite semi-linear set  such that for all . The existence of follows from the proof of Theorem 2 below.

” Consider first the case of being a conjunction of disequalities and a size constraint . Since the ADT is expanding, satisfiability of reduces to the question whether the size images of the -domains contain elements large enough, and compatible with , that all disequalities can be satisfied. Systematic unfolding of will add size image constraints  for all sub-terms, and either find a set of satisfying term sizes (and corresponding terms), or conclude unsatisfiability because the conjunction of size images and size constraint  becomes inconsistent.

Adding constructor, selector, or test literals does not change the argument, since solutions of such literals can be represented in the form of a most-general unifier [17].

Non-expandingness turns out to be a corner case: all non-expanding ADTs more or less look like the natural numbers (Example 8), and most other practical ADTs are expanding. For instance, both ADT sorts in Example 1 expand.

Vi-C Effective Characterisation of Expanding ADTs

To characterise expanding ADTs, we first make the simplifying assumption that all ADT sorts  contain at least two constructor terms; sorts with only a single term can obviously be eliminated easily from a constraint. As a further piece of notation, we need a relativised version of the size image: for an ADT sort  and a constructor , we write

for the size image restricted to terms with head symbol .

Consider then the bipartite dependency graph  with vertices  being sorts and constructors, and the edge set

Fig. 1 gives the graph for the list ADT in Example 1.

Theorem 2

An ADT is not expanding if and only if the graph  contains a simple cycle  with the following properties:

  1. is the only path from to itself, i.e., every cycle starting and ending in is a repetition of ;

  2. all constructors on are unary;

  3. the cycle  unboundedly contributes to the size image , i.e.,

The characterisation theorem implies that every non-expanding ADT has a set of cyclically connected sorts S1, …, Sn, each of which might contain further constructors c1_1, c1_2, … that do not lead back to S1, …, Sn:

1\sorts {
2  // ...
3  S1 { f1(S2 s2); c1_1; c1_2; /* ... */ };
4  S2 { f2(S3 s3); c2_1; c2_2; /* ... */ };
5  // ...
6  Sn { fn(S1 s1); cn_1; cn_2; /* ... */ };
7  // ...
8}

The conditions of the theorem are clearly satisfied for the ADT of natural numbers (Example 8). Condition 3) is satisfied whenever is finite for all , but there are more subtle situations:

Example 12

We extend the list ADT from Example 1 by adding two further sorts:

1\sorts {
2  S1 { f1(S2 s2); };
3  S2 { f2(S1 s1); null; col(CList list); };
4}

The domain of sort S1 contains terms of any size greater than one. However, while the number of terms of size  grows exponentially with , there is exactly one term of size  for every , proving non-expandingness.

A cycle of length 3, in contrast, yields an expanding ADT:

1\sorts {
2  S1 { f1(S2 s2); };
3  S2 { f2(S3 s3); };
4  S3 { f3(S1 s1); null; col(CList list); };
5}

We can note that condition 3) of Theorem 2 now fails. ∎

Before we can prove Theorem 2, we need some further results about ADTs. Consider constructor terms  with a unique hole ; such terms can alternatively be seen as terms with a single occurrence of a sorted variable. Composition of terms with holes is defined as . Two terms  with holes are incomparable if for all constructor terms  of the right sort. The size  of a term with hole is the number of constructor symbol occurrences in , i.e., the hole does not count. This implies .

Lemma 4

Suppose an ADT sort  contains two incomparable constructor terms  with holes  of sort . Then expands.

Proof:

Fix some . We need to show that there is a bound  ensuring or for every . Observe that for every there are pairwise incomparable terms with holes , all of which have size . The size image  can be represented as a finite union of arithmetic progressions, with