1 Introduction
Bounded Model Checking [3] (BMC) is a model checking technique that allows for highly automated and scalable SAT/SMTbased verification and has been widely used to find errors in Clike languages [4, 10, 5, 1]. BMC amounts to bounding the executions of programs by unfolding loops only up to a given bound, and model checking the resulting execution graph. Since the advent of Cbmc [4], the mainstream approach additionally proceeds by symbolically executing program paths and gathering the resulting path conditions in propositional formulas which can then be passed on to SAT/SMT solvers. Thus, BMC performs a syntactic translation of program source code into a propositional formula, and uses the power of SAT/SMT solvers to check the bounded behaviour of programs.
Being a Model Checking technique, BMC has the ability to produce counterexamples, which are execution traces that lead to the violation of desired properties. A specific advantage of BMC over unbounded techniques is that it avoids the full effect of statespace explosion at the expense of full verification. On the other hand, since BMC is inconclusive if the formula is unsatisfiable, it is generally regarded as a bugfinding or underapproximation technique, which lets it avoid spurious errors. While it tends to be the most empirically effective approach for “shallow" bugs [5, 1], bugs in deep loops and recursion are often a weakness. It is only possible to prove complete correctness if bounds for loops and recursion are determinable.
The above approach has been predominantly applied to imperative, firstorder languages and, while tools like Cbmc can handle C++ (and, more recently, Java bytecode), the foundations of BMC for higherorder programs have not been properly laid. This is what we address herein. We propose a symbolic BMC procedure for higherorder functional/imperative programs that may contain free variables of ground type. Our contributions involve a syntactical translation to apply BMC to higherorder languages with higherorder state, a proof that the approach is sound, an optimisation based on pointsto analysis to improve scalability, and a prototype implementation of the procedure with experimental results showcasing its performance.
As with most approaches to software BMC, we translate a given higherorder program into a propositional formula for an SMT solver to check for satisfiability, where formulas are satisfiable only if a violation is reachable within a given bound. Where in firstorder programs BMC places a bound on loop unfolding, in the higherorder setting we place the bound on nested recursive calls. The main challenge for the translation then is the symbolic execution of paths which involve the flow of higherorder terms, by either variable binding or use of the store. This is achieved by adapting the standard technique of Static Single Assignment to a setting where variables/references can be of higher order. To handle higherorder terms in particular, we use a nominal approach to methods, whereby each method is uniquely identified by a name. We capture program behaviour by also uniquely identifying every step in the computation tree with a return variable; analogous to how Cbmc [4] captures the behaviour of sequencing commands in ANSIC programs.
To give a simple example of the approach, consider the following code, where is a reference of type , and are variables of type , and are variables of type int.
In the code above, a function is assigned to reference . In a symbolic setting, it is not immediately obvious which function to call when dereferencing in line 4. Luckily, we know that when calling in line 3, its value can only be the one bound to it in line 1. Thus, a first transformation of the code could be:
The assignment in line 3 can be facilitated by using a return variable and method names for and :
We now need to symbolically decide how to dereference . The simplest solution is try to match with all existing functions of matching type, in this case and :
Performing the substitutions of , we can read off the following formula for checking falsity of the assertion:

The above is true e.g. for , and hence the code violates the assertion.
These ideas underpin our first BMC translation, which is presented in Section 3. The language we examine, HORef, is a higherorder language with general references and integer arithmetic. While correct, one can quickly see that our first translation is inefficient when trying to resolve the flow of functions to references and variables. In effect, it explores all possible methods of the appropriate type that have been created so far, and relies on the solver to pick the right one. In Section 5, we optimise the translation by restricting such choices according to an analysis akin to pointsto analysis [15, 2]. Finally, in Section 6 we present an implementation of our technique in a BMC tool for a higherorder OCamllike syntax extending HORef and test it on several example programs adapted from the MoCHi benchmark [13].
2 The Language: HORef
Terms  
Values 
1[] fail: θ 1[] () : unit 1[] i : int x∈Varsθ 1[] x : θ m ∈Methsθ,θ’ 1[] m : θ→θ’ M1, M2 : int 1[] M1 ⊕M2 : int 
M : int M0 : θ M1 : θ 3[] if then else : θ M1 : θ1 M2 : θ2 2[] ⟨M1 , M2 ⟩: θ1 ×θ2 ⟨M1 , M2 ⟩: θ1 ×θ2 1[] πi ⟨M1 , M2 ⟩: θi (i=1,2) 
r ∈Refsθ 1[] !r : θ r ∈Refsθ M : θ 2[] r := M : unit M : θ’ x : θ 2[] λx . M : θ→θ’ M : θ N : θ’ x : θ 3[] let in : θ’ 
M : θ’ N : θ” x : θ→θ’ y : θ 4[] letrec in : θ” x: θ→θ’ M : θ 2[] x M : θ’ m ∈Methsθ→θ’ M : θ 2[] m M : θ’ 
Here we present a higherorder language with (higherorder) state, which we call HORef, as an idealised setting for languages like Java and OCaml. The syntax consists of a callbyvalue calculus with types
and references of arbitrary types. We assume countable disjoint sets Vars, Refs and Meths, for variables, references and methods respectively. Variables are ranged over by and variants; references by and variants; and methods by and variants. We assume these sets are typed, that is:
The syntax and typing rules are given in Figure 1. Note that we assume a set of arithmetic operators , which we leave unspecified as they do not affect the analysis. We extend the syntax with usual constructs for sequencing and assertions: stands for let in ; while is ; and is if then () else fail (with boolean values represented by ).
Note that the use of typed variables allows us to type terms without need for typing contexts. As usual, a variable occurrence is free if it is not in the scope of a matching (/let/letrec)binder. Terms are considered modulo equivalence and in particular we may assume that no variable occurs both as free and bound in the same term. We call a term closed if it contains no free variables.
References in our language are global, and there is no fresh reference creation construct (this choice made for simplicity). On the other hand, methods are dynamically created in terms, and for that reason we will be frequently referring to them as names. The terminology comes from nominal techniques [6, 12]. On a related note, abstractions are not values in our language. This is due to the fact that in the semantics these get evaluated to method names.
In our approach, checking for violations of safety properties is reduced to the reachability of failure. We have therefore included the fail primitive for when a program reaches a failure. Accordingly, our bounded model checking routine will return fail when it aborts on reaching a failure and nil when it aborts on reaching the bound. The use of nil is analogous to the unwinding assertions used in Cbmc. It is not part of the syntax of HORef.
Bounded Operational Semantics
We next present a bounded operational semantics for HORef, which is the one captured by our bounded BMC routine. The semantics is parameterised by a bound which, similarly to loop unwinding in procedural languages, it bounds the depth of method (i.e. function) calls within an execution. A bound in particular means that, unless no method calls are made, execution will terminate returning nil. Consequently, in this bounded operational semantics, all programs must halt; either when the program itself halts (returning a value or fail), or when the bound is exceeded. Note at this point that the standard (unbounded) semantics of HORef, allowing arbitrary recursion, can be obtained e.g. by allowing bound values .
To describe this behaviour, we chose a bigstep operational semantics representation with rules of the form
where . In other words, all terms must eventually evaluate to a value, fail or nil. A configuration is a quadruple where is a typed term and:

is a finite map, called a method repository, such that for all , if then .

is a finite map, called a store, such that for all , if then .

is the nested calling bound, where decrementing beyond zero results in nil.
A closed configuration is one all of whose components are closed. We call a configuration valid if all methods and references appearing in are included in and respectively. A closed configuration is one all of whose components are closed.
Definition 1
The operational semantics is defined on closed valid configurations, by the rules given in Figure LABEL:fig:semantics.
Nominal determinacy
While the operational semantics is bounded in depth, the reduction tree of a given term can still be infinite because of the nondeterminacy involved in evaluating abstractions (rule ): the rule nondeterministically creates a fresh name and extends the repository with mapped to the given abstraction. This kind of nondeterminism, which can be seen as determinism up to fresh name creation, is formalised below.
Let us consider permutations such that, for all , if then . We call such a permutation finite if the set is finite. Given a syntactic object (e.g. a term, repository, or store) and a finite permutation , we write for the object we obtain from if we swap each name appearing in it with . Put otherwise, the operation is an action from finite permutations of Meths to the set of objects . Given a set and objects , we write whenever there exists a finite permutation such that:
and say that and or nominally equivalent up to .
Lemma 1
Given , for all
we have
iff
.
To illustrate bounding of method application and the use of names in place of methods, we provide the following example.
Example 1
Consider the following recursive higherorder program of HORef (with some syntactic sugar) which we shall unwind with a bound .
Let us write the above program as , and as letrec in . We can attempt to evaluate as follows. Below we let , and .

First, ,

next, evaluate ,

i.e. , with ,

i.e. ,

now, first evaluate ,

i.e. ,

i.e. ,

i.e. ,

i.e. ,

i.e. ,

i.e. ,

i.e. ,
which returns nil.
Hence, the evaluation aborts with nil. The interesting part of the program is the assertion, which is not reached with this bound. Setting the bound to 6, will eventually be called with 0 and return the function . The latter will be bound to and called on 5, and at that point will have value 5, so the assertion will pass. Setting initially would lead to failure.
Intuitively, the bounded semantics is equivalent to a bounded inlining of methods. As such, evaluating the example with can be seen as unwinding the program as follows (where we have also included the function definitions for clarity).
We will come back to this example in the next section.
3 A Bounded Translation for HORef
We present an algorithm which, given a term and a bound , produces a propositional formula which captures the bounded semantics of , for the bound . More precisely, the algorithm receives a valid configuration as input, where may only contain free variables of ground type, and produces a formula and a variable . Then, for any substitution closing the configuration, and any corresponding formula , reaches some iff
is satisfiable. The formal statement and proof of the above is given in Theorem 4.1. It is slightly more elaborate as it takes into account the possibly different choices of fresh method names in the translation and evaluation.
The translation operates on intermediate symbolic configurations, of the form , where:

are static single assignment (SSA) maps where SSAVars is the set of SSA variables of the form such that is the number of times has been assigned to so far. The map is counting all the assignments that have taken place so far in the translation, whereas only counts those in the current path. E.g. if has been assigned to five times so far. We write to mean update with reference : if , then , where is fresh.

is a propositional formula containing the behaviour of the current path so far.
Moreover, is a repository storing all methods created so far, and is the bound. The translation returns tuples of the form , where have the same interpretation, albeit for after reaching the end of all paths for the term . The variable represents the return value of the initial configuration.
The algorithm uses a freshname generator, which is left unspecified but deterministically produces the next fresh name, or variable, or SSA variable of appropriate type. Following the SSA approach, the variables in particular are always chosen fresh, so that each identifies a unique evaluation point in the translation. We use SSA form because it allows us reason about assignment as equations. We compute the SSA form on the fly by substituting all free variables with their corresponding at binding, and through the use of SSAmaps and for references.
We now describe the translation. The translation stops when either the bound nil, a fail, or a value has been reached. The base cases add clauses mapping return variables to actual values of evaluating .
Inductive cases build the symbolic trace of by recording in all changes to the store, and the return values () at each step in the computation tree. These steps are then chained together using the guard :
which propagates nil and fail, and the SSA maps .
The difference between reading () and writing () is noticeable when branching. There are two branching cases here: the conditional case, and the one for application . In the former one, we branch according to the return value of the condition (denoted by ), and each branch translates and respectively. In this case, both branches read from the same map , but may contain different assignments, which we accumulate in . The formula
encodes a binary decision tree with guarded clauses that represent the path guards.
When applying variables as methods (, with ), we encode in an ary decision tree where is the number of methods to consider. This is necessary since the algorithm is symbolic and therefore agnostic to what is pointing at. In such cases, we assume nondeterminism, meaning that could be any method in the repository restricted to type (denoted ). We call this case nondeterministic method application. This case seems to be fundamental for applying BMC to higherorder terms, and higherorder references. It is made possible by the introduction of names for methods, as it allows for comparison of higherorder terms as values. Nondeterministic method application is a primary source of scalability problems, however, and will be discussed in more detail later.
The BMC translation is given as follows. It transforms each symbolic configuration to . In all of the cases below, is a fresh variable and . We also assume a common domain , which is the finite subset of Refs containing all references that appear in and .
Base Cases:
Inductive Cases:
To illustrate the algorithm, we look at two characteristic cases. In , we first compute the translation of . Using the results of said translation, we can substitute in the fresh variable for , and compute its translation. To finish, we return , chain it to using predicate in , and return the remaining results of translating .
In we see nondeterministic method application in action. We first translate the argument and obtain . We then restrict the repository to type to obtain the set of names identifying all methods of matching type for . If no such methods exist, this means that the binding of had not succeeded (because of ) and we are examining a dead branch, so we immediately return. Otherwise, for each method in this set, we obtain the translation of applying to the argument . This is done by substituting for in the body of . After translating all method applications, all paths are joined in , as described earlier, by constructing an ary decision tree that includes the state of the store in each path. We do this by incrementing all references in , and adding the clauses for each path. These paths are then guarded by the clauses . Finally, we return a formula that propagates nil and fail in case reaches either of them. Note that we return as both the and resulting from translating this term. This is because all branches have been joined, and any term sequenced after this one should have all updates available to it.
We now come back to Example 1 to illustrate the intuition of SSA and return variables, nondeterministic method application, and formula construction.
Example 2
Consider Example 1 modified with free variables and .
We transform it to produce the program in SSA form with nondeterministic method application at line 11, again unwinding with . Note that all assignments have been replaced with letbindings. This is because, in SSA form, we think of references as SSA variables. In addition, we use keyword new to add new for names to the repository.
We can then build model for the example. For economy, we hide the nuances of propagating nil and fail in predicate , which is shorthand for . We also omit wherever no fail or nil appears in the term, and directly return constants instead of translating them. To construct the formula, we traverse the term in order, and add clauses in order of traversal. Note that the “else" branch is always explored first in conditionals.
(line 1)  
(line 6)  
(line 7)  
(line 10)  
(line 7)  
(line 8)  
(line 9)  
(line 8)  
(line 8)  
(line 13)  
(line 13)  
(line 14)  
(line 14) 
In this case, if we set , recalling , then is satisfiable with a minimum , since we need at least iterations to reach (which is also the case for a negative , as the program would diverge). With , however, we cannot violate the assertion, i.e. is not satisfiable. Setting , on the other hand, causes to be satisfiable with or .
Bounded Model Checking with the Translation
The steps to do a bounded model checking of some configuration using the bounded translation algorithm described previously are as follows:

Build the initial axioms/preconditions:
. 
Build the initial SSA maps:
. 
Compute the translation:
. 
This is where the expressiveness of fail and nil becomes relevant. To check for:

sound errors:

reached bounds (for verification):

a specific return property :
, e.g. 
a specific store property :
, e.g.


Transform to the relevant SMT solver format (e.g. SMTLib), and use the SMT solver to get a satisfying assignment.

When checking for fail, if the formula is unsatisfiable, we can increase the bound given checking for nil is satisfiable. If nil is not satisfiable either, then the program has been verified.
Note that checks for store properties (d) can be combined with any of the properties mentioned in step (6), including other store properties. It is only possible to check other properties (a,b, and c) independent of each other, however. This is because the translation is deterministic and will always output a unique result. For instance, the return value cannot be both fail and nil in the same satisfying assignment, i.e. the formula is unsatisfiable. Moreover, while the semantics requires closed terms, the translation is indifferent towards free variables. As such, it will handle toplevel input arguments and said free variables by simply adding them into the formula, which will produce a unique return for each valid assignment of the input arguments. This is, in fact, one of the most useful applications of BMC, since it then generates counterexamples from said input arguments. These free variables, however, must be of ground type. The translation will not mind if a free variable is given a higherorder type. But then the resulting formula becomes unsound, since we do not model unknown program code. The simplest solution is to make the formula always unsatisfiable to avoid spurious errors. Handling open terms with higherorder free variables will be discussed in more detail as future work.
4 Soundness of the BMC translation
In this section we prove that our BMC algorithm is sound for input terms that are closed or contain open variables of ground type.
We start off with some definitions. An assignment is a finite map from variables to closed values. Given a term , we write for the term obtained by applying to . On the other hand, applying to a method repository , we obtain the repository – and similarly for stores . Then, given a valid configuration , we have .
Given a formula and an assignment , we say represents , and write , if:

satisfies (written );

implies : .
Given assignment , we define a formula representing it by: .
Given a valid configuration , let us set:
and define: .
Theorem 4.1 (Soundness)
Given a valid configuration whose open variables are of ground type, suppose . Then, for all assignments closing , the following are equivalent:


.
Proof
Take and . By construction then, . Let us set . From the assumption and the fact that the translation propagates the formula from the initial condition (Lemma 5), we have:
Moreover, is valid and closed so, by Lemma 2, we have that and .
Suppose now holds. By Lemma 1, we have that , so taking we obtain (2).
On the other hand, if holds then implies and . Since , we get . Hence, and we conclude using Lemma 1.
Theorem 4.1 uses the following main lemma, which is shown in the Appendix. Below we assume that