1. Introduction
What would it take to tempt functional programmers to verify their code?
Certainly, better tools would help. We see that functional programmers who use dependentlytyped languages or proof assistants, such as Coq (Coq:manual), Agda (agda), Idris (idris), and Isabelle (isabelle), do verify their code, since their tools allow it. However, adopting these platforms means rewriting everything from scratch. What about the verification of existing code, such as libraries written in mature languages like Haskell?
Haskell programmers can reach for LiquidHaskell (Vazou:2014:RTH:2628136.2628161) which smoothly integrates the expressive power of refinement types with Haskell, using SMT solvers for fully automatic verification. But some verification endeavors require the full strength of a mature interactive proof assistant like Coq. The hstocoq tool, developed by hstocoqcpp, translates Haskell types, functions and type classes into equivalent Coq code – a form of shallow embedding – which can be verified just like normal Coq definitions.
But can this approach be used for more than the small, textbooksized examples it has been applied to so far? Yes, it can! In this work, we use hstocoq to translate and verify the two set data structures from Haskell’s containers package.^{1}^{1}1Specifically, we target version 0.5.11.0, which was released on January 22, 2018 and was the most recent release of this library at the time of publication; it is available at https://github.com/haskell/containers/tree/v0.5.11.0. This codebase is not a toy. It is decades old, highly tuned for performance, typeclass polymorphic, and implemented in terms of lowlevel features like bit manipulation operators and raw pointer equality. It is also an integral part of the Haskell ecosystem. We make the following contributions:

We demonstrate that hstocoq is suitable for the verification of unmodified, realworld Haskell libraries. By “realworld”, we mean code that is uptodate, in common use, and optimized for performance. In Section 2 we describe the containers library in more detail and discuss why it fits this description.

We present a case study not just of verifying a popular Haskell library, but also of developing a good specification of that library. This process is worth consideration because it is not at all obvious what we mean when we say that we have “verified” a library. LABEL:sec:specs discusses the techniques that we have used to construct a rich, twosided specification; one that draws from diverse, crossvalidated sources and yet is suitable for verification.

We extend hstocoq and its associated standard libraries to support our verification goal. In particular, in LABEL:sec:translating we describe the challenges that arise when translating the Data.Set and Data.IntSet modules, and our solutions. Notably, we drop the restriction in previous work (hstocoqcpp)
that the input of the translation must be intrinsically total. Instead, we show how to safely defer reasoning about incomplete pattern matching and potential nontermination to later stages of the verification process.

We increase confidence in the translation done of hstocoq. In one direction, properties of the Haskell test suite turn into Coq theorems that we prove. In the other direction, the translated code, when extracted back to Haskell, passes the original test suite.

We provide new implementationagnostic insight into the verification of the weightbalanced tree data structure, as we describe in LABEL:sec:additional. In particular, we find the right precondition for the central balancing operations needed to verify the particular variant used in Data.Set.
Our work provides a rich specification for Haskell’s finite set libraries that is directly and mechanically connected to the current implementation. As a result, Haskell programmers can be assured that these libraries behave as expected. Of course, there is a limit to the assurances that we can provide through this sort of effort. We discuss the verification gap and other limitations of our approach in LABEL:sec:limitations.
We would like to have been able to claim the contribution of findings bugs in containers, but there simply were none. Still, our efforts resulted in improvements to the containers library. First, an insight during the verification process led to an optimization that makes the Data.Set.union function 4% faster. Second, we discovered an incompleteness in the specification of the validity checker used in the test suite.
The tangible artifacts of this work have been incorporated into the hstocoq distribution and are available as open source tools and libraries.^{2}^{2}2https://github.com/antalsz/hstocoq.Don’t forget to submit the tarball
2. The containers library
We select the containers library for our verification efforts because it is a critical component of the Haskell ecosystem. With over 4000 publicly available Haskell packages using on containers, it is the thirdmost dependentup package on the Haskell package repository Hackage, after base and bytestring.^{3}^{3}3http://packdeps.haskellers.com/reverse
The containers library is both mature and highly optimized. It has existed for over a decade and has undergone many significant revisions in order to improve its performance. It contains seven container data structures, covering support for finite sets (Data.Set and Data.IntSet), finite maps (Data.Map and Data.IntMap), sequences (Data.Sequence), graphs (Data.Graph), and trees (Data.Tree). However most users of the containers library only use the map and set modules;^{4}^{4}4We calculated that 78% of the packages on Hackage that depend on containers use only sets and maps.moreover, the map modules are essentially analogues of the set modules. Therefore, we focus on Data.Set and Data.IntSet in this work.
2.1. Weightbalanced trees and bigendian Patricia trees
The Data.Set module implements finite sets using weightbalanced binary search trees. The definition of the Set datatype in this module, along with its membership function, is given in Figure 1. These sets and operations are polymorphic over the element type and require only that this type is linearly ordered, as expressed by the Ord constraint on the member function. The member function descends the ordered search tree to determine whether it contains a particular element.
The Size component stored with the Bin constructor is used by the operations in the library to ensure that the tree stays balanced. The implementation maintains the balancing invariant
where and are the sizes of the left and right subtrees of a Bin constructor. This definition is based on the description by adamstr, who modified the original weightbalanced tree proposed by nievergelt. Thanks to this balancing, operations such as insertion, membership testing, and deletion take time logarithmic in the size of the tree.
This type definition has been tweaked to improve the performance of the library. The annotations on the Bin data constructor instruct the compiler to unpack the size component, removing a level of indirection. The ! annotations indicate that all components should be strictly evaluated.
The Data.IntSet module also provides search trees, specialized to values of type Int to provide more efficient operations, especially union. This implementation is based on bigendian Patricia trees, as proposed in Morrison:1968’s work on PATRICIA (Morrison:1968) and described in a pure functional setting by okasakigill.
The definition of this data structure is shown in Figure 2. The core idea is to use the bits of the stored values to decide in which subtree of a node they should be placed. In a node Bin p m s1 s2, the mask m has exactly one bit set. All bits higher than the mask bit are equal in all elements of that node; they form the prefix p. The mask bit is the highest bit that is not shared by all elements. In particular, all elements in s1 have this bit cleared, while all elements in s2 have it set. When looking up a value x, the mask bit of x tells us into which branch to descend.
Instead of storing a single value at the leaf of the tree, this implementation improves time and space performance by storing the membership information of consecutive numbers as the bits of a machinewordsized bitmap in the Tip constructor.
The Nil constructor is the only way to represent an empty tree, and will never occur as the child of a Bin constructor. Every wellformed IntSet is either made of Bins and Tips, or a single Nil.
2.2. A history of performance tuning
The history of the Data.Set module can be traced back to 2004, when a number of competing search tree implementations were debated in the “Tree Wars” thread on the Haskell libraries mailing list. Benchmarks showed that Daan Leijen’s implementation had the best performance, and it was added to containers in 2005 as Data.Set.^{7}^{7}7https://github.com/haskell/containers/commit/bbbba97c
In (straka), Milan Straka thoroughly evaluated the performance of the containers library and implemented a number of performance tweaks (straka). This change^{8}^{8}8https://github.com/haskell/containers/commit/3535fcbe replaced a fairly readable balance and several small and descriptive helper functions with a single dense block of code. A later change^{9}^{9}9https://github.com/haskell/containers/commit/d17d7182 by straka created two copies of this scarylooking balance, each specialized and optimized for different preconditions.
adamstr describes two algorithms for union, intersection, and difference: “hedge union” and “divide and conquer”. Originally containers used the former, but in 2016 its maintainers switched to the latter,^{10}^{10}10https://github.com/haskell/containers/commit/c3083cfc again based on performance measurement.
The module Data.IntSet (and Data.IntMap) has been around even longer. okasakigill mention in their (okasakigill) paper (okasakigill) that GHC had already made use of IntSet and IntMap for several years. In 2011, the Data.IntSet module was rewritten to use machinewords as bit maps in the leaves of the tree, as discussed at the end of Section 2.1.^{11}^{11}11https://github.com/haskell/containers/pull/3 This moved the containers library further away from the literature on Patricia trees and introduced a fair amount of lowlevel bit twiddling operations (e.g., highestBitMask, lowestBitMask, and revNat).
3. Overview of our verification approach
In order to verify Set and IntSet, we use hstocoq to translate the unmodified Haskell modules to Gallina and then use Coq to verify the translated code. For example, consider the excerpt of the implementation of Set in Figure 1. The hstocoq tool translates this input to the following Coq definitions.^{12}^{12}12In the file examples/containers/lib/Data/Set/Internal.v The strictness and unpacking annotations are ignored, as they do not make sense in Coq, and the type name Set is renamed to Set_ to avoid clashing with the Coq keyword.
B@>l<@6@>l<@10@>l<@11@>c<@11E@l@15@>l<@20@>l<@41@>l<@46@>l<@52@>l<@E@>l<@[B]Definition Size := GHC.Num.Int%type. [E]
[B]Inductive Set_ a : Type [E]
[B] [6][6]:= [10][10]Bin : Size > a > (Set_ a) > (Set_ a) > Set_ a [E]
[B] [6][6] [10][10]Tip : Set_ a. [E]
[B]Definition member {a} `{GHC.Base.Ord a} : a > Set_ a > bool := [E]
[B] [6][6]let fix go arg_0__ arg_1__ [E]
[B] [6][6] [11][11]:= [11E][15]match arg_0__, arg_1__ with [E]
[B] [15][15] [20][20] _, Tip => false [E]
[B] [20][20] x, Bin _ y l r => [41][41]match GHC.Base.compare x y with [E]
[B] [41][41] [46][46] Lt [52][52]=> go x l [E]
[B] [46][46] Gt [52][52]=> go x r [E]
[B] [46][46] Eq [52][52]=> true [E]
[B] [41][41]end [E]
[B] [15][15]end [E]
[B] [6][6]in go.[E]
These definitions depend on hstocoq’s preexisting translated version of GHC’s standard library base. Here, we use the existing translation of Haskell’s Int type, the Ord type class, and Ord’s compare method.
We carry out this translation for the Set and IntSet along with their attendant functions, and then verify the resulting Gallina code. In LABEL:sec:specs we discuss the properties that we prove about the two data structures.
To further test that the translation from Haskell to Coq, we also used Coq’s extraction mechanism to translate the generated Gallina code, like that seen above, back to Haskell. This process converts the implicitlypassed type class dictionaries to ordinary explicitlypassed function arguments, but otherwise preserves the structure of the code. By providing an interface that restores the typeclass based types, we can run the original containers test suite against this code. This process helps us check that hstocoq preserves the semantics of the original Haskell program during the translation process.