Implementing Efficient All Solutions SAT Solvers

10/02/2015 ∙ by Takahisa Toda, et al. ∙ University of Electro-Communications 0

All solutions SAT (AllSAT for short) is a variant of propositional satisfiability problem. Despite its significance, AllSAT has been relatively unexplored compared to other variants. We thus survey and discuss major techniques of AllSAT solvers. We faithfully implement them and conduct comprehensive experiments using a large number of instances and various types of solvers including one of the few public softwares. The experiments reveal solver's characteristics. Our implemented solvers are made publicly available so that other researchers can easily develop their solver by modifying our codes and compare it with existing methods.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

Propositional satisfiability (SAT for short) is to decide if a Boolean formula is satisfiable. SAT is ubiquitous in computer science. Because of its significance, it has attracted the attention of many researchers from theory to practice. Many applications have motivated empirical studies, in particular the development of SAT solvers, softwares to solve satisfiability. A fundamental task of SAT solvers is to solve as many instances as possible in a realistic amount of time. To this end, various practical algorithms and elegant implementation techniques have been developed [56] [9] [40].

There are many variants of SAT. All solutions SAT (AllSAT for short) or model enumeration is studied in the paper. It is, given a CNF formula, to generate partial satisfying assignments such that they form a logically equivalent DNF formula. Compared to neighboring areas, AllSAT has been relatively unexplored. This is mentioned in the literature [29] and also supported by the fact that there are only a few recent papers, almost no software111A few exceptions are clasp [21], picosat [10], and relsat [6]. Although they support solution generation, they are positioned as an answer set solver, a single solution SAT solver, and a #SAT solver rather than AllSAT solvers, respectively. is publicly available, and it has not even been taken up in major handbooks related to satisfiability.

A recent application of AllSAT is data mining. A fundamental task in data mining is to generate all interesting patterns from a given database [26]. Examples include frequent itemsets, maximal frequent itemsets, or closed itemsets in transaction databases. Although algorithms for generating various patterns have been proposed, they are basically specialized for their target patterns. This means that different patterns require new algorithms. For this reason, a framework based on declarative paradigm has recently been proposed [24]. A basic flow is that constrains of patterns to be generated are formulated as logical formulae and solved with a generic solver. Hence, all that users do is simply to model their problems, not to design algorithms. Among much related work, an approach on which problems are encoded into CNF formulae and solved with AllSAT solvers has been studied [31]. An advantage of declarative paradigm is its ability to handle new patterns in a flexible manner. There is no need to see details of algorithms on which solvers are based, thereby it is opened to wider users. It is instead inferior in efficiency to problem-specific approaches. In practice it is necessary to balance between efficiency and flexibility. Therefore, improving solver’s performance is essential in the declarative framework.

Besides the data mining, there have been many studies on the application of ALLSAT, in particular to formal verification, such as network verification [58] [38] [49], predicate abstraction [14], backbone computation [41], image and preimage computation in unbounded model checking [25] [43] [51] [37] [20] [23].

Considering the above, we find it important to clarify state-of-the-art techniques of AllSAT solvers and to improve them on a firm basis. However, there are the following issues in the existing researches of ALLSAT.

  • Several methods are proposed but they are not globally compared. It is thus difficult to decide which method is effective for which kinds of ALLSAT instances.

  • Experiments are not carried out on comprehensive benchmarks.

  • There is few public ALLSAT solver, which makes it difficult to compare existing techniques.

We thus would like to survey major techniques of AllSAT solvers and try to complement past references by gathering and organizing existing techniques. We further add some novel techniques. To evaluate solvers, we conduct experimental comparisons, including clasp, one of the few off-the-shelf softwares with solution generation support. Our implemented solvers are made publicly available with expectation that further improvement on solvers and their evaluation are easily done and the AllSAT research is stimulated.

The paper is organized as follows. Section 2 provides related work of AllSAT. Section 3 provides necessary notions, terminology, and results. Section 4 surveys major techniques of AllSAT solvers, where those including our original ideas are indicated by adding asterisks to their titles. Section 5 provides experimental results. Section 6 concludes the paper.

2. Related Work

Another variant of SAT is the dualization of Boolean functions: given a DNF formula of a Boolean function , it is to compute the complete DNF formula of the dual function . Since a CNF formula of can be easily obtained by interchanging logical disjunction with logical conjunction as well as the constants with , a main part is to convert CNF to the complete DNF. Hence, an essential difference from AllSAT is that the resulting DNF formula must be complete. Dualization has been well-studied in terms of complexity [18] [15], while there seems no recent empirical study with a few exceptions [33] [11] [30] [54]. Practical algorithms for a restricted form of dualization have been presented [53] [46] and some implementations are available222Hypergraph Dualization Repository, by Keisuke Murakami and Takeaki Uno,, accessed on 19th Jan., 2013. HTC-DD: Hypergraph Transversal Computation with Binary Decision Diagrams, by Takahisa Toda at ERATO MINATO Discrete Structure Manipulation System Project, Japan Science and Technology Agency, at Hokkaido University,, accessed on 10th Sept., 2015., though they are not for arbitrary Boolean functions.

Another variant is the problem of counting the number of total satisfying assignments, called propositional model counting or #SAT. It has been well-studied because of good applications such as probabilistic inference problems and hard combinatorial problems, and some solvers are available [9]. Although #SAT is apparently similar to AllSAT, techniques such as connected components and component caching are inherent in counting, and they are not applicable to AllSAT as is [44].

3. Preliminaries

Necessary notions, terminology, and results concerning Boolean functions, satisfiability solvers, and binary decision diagrams are presented in this section.

3.1. Boolean Basics

A literal is a Boolean variable or its negation. A clause is a finite disjunction of literals, and a term is a finite conjunction of literals. A propositional formula is in conjunctive normal form (CNF for short) if it is a finite conjunction of clauses and in a disjunction normal form (DNF for short) if it is a finite disjunction of terms. We identify clauses with sets of literals and CNF formulae with sets of clauses. The same applies to terms and DNFs. The dual of a Boolean function is the function defined by . An implicant of a Boolean function is a term with , where is considered as a Boolean function and the order of Boolean functions is introduced as if for all . An implicant is prime if the removal of any literal results in a non-implicant. A DNF formula is complete if it consists of all prime implicants.

An assignment to a set of Boolean variables is a partial function from to . A satisfying assignment for a CNF formula is an assignment such that the CNF formula evaluates to . An assignment to is total (or complete) if it is a total function, that is, all variables in are assigned values. A Boolean formula is satisfiable if it has a satisfying assignment. For simplicity, we say that a literal is assigned a value if the assignment to the underlying variable makes the literal evaluate to . If there is no fear of confusion, we identify an assignment function over with the set of the form . We further identify the assignment with if and with if . In this way, assignments and literals are used interchangeably throughout the paper.

Example 1.

Consider the sequence of literals . This means that are selected in this order and the values are assigned to them, respectively.

3.2. Satisfiability Solvers

Propositional satisfiability problem (SAT for short) is the problem of deciding if there exists a satisfying assignment for a CNF formula. Algorithm 1

shows a basic framework on which modern SAT solvers are based. For simplicity, other techniques such as lazy data structures, variable selection heuristics, restarting, deletion policy of learnt clauses are omitted. See for details 

[56] and [9].

A basic behavior of Algorithm 1 is to search a satisfying assignment in such a way that a solver finds a candidate assignment by assigning values to variables and if the assignment turns out to be unsatisfying, the solver proceeds to the next candidate by backtracking. The extension of an assignment is triggered at the decide stage, where an unassigned variable is selected and a value is assigned to , and it is then spread at the deduce stage, where assignments to other variables are deduced from the most recent decision .

Decision assignments are those given at the decide stage and decision variables

are those assigned values there. Consider a decision tree that branches at each decision assignment. A

decision level is the depth of that decision tree, which is maintained by the variable in Algorithm 1. The decision level of a variable , denoted by , is one at which was assigned a value. For a literal , the notation is defined as that of the underlying variable, and denotes that , seen as an assignment, was given at level , i.e. .

The deduce stage is described below. A clause is unit if all but one of literals are assigned the value and the remaining one is unassigned. The remaining literal is called a unit literal. Unit clauses are important in the deduce stage because assignments to the underlying variables of unit literals are necessarily determined so that unit literals evaluate to . After some assignments are determined by unit clauses, non-unit clauses may become unit. Hence, all implications are deduced until an unsatisfied clause exists or no unit clause exists. This process is called unit propagation. The function performs unit propagation. Implied assignments are those given at this stage and implied variables are those assigned values there. The decision level of an implied variable , the notations and , where is a literal representing an implied assignment, are defined in the same way.

Input: a CNF formula , an empty assignment .
Output: SAT if is satisfied; UNSAT, otherwise.
  /* Decision level */
1 while true do
       /* Deduce stage */
2      if conflict happens then
3           if then return UNSAT;
            /* Diagnose stage */
4           ;
5           ;
7           else
8                if all variables are assigned values then
9                     report ;
10                     return SAT;
11                     else
                           /* Decide stage */
12                          select an unassigned variable and a value ;
13                          ;
15                          end if
17                          end if
19                          end while
ALGORITHM 1 DPLL procedure with conflict driven clause learning, where denotes the decision level of a variable .
Example 2.

Consider the CNF formula that consists of the following clauses.

Assume the decision assignment . The implied assignment is obtained from . Assume the decision assignment . The implied assignments and are obtained from and in this order. Assume the decision assignment , and the CNF formula is satisfied.

It should be noted that in the middle of unit propagation, we may encounter with an unsatisfied clause. This case is called conflict. As soon as conflict happens, unit propagation halts, even though unit clauses still remain. In conflict case, if all assigned variables are those assigned prior to any decision (i.e. ), it means that there are no other assignments to be examined, thereby a CNF formula must be unsatisfiable. In that case, a solver halts, reporting UNSAT. If decision has been made at least once, we enter into the diagnose stage to resolve conflict.

At the diagnose stage, a ”cause” of the conflict we have just met is analyzed, a new clause is learnt as a result, and it is added to a CNF formula, by which a solver is guided not to fall into the conflict again (and other conflicts related to it). To do this efficiently, modern solvers maintain an implication graph during search, which represents an implication relation between assignments over unit propagation. Specifically, an implication graph is a directed acyclic graph such that

  • vertices in correspond to literals representing assignments to their variables;

  • arcs in correspond to implications so that if a unit clause with unit literal yields in unit propagation, then arcs from all assignments to underlying variables in to the implied assignment are added;

  • if an unsatisfied clause exists, then arcs from all assignments to underlying variables in that clause to the special vertex are added.

An implication graph might be implemented so that whenever a variable is implied, it is associated with the clause that determined the assignment to as a unit clause. This clause is called the antecedent of .

Figure 1. A conflict graph, where arcs are labeled with antecedents of their target vertices.
Example 3.

Consider the CNF formula given in Example 2. Assume in turn the decision assignments , , and in this order. The resulting implication graph is shown in Fig. 1. This case results in conflict because becomes unsatisfied.

A conflict graph is a subgraph of an implication graph obtained by restricting so that all vertices have paths to . For a subset of vertices in a conflict graph , the arc-cut (hereafter cut) corresponding to is the set of arcs that connect vertices in with those in . Examples are illustrated by dotted curves in Fig. 1.

We are now ready to describe a clause learning scheme, which is performed by the function . Consider cuts such that all decision assignments are on one side, called the reason side, and the special vertex is on the other side, called the conflict side. Take negation of literals on reason side that are incident to arcs in a cut, which form a conflict-driven clause (or conflict clause). This clause is considered as a ”cause” of conflict. Indeed,if variables are assigned values following literals on reason side that are incident to arcs in a cut, then the same implications on conflict side are derived and the same conflict must take place. Therefore, to avoid the conflict, it is necessary for variables to be assigned values so that at least one of those literals is negated. This condition is formulated as the conflict clause obtained above.

As illustrated in Fig. 1, there are many choices for cuts that induce conflict clauses. Among them, conflict clauses that contain exactly one literal from the current decision level are known to be effective. A unique implication point (UIP for short) is a vertex in a conflict graph such that every path from the decision at the current decision level to passes through it. Note that at least one UIP exists, because the decision at the current decision level is a UIP. The first UIP scheme is to find the UIP that is the closest to .

Example 4.

Consider the conflict graph given in Fig. 1. The middle curve gives the first UIP . Hence, a conflict clause is .

The first UIP scheme can be efficiently performed by traversing a conflict graph from in the reverse order of implications, based on the implementation of an implication graph stated above. During the traversal, it is easy to decide if the current literal is a UIP. Consider the cut that induces a UIP. For all literals on the reason side, their assignments are determined prior to those on the conflict side. Since the traversal is in the reverse order of implications, the UIP can then be located by keeping track of the number of unvisited vertices that are immediate neighbors of visited vertices.

All that remains is to decide a backtrack level , cancel all assignments above , and set to the current decision level. To decide a backtracking level, there are two choices. Suppose that the decision at level resulted in conflict and there is no solution that extends the current assignment. Chronological backtracking cancels all assignments of level including the decision and attempts to find a solution by extending the assignment from level with a conflict clause. In this way, chronological backtracking undoes assignments from a higher to a lower level. A drawback is that if there is no solution in higher levels, it is hard to get out of those levels. On the other hand, non-chronological backtracking jumps at once to a lower level by canceling all assignments above and attempts to find a solution upward by extending assignment from level . The backtracking level is commonly determined as the largest level of a variable in a conflict clause below the current level . For each canceled assignment between the levels and , the possibility for becoming a solution is left in general, though this does not mean that a solver loses an opportunity to find solutions.

Example 5.

Consider the conflict graph given in Fig. 1. Since a conflict clause is , the non-chronological backtrack level is . Hence, the assignments are canceled and search restarts from , however a solution could be obtained by backtracking to the level .

Remark 1.

When a conflict clause is learnt, an assignment to its unit literal is implied. Hence, the function adds the implied assignment to the current assignment function , by which its effect to other assignments is taken at a subsequent propagation.

3.3. Binary Decision Diagrams

A binary decision diagram (BDD for short) is a graphical representation of Boolean functions in a compressed form [36] [1] [12]. We follow the notation and terminology in Knuth’s book [35].

Figure 2 shows an example of a BDD. Exactly one node has indegree , which is called the root. Each branch node has a label and two children. Node labels are taken from variable indices, and the children consists of the LO child and the HI child. The arc to a LO child is called LO arc, illustrated by a dotted arrow, and the LO arc of means assigning the value to the variable of . Similarly, the arc to a HI child is called HI arc, illustrated by a solid arrow, and the HI arc of in turn means assigning the value to its variable. There are two sink nodes, denoted by and . Paths from the root to and mean satisfying and unsatisfying assignments, respectively.

BDDs are called ordered if for any node with a branch node as its child, the index of is less than that of . BDDs are called reduced if the following reduction operations can not be applied further.

  1. If there is a branch node whose arcs both point to , then redirect all the incoming arcs of to , and then eliminate (Fig. 3(a)).

  2. If there are two branch nodes and such that the subgraphs rooted by them are equivalent, then merge them (Fig. 3(b)).

In this paper, ordered reduced BDDs are simply called BDDs. Ordered BDDs that need not be fully reduced are distinguished from ordinary BDDs by calling OBDDs. Note that each node in a BDD (or an OBDD) is conventionally identified with the subgraph rooted by it, which also forms BDD (or OBDD).

Figure 2. The BDD representation for the function: .

(a) Node elimination

(b) Node merging
Figure 3. The reduction rules

4. Techniques of All Solutions SAT Solvers

In order to implement an efficient AllSAT solver, we have to carefully determine an appropriate suite of techniques by considering various factors and their characteristics. Some details have been mentioned only partly and scattered in past references. We survey major existing techniques of AllSAT solvers and try to complement past references by gathering and organizing existing techniques. We further add some novel techniques.

This section is organized as follows. Three major types of solvers are presented in their own subsections. Each subsection starts with an overview, then provides specific techniques, and ends with the configuration of our implemented solvers. We added an asterisk to the title of each specific technique that contains our original ideas.

4.1. Blocking Solvers

4.1.1. Overview

One of the easiest ways of implementing an AllSAT solver is to repeatedly run an ordinary SAT solver as a black box and find satisfying assignments one by one. A specific procedure is as follows.

  1. Run a SAT solver with a CNF formula .

  2. If is unsatisfiable, halt.

  3. Report a found total satisfying assignment .

  4. Compute the clause of the form in a set notation.

  5. Add to and go to Step 1.

The clause obtained at Step 4 is called a blocking clause [43]. Since the blocking clause is the complement of the term corresponding to , the extended CNF formula is not satisfied by in a later search, thereby a new solution will be found in each repetition. Furthermore, since is total, no assignment other than is blocked.

Example 6.

Execute the procedure with the CNF formula:

Suppose a solver returns the satisfying assignment . The blocking clause is added to :

The solver then returns the satisfying assignment . The blocking clause is added to :

This time a solver returns UNSAT, which means all satisfying assignments are found.

Since a blocking clause has size equal to the number of variables, unit propagation is likely to slow down. Hence, it is arguably better to consider blocking clauses that consist only of decisions, which we call a decision-based blocking clause for convenience. Since decisions determine the other assignments, decision-based blocking clauses have the same effect as those of all literals. However, to know which literals are decisions, we need to modify a solver code. Algorithm 2 is a pseudo code obtained by modifying Algorithm 1. For convenience, we simply call this blocking procedure to distinguish it from the other procedures presented later. Only lines 5,12-16 are changed. At line 5 and 12, a solver halts because means that all variables are implied without any decision and all solutions are found. At line 15, all assignments except for those determined without any decision are canceled, and at line 16 a solver backtracks to the root level.

A decision-based blocking clause only blocks a single assignment and there are as many number of blocking clauses as total satisfying assignments. Since they are stored, it is likely to result in space explosion and slow down of unit propagation. This is considered as a serious issue since unit propagation in modern SAT solvers occupies the majority of a whole processing time. Another disadvantage is that whenever a solution is found, a solver is enforced to restart from scrach with an extended CNF formula, not resuming search.

On the other hand, the blocking clause-based implementation might be considered as a good choice for such instances as even one solution is hard to find or for instances with a small number of solutions. It is easily implementable, because the blocking clause mechanism can be realized outside a solver or with a small modification on a solver code as demonstrated in Algorithm 2. We can benefit from powerful techniques of modern SAT solvers such as conflict-driven clause learning, non-chronological backtracking, and so on.

Input: a CNF formula , an empty assignment .
Output: all satisfying assignments.
  /* Decision level */
1 while true do
       /* Deduce stage */
2      if conflict happens then
3           if then halt;
            /* Diagnose stage */
4           ;
5           ;
7           else
8                if all variables are assigned values then
9                     report ;
10                     if then halt;
11                     compute a blocking clause from ;
12                     ;
13                     ;
14                     ;
16                     else
                           /* Decide stage */
17                          select an unassigned variable and a value ;
18                          ;
20                          end if
22                          end if
24                          end while
ALGORITHM 2 Blocking procedure, where denotes the decision level of .
Remark 2.

Once blocking clauses are added to a CNF formula, they are not deleted afterward and must not be treated in the same way as conflict clauses. Otherwise, solutions would be rediscovered many times, which is not allowed in the paper.

4.1.2. Simplifying Satisfying Assignments

Simplification of satisfying assignments is to obtain from a satisfying assignment to a CNF formula a smaller assignment333We say that an assignment is smaller than if all variables assigned values in are also assigned values in and their values coincide, i.e., . that still makes evaluate to . This is done by canceling assignments to redundant variables in , where redundant means either value is assigned without effect on the value of . A simplified assignment is partial in general and it represents a set of total assignments, including the original assignment and possibly other satisfying assignments. Variable lifting refers to a number of such simplification techniques [50]. Since this topic is well-summarized in the literature [44], we do not go into details. The interested readers are referred to it, as well as the references therein. For recent results, see also the literature [57].

Simplification allows us to obtain from a single solution possibly exponentially many solutions in a compact form, i.e., as a partial assignment. It is desirable if we can obtain a partial assignment of minimum size, however minimization is known to be computationally hard, and in practice we have to compromise with near-minimum assignment by means of approximation444Even computing a minimal satisfying assignment requires quadratic time, which is still expensive.. To combine simplification with the blocking mechanism of Algorithm 2, it suffices to perform simplification just before line 13 and then to take complement of decisions in a simplified assignment. It should be noted that if a simplified blocking clause is empty, that is, all variables except for implied ones turn out to be redundant, then it means that all remaining solutions are covered. Thus, in that case, a solver must be halted. Thanks to simplification, the number of blocking clauses may be largely reduced, which leads to a good effect on unit propagation at the cost of performing simplification.

Example 7.

Consider the CNF formula given in Example 2. The assignments were given in this order. The last decision is redundant. By removing it, we obtain the partial assignment . Consider the decision assignments and , by which the other variables are implied or can be assigned either value. By flipping those assignments, we obtain the simplified blocking clause .

4.1.3. Continuing Search

In Algorithm 2, whenever a solution is found, a solver is enforced to backtrack to the root level. After that, due to a variable selection heuristic, a different assignment will be examined and a region of the search space in which a solution was just found remains incomplete, which may give rise to unnecessary propagations and conflicts in a later search for that region. Restart is, however, essential in AllSAT solving in particular when simplification is used. Indeed after simplification is performed and a new clause is added, the state of implications such as which literals are decisions becomes inconsistent, and it is necessary to deduce implications again. It is not straightforward to answer how to continue search [32].

The problem of over-canceling due to backtracking was addressed [47], and a simple technique, called progress saving, that stores recent canceled decisions in an array and simulates them after backtracking was proposed. Specifically, any time a solver enters into the decide stage, it checks if an assignment to a selected variable is stored, and it simulates the previous decision if exists; otherwise, it follows a default heuristic. Although this technique was proposed in the context of SAT, it is also applicable to AllSAT.

Example 8.

Continuing Example 7, suppose that we added the blocking clause to and backtracked to the root level with progress saving enabled. At this point, all assignments above the root level are canceled, yet the previous decisions are stored in an array. If or is selected, the previous decision is made again.

4.1.4. Implementation

We implemented 4 programs based on blocking procedure according to whether simplification and continuation techniques are selected or not.

For a simplification technique, we used a method related to set covering model [44] and decision-based minimal satisfying cube [57]. For the sake of efficiency, near-minimal satisfying assignments are computed. A basic idea is that given a total assignment, we select as a small number of decision variables as make a CNF formula evaluate to . This is done in the following way:

  1. select all decision variables that related to implications of at least one variable;

  2. for each clause that is not satisfied by selected variables, select arbitrary decision variable that makes the current clause satisfied.

A simplified satisfying assignment then consists of assignments to the selected decision variables and all implied variables. By flipping the assignments to the selected decisions, we obtain a blocking clause, which blocks all total assignments represented by the simplified assignment.

We implemented a continuation technique so that before backtracking at line 15, all decisions are stored in an array and after backtracking, we simulate these decisions whenever possible in the order of their decision levels. Clearly, not all decisions are assumed due to blocking clause, and conflict or contradiction to the previous decision will happen. It is this point where a solver continues search, and we will then enter into the conflict resolution or the decide stage.

4.2. Non-blocking Solvers

4.2.1. Overview

We give a basic idea for an AllSAT procedure without the aid of blocking clauses. Like blocking procedure, we modify Algorithm 1. A main feature is to employ chronological backtracking instead of non-chronological backtracking. The chronological backtracking used here is a bit different from the ordinary one described in Section 3.2. As shown in Fig. 3, only differences are to insert the flipped decision in and register it to an implication graph so that it has no incomming arc. This is because there is no reason that implied flipped decisions because of the absence of blocking clauses.

We say that a literal, seen as an assignment, has NULL antecedent if it has no reason that implied it and there is no incomming arc in an implication graph. The chronological backtracking given above is hereafter abbreviated as BT for convenience, and it is performed by the function . We collectively call a number of procedures for AllSAT solving based on BT non-blocking procedure in contrast to blocking procedure presented in Section 4.1. If there is no fear of confusion, chronological backtracking always means BT.

Input: an assignment , the current decision level .
Output: updated objects , .
1 the decision assignment of level ;
2 ;
3 ;
4 , where is the opposite value from ;
5 return , ;
ALGORITHM 3 Chronological backtracking in non-blocking procedure, where denotes the decision level of .
(a) Solution Found (b) Conflict (c) Conflict
Figure 4. Snapshots of a non-blocking solver’s state in the three different cases (a),(b), and (c). Assignments are given from left to right, top to bottom, separated by line for each level. Integers specify assignments such that the variable is assigned the value if and if . Decision assignments have an asterisk as a superscript. Non-decision assignments with NULL antecedent have an asterisk as a subscript.

An important point in this approach is how we make BT compatible with conflict-driven clause learning. Consider a conflict graph in clause learning phase. Due to BT, a conflict graph may contain several roots, i.e. assignments with NULL antecedent, in the same decision level. See for example the literals and in Fig. 5. Since an ordinary first UIP scheme commonly assumes a unique root in the same decision level, implementations based on that assumption get stuck in non-decision literals with NULL antecedent. To resolve this problem, two techniques are presented later.

Example 9.

Look at Fig. 4. (a) All variables are assigned values without conflict, which means a solution is found. (b) Following BT, all assignments of level are canceled and the flipped decision is inserted as a non-decision assignment at level . Since no propagation takes place, a new decision is made and a subsequent propagation results in conflict. The decision is the only one that has NULL antecedent in the current decision level, and ordinary first UIP scheme suffices in this case. (c) From the conflict we have just met, the conflict clause is learnt, and a solver backtracks to level . 555In this case, there is no oppotunity to rediscover solutions by ordinary chronological backtracking, and hence the flipped decision is not inserted. This technique is explained in more detail when non-chronological backtracking with level limit is introduced. The assignment is implied by the conflict clause. A subsequent propagation results in conflict again. This time, there is a non-decision assignment with NULL antecedent in the same decision level.

Input: a CNF formula , an empty variable assignment .
Output: all satisfying assignments.
  /* Decision level */
  /* Limit level */
1 while true do
       /* Deduce stage */
2      if conflict happens then
3           if then halt;
            /* Resolve stage */
5           else
6                if all variables are assigned values then
7                     report ;
8                     if then halt;
9                     ;
10                     ;
12                     else
                           /* Decide stage */
13                          select an unassigned variable and a value ;
14                          ;
16                          end if
18                          end if
20                          end while
ALGORITHM 4 Non-blocking procedure with decision level-based first UIP scheme.

A major advantage of non-blocking approach is that no matter how many solutions exist, a performance of unit propagation does not deteriorate thanks to the absence of blocking clauses. Instead, it has to find total satisfying assignments one by one. Hence, there is a limit in the number of solutions to be generated in a realistic mount of time.

4.2.2. Sublevel-based First UIP Scheme

Grumberg et al. introduced the notion of sublevels and presented a sublevel-based first UIP scheme that is compatible with non-blocking approach [23]. A basic idea is to divide a single decision level into sublevels. Specifically, a new sublevel is defined whenever BT is performed, and sublevels are undefined as their decision levels are undefined. An ordinary first UIP scheme can then be applied in the current sublevel.

A conflict clause obtained by this approach may contain many literals that are below the current sublevel yet in the same decision level. Among literals in that conflict clause, those with NULL antecedent are necessary for avoiding rediscovery of solutions and can not be removed if exist, however it is expected that other literals are reduced further.

4.2.3. Decision Level-based First UIP Scheme*

We present an alternative first UIP scheme that need not require sublevels. Our scheme can be realized with a small modification: it is simply not to stop at literals with NULL antecedent and attempt to find the first UIP in the current decision level. A specific procedure is to traverse a conflict graph from in the reverse order of implications and construct a conflict clause , repeating the following procedure until the first UIP appears:

  1. if the current literal is below the current decision level, add the negated literal to and do not go up through the incomming arcs of the current literal;

  2. if the current literal has NULL antecedent, add the negated literal to ;

  3. for the other case, go up through the incoming arcs of the current literal if their source vertices have not yet been visited.

After the first UIP is found, add the negated literal to .

Compared to the sublevel-based first UIP scheme, the decision level-based scheme might be considered better. First of all, it is simple. Secondly, a conflict clause contains unique literal from the current decision level except for those with NULL antecedent. However, it should be noted that conflict clauses obtained by the decision level-based scheme are not necessarily smaller, because the unique implication point is further from and thus a conflict clause may contain more literals below the current decision level. A pseudo code for the modified scheme is omitted, because modification would be straightforward.

Example 10.

Continuing Example 9, consider the conflict case (c) in Fig. 4. Figure 5 illustrates the difference of the two schemes: the sublevel-based scheme finds as a first UIP and learns as a conflict clause, while the decision level-based scheme finds as a first UIP and learns as a conflict clause. Note that the conflict clause in either case does not become unit after backtracking, though it does not menace algorithmic correctness because of the flipped decision .

Remark 3.

Recall that when the function is performed, an assignment to its unit literal is set to the current assignment function . In non-blocking procedure, a conflict clause is not necessarily unit as seen in Example 10. Hence, the function in non-blocking procedure only adds a conflict clause, and it does not consider an induced assignment.

Algorithm 4 is a pseudo code for the non-blocking approach using clause learning with the decision level-based first UIP scheme. Recall that the function is to backtrack chronologically, following Algorithm 3. There are some choices for it, where a generic function, named , is called. A simple way of realizing the resolve stage is to perform clause learning, based on either first UIP scheme and then perform BT. More elaborate methods are introduced later, with which can also be replaced. Since the variable is used in one of those methods, it is introduced when needed.

Figure 5. An implication graph for the conflict case (c) in Fig. 4.

4.2.4. Conflict Directed Backjumping

Grumberg et al. augmented non-blocking approach with conflict resolution by means of a restricted non-chronological backtracking [23]. Their backtracking method can be considered as a form of conflict directed backjumping (CBJ for short). CBJ has been studied as one of tree search algorithms for constraint satisfaction problem [48] [13] [17].

A basic idea is described below. Consider the scenario where conflict happens at decision level , and after backtracking to level , conflict happens again. In this case, obtain a conflict clause from the former conflict, while from the latter conflict, obtain a conflict clause so that its UIP is , where is the first UIP in the former analysis. Perform resolution of and and obtain a resulting clause . After that, backtrack to the level preceding the highest level in .

A pseudo code is given in Algorithm 5, which is almost faithfully rephrased in our setting from the code given in the literature [23]. The call of at the end of the while loop was not explicitly written in the original code, which we consider necessary. At this call, unit propagation considers the effect of the recent flipped decision inserted as a result of . Note that the effect of an assignment implied by the recent conflict clause is not considered here, because our non-blocking procedure assumes that the function simply records a conflict clause and does not insert an implied assignment, which is separately inserted at line 12. The halt means that non-blocking procedure is halted too.

Input: a CNF formula , an assignment , the current decision level .
Output: updated objects , .
1 an empty stack;
2 while true do
3      if conflict happens then
4           if then halt;
5           ;
6           push the learnt conflict clause into ;
7           ;
9           else if is not empty then
10                the clause popped from ;
11                if is a unit clause then
12                     the unit literal in ;
13                     add , seen as an assignment, to so that it has antecedent ;
14                     ;
15                     if conflict happens then
16                          if then halt;
17                          the conflict clause from the recent conflict with as UIP;
18                          the resolution of and ;
19                          push into ;
20                          the highest level in ;
21                          ;
23                          end if
25                          end if
27                          else
28                               break;
30                               end if
31                               ;
33                               end while
34                               return , ;
ALGORITHM 5 Conflict resolution based on conflict-directed backjumping

4.2.5. Non-chronological backtracking with Level Limit And Its Combination with CBJ*

We present an alternative conflict resolution by means of non-chronological backtracking with backtrack level limit. To our knowledge, this method was first presented by [22], though it was in the context of answer set programming. We thus would like to import their idea to our non-blocking procedure.

Since non-blocking procedure does not record blocking clauses, it must not backtrack to arbitrary level, even though a backtracking level is one that is legitimately derived from a conflict clause. However, we can do this if there is no opportunity of rediscovering found solutions from the derived level. We use the variable that holds a ”safe” level to be backtracked, that is, the first level at which the current assignment and the previous satisfying assignment differ.

A pseudo code is given in Algorithm 6. We call this approach non-chronological backtracking with level limit, denoted by BJ, where the underline means a backtrack level is limited. Since is always less than or equal to , BT is performed if and only if . If , backtracking does not entail inserting a flipped decision (see Example 9). This is because implies no solution yet found, and hence no opportunity to rediscover solutions by backtracking.

Input: a CNF formula , an assignment , the current decision level , a level limit .
Output: updated objects , , .
  /* Diagnose stage */
1 if then
2      if then ;
3      ;
4      ;
6      else
7           ;
8           ;
10           end if
11           return , , ;
ALGORITHM 6 Conflict resolution based on non-chronological backtracking with level limit

We furthermore present a combination666It is mentioned [22] that a combination of conflict-directed backjumping and non-chronological backtracking is proposed. However, their pseudo code almost corresponds to Algorithm 6 and it is different from that stated in this paper. of BJ and CBJ. This is obtained by replacing the else part in BJ, in which BT is performed, with CBJ: that is,

  1. if is less than the current decision level, then perform BJ;

  2. otherwise, perform CBJ.

Step 2 is selected only when equals the current decision level: in other words, when conflict happens just after the current assignment diverged from the previous satisfying assignment. Hence, BJ is preferentially applied when one or more decisions from the diverging point are made. This is designed with expectation that more decisions are made from the diverging point, more effectively BJ prunes search space. Since BJ is likely to be more frequently applied, this approach is denoted by BJ+CBJ.

4.2.6. Implementation

We implemented 8 programs based on non-blocking procedure according to which of the two first UIP schemes is selected and which of the conflict resolution methods, i.e. BT, BJ, CBJ, or BJ+CBJ, is selected, where BT means performing BT after clause learning.

4.3. Formula-BDD Caching Solvers

4.3.1. Overview

Formula caching refers to a number of techniques to memorize formulae to avoid recomputation of subproblems [7]. Examples include a caching technique in probabilistic planning [39], conflict clauses in SAT [6] [42], component caching and other cachings in #SAT [5] [4], and blocking clauses in AllSAT [43].

Another type of formula caching in which formulae are associated with propositional languages such as FBDD, OBDD, and a subset of d-DNNF has been studied in the context of knowledge compilation [28]. Their work revealed a correspondence between exhaustive DPLL search and propositional languages. They also proposed speeding up compilation by exploiting techniques of modern SAT solvers through the correspondence. Although exhaustive DPLL search is simply used for efficiency in their compilation approach, compilation in turn can contribute to speeding up exhaustive DPLL search. Actually, if a CNF formula is compiled into a BDD, all satisfying assignments can be generated simply by traversing all possible paths from the root to the sink node . This seems like taking a long way around to AllSAT solving, however thanks to the caching mechanism, recomputation of many subproblems can be saved. A connection to AllSAT was mentioned, however their primary concern is on the compilation to suitable languages for required queries, not restricted to AllSAT. To our knowledge, comparisons have been conducted only between various compilers. An application to an AllSAT solver itself was more explicitly mentioned in the literature [55] and a compiler-based AllSAT solver is released. However, comparisons with other AllSAT solvers have not been conducted yet and its power remains unknown. Similar caching techniques appear in other areas such as preimage computation in unbounded model checking [51] [37] [34], satisfiability [45], and discrete optimization [3] [8].

The paper only deals with the caching method that records pairs of formulae and OBDDs, which we call formula-BDD caching. A formula-BDD caching can be embedded in either blocking procedure or non-blocking procedure777Only the combination with blocking procedure has been presented in the past work [27] [55].. This is done without almost any loss of optimizations employed in an underlying procedure. An exception is that variables must be selected in a fixed order at the decide stage. This effect is far from negligible in terms of efficiency, as is well-recognized in a single solution SAT. It is, however, confirmed in experiments that formula-BDD caching solvers exhibit a quite good performance on the whole, and it provides an efficient solution method for instances that have a huge number of solutions and can not possibly be solved by other means.

4.3.2. Caching Mechanism

We give a basic idea of formula-BDD caching, using a simple BDD construction method with formula-BDD caching. This method is elaborated later by implementing on top of a SAT solver. We first introduce terminology. A subinstance of in an assignment is the CNF formula derived from by applying all assignments defined in to . The current subinstance refers to the subinstance induced by the current assignment.

Consider the following procedure with a CNF formula and an empty variable assignment as initial arguments:

  1. if an unsatisfied clause exists in the current subinstance, then return ;

  2. if all variables are assigned values, then return ;

  3. the smallest index of an unassigned variable;

  4. the result obtained by a recursive call with and ;

  5. the result obtained by a recursive call with and ;

  6. return a node with the label , the references to LO child and to HI child ;

Since different assignments can yield subinstances that are logically equivalent, we want to speed up the procedure by applying dynamic programming. To do this, we need to quickly decide if the current subinstance is solved. If it is unsolved, we compute a BDD for all solutions of the instance and memorize it, associating with the instance. Otherwise, the result is obtained in a form of BDD and recomputation is avoided.

However, this approach involves the equivalence test of CNF formulae, which is computationally intractable, as it includes satisfiability testing. Hence, we consider a weaker equivalence test. That is, we encode subinstances into formulae so that if two subinstances are not logically equivalent, then the encoded formulae are not identical. To decide if the current subinstance is solved, it suffices to search the encoded formula in the set of registered formula-BDD pairs. All requirements for formula-BDD caching to work is simply the sentence in italic above, and any encoding that meets it will do. It should be noted that our test is sound in that acceptance always is a correct decision, however if we prioritize efficiency of encoding excessively, logically equivalent subinstances are very likely to result in non-identical formulae, i.e., a wrong decision.

Examples of formula-BDD cachings include those induced by cutsets and separators [27], defined below, and a variant of cutsets [55].

Definition 1.

The -th cutset of a CNF formula is the set of clauses in such that has literals with their underlying variables and satisfying . The cutwidth of is the maximum size of a cutset of .

Definition 2.

The -th separator of a CNF formula is the set of variables such that some clause in the -th cutset has a literal with its underlying variable satisfying . The pathwidth of is the maximum size of a separtor of .

Figure 6. A CNF formula (left )and an OBDD for the CNF (right), where arcs to the sink node is omitted. Cutsets are associated with arcs, where underlined clauses mean they are satisfied.
Example 11.

Look at the CNF formula illustrated in Fig. 6. The -rd cutset of consists of and , while the -rd separator of consists of , , and . The cutwidth and the pathwidth of are both .

The following proposition states that clauses (and variables) in cutsets (and separators) meet the requirement of formula-BDD caching, respectively. Proof is omitted. See [27].

Proposition 1.

Let be a CNF formula, where variables are ordered according to their indices. Let and be assignments with the -th or less variables assigned values and other variables unassigned.

  1. If satisfied clauses in the -th cutset of in coincide with those in , then the subinstance in is logically equivalent to that in .

  2. If variables assigned the value in the -th separator of in coincide with those in , then the subinstance in is logically equivalent to that in .

Example 12.

Figure 6 illustrates an OBDD constructed using cutsets as formula-BDD caching. Cutsets are associated with arcs, and satisfied clauses in them are underlined. If two arcs have the same set of satisfied clauses, Proposition 1 implies that their target vertices can be merged safely.

It should be noted that our weaker equivalence test may reject logically equivalent subintances; the subgraphs in a constructed BDD that correspond to those subinstances are not merged. This means that a constructed BDD is not fully reduced, i.e., an OBDD, though the OBDD give in Fig. 6 happens to be fully reduced.

An important point of the formula-BDD caching approach is how to balance quality with efficiency in our weaker equivalence test. The quality here refers to how many correct decisions are made. Theoretically, it holds that a correct decision of the separator approach always implies a correct decision of the cutset approach. On the other hand, the efficiency refers to how much time is taken to create formulae from subinstances, which substantially amounts to evaluating clauses and variables in cutsets and separators, respectively. In terms of efficiency, evaluating clauses in a cutset would require time linear to the total size of clauses due to lazy evaluation mechanism if it is implemented on top of modern SAT solvers. On the other hand, evaluating variables in a separator requires time linear to the number of variables.

From the argument above, we can say that for instances with small cutwidth, evaluation cost of cutset is negligible compared to separator and hence cutset is a better choice, and for instances with many clauses, separator should be used instead.

4.3.3. Embedding Formula-BDD Caching in AllSAT Procedure*

We demonstrate how to embed formula-BDD caching in a concrete AllSAT procedure. We take non-blocking procedure as an example. To our knowledge, this is the first time that the combination is presented. The other combination, i.e. blocking procedure with formula-BDD caching, is omitted.

We have assumed so far that conflict clauses (and blocking clauses in blocking procedure) are added to a CNF , but from now on we will assume that they are separately maintained from . This makes unchanged throughout the execution of our AllSAT procedure. Accordingly, the cutset and the separator in each level are unchanged too.

Algorithm 7 is a pseudo code of non-blocking procedure with formula-BDD caching embedded. Formula-BDD caching mechanism consists of the encode stage, the extend stage, and the enroll stage.

At the encode stage, the function receives a CNF , the current assignment , and the index of the largest assigned variable. It computes a formula for the current subinstance. No specific encoding is presented here. See Section 4.3.2 for examples of encoding. Note that if all variables are assigned values, then must be , and hence we have . In this case, let the function return , which is the special formula representing true.

At line 15, we search an entry with the key in , where holds registered formula-BDD pairs. If it exists, the current subinstance is already solved, and the result is the OBDD associated with the key, which appears as a subgraph of . Hence, at the extend stage, the function augments an OBDD by adding a path from the root of to the root of , following the current assignment . It returns the pair of the extended OBDD and the added path. Since this stage is straightforward, we omit a pseudo code of it.

At the enroll stage, we associate formulae for solved subinstances with the corresponding OBDDs and insert these formula-BDD pairs to . To do this, important points are how to identify when subinstances are solved and how to find their OBDDs. Let be a subinstance with the smallest unassigned variable . Since unit propagation is performed at the beginning of each repetition, without loss of generality we assume that is a decision variable. This means that a formula has been made from , thereby is in . Let and be the assignment and the decision level at this point. Without loss of generality we assume that there exist one or more solutions extending , because otherwise, the OBDD for is not created. After adding at least one path to , we have the node that corresponds to the decision at variable . Clearly, this node is reachable from the root of through the path following , and it is the root of the OBDD for . When all solutions for are found, to go out of the exhausted search space, backtracking to a lower level than is performed, which is directly triggered by an occurrence of conflict implying no solution left or by the discovery of the last solution. Backtracking to a lower level can also be performed without exhausting all solutions, however this case only happens when no solution of is yet found. Hence, we can distinguish them. Summarizing the above, if a backtracking level is less than and at least one solution for is found, then is solved, and the root of the OBDD for is located at the end of the path following , which is a part of the assignment just before backtracking.

The function is in charge of the enroll stage. With the observation above, it is called whenever backtracking is performed. A procedure for it is given as follows. We scan all nodes in the most recently added path until the assignment at taken in contradicts to the current assignment. Note that scanned nodes correspond to subinstances such that turns out to be satisfiable and the assignment that induced is a part of the current assignment. For each scanned node , we test if a backtracking level is less than the decision level of and a formula was made888 This is equivalent to finding an entry in , where is the label of . from . If the test is passed, the pair of and is inserted into , where is the label of .

The function behaves in the same way as except that and are updated each time backtracking is performed, as in lines 19-20.

Example 13.

Figure 7 illustrates how Algorithm 7 constructs an OBDD for the CNF formula given in Fig. 6.

Figure 7. Progress of OBDD construction. As paths are added one by one, an OBDD is augmented from left to right. In each step, thick arcs represent a path that is about to be added, and gray nodes mean that the subintances corresponding to them were solved.
Input: a CNF formula , an empty variable assignment .
Output: an OBDD for all satisfying assignments.
  /* Decision level */
  /* Limit level */
  /* OBDD */
  /* Set of formula-BDD pairs */
  /* Set of formulae with indices */
  /* Sequence of BDD nodes */
1 while true do
       /* Deduce stage */
2      if conflict happens then
3           if then return ;
            /* Resolve and enroll stage */
5           else
6                ;
                 /* Encode stage */
7                if an entry with key exists in then
8                     the OBDD node associated with in ;
                      /* Extend stage */
9                     if then return ;
                      /* Enroll stage */
10                     ;
11                     ;
12                     ;
14                     else
15                          ;
                           /* Decide stage */
16                          select a value ;
17                          ;
19                          end if
21                          end if
23                          end while
ALGORITHM 7 Non-blocking procedure with formula-BDD caching, where denotes the decision level of a variable .

4.3.4. Refreshing OBDDs*

A constructed OBDD may become too large to be stored on memory, though it would be in practice much better in size than the list representation of all solutions and in the worst case equal to it except for constant factor. We present a simple technique to resolve this problem. Let be the number of variables. We introduce a threshold of an OBDD size, where . Insert the following procedure after an BDD is extended and backtracking is performed. If the size of an OBDD is larger than or equal to , then the current OBDD is dumped to a file in a secondary storage, and all objects , , , and of formula-BDD caching mechanism are refreshed with initial states. Since formula-BDD caching is almost independent of the underlying non-blocking procedure, after refreshing, the procedure simply attempts to examine unprocessed assignments with formula-BDD caching empty.

4.3.5. Implementation

We implemented 2 programs on each non-blocking procedure and 2 programs on each blocking procedure according to which formula-BDD caching is selected, cutset or separator.

5. Experiments

5.1. Implementation And Environment.

All solvers are implemented in C on top of MiniSat-C v1.14.1 [19]. Clasp 3.1.2 was taken from Potassco project999Potassco, the Potsdam Answer Set Solving Collection, bundles tools for Answer Set Programming developed at the University of Potsdam,, accessed on 13rd Sept., 2015.. As far as we are aware, clasp [21], picosat [10], and relsat [6] are only SAT solvers which support the enumeration of all solutions. Among them, we used clasp for the comparison, because it achieved better performance than picosat, and relsat does not support quiet mode in solution generation and generated solutions may be too large to be stored.

All experiments were performed on 2.13GHz Xeon®E7- 2830 with 512GB RAM, running Red Hat Enterprise Linux 6.3 with gcc compiler version 4.4.7. In the execution of each AllSAT solver, time limit and memory limit were set to 600 seconds and 50GB, respectively. If either limit is exceeded, the solver is enforced to halt. All solvers simply touch found solutions and never output them.

The types of compared solvers are a blocking solver, a non-blocking solver, a formula-BDD caching solver, and clasp. The first three types have some variations according to which techniques are used (see the end of each subsection in Section 4). Among solvers of the same type, we selected a solver with the most solved instances. The selected solvers, called representative solvers, are as follows.

  • Blocking_NoSimple_Cont: the blocking solver with simplification unselected and continuation selected.

  • NonBlocking_DLevel_BJ: the non-blocking solver with decision level first UIP scheme and non-chronological backtracking with level limit both selected.

  • BDD_Cut_NonBlocking_DLevel_BJ: the formula-BDD caching solver with cutset caching selected and it is implemented on top of NonBlocking_DLevel_BJ.

Throughout the section, if there is no fear of confusion, they are abbreviated as Blocking, NonBLocking, and BDD. Notation for solvers with other configurations is introduced in the same way.

It is known that variable orderings significantly affect the performance of BDD compilation. Hence, we used the software MINCE version 1.0 [2] to decide a static variable order before the execution of formula-BDD caching solvers. The execution of MINCE failed for some instances, and for that case, we used the original order. The time required for deciding a variable order is included. Although for some instances time limit exceeded in preprocessing, it was negligible for many instances.

5.2. Problem Instances.

We used total 2867 CNF instances (all satisfiable), which are classified as follows.

Among instances released in each repository, we selected all instances such that satisfiability could be decided in 600 seconds by either one of the SAT solvers clasp 3.1.2, glucose4, minisat 2.2, and minisat 1.3, and its result was satisfiable. For satlib and sc14, random instances are excluded.

5.3. Comparison of Running Time.

Figure 8 shows a cactus plot of representative solvers. Solved instances are ranked with respect to the times required to solve them. Each point represents a solved instance with its rank (the horizontal coordinate) and the required time (the vertical coordinate). Since one wants to solve as many instances as possible in a given amount of time, it is thought that gentler the slope of plotted points is, more efficient a solver is. The formula-BDD caching solver clearly outperforms the other solvers. It is then followed by the non-blocking solver, clasp, and the blocking solver in this order.

Figures 910, and 11 depict differences between solvers of the same types. From Figure 9, we can observe that continuation of search is effective yet simplification degrades performance. For some instances, simplification enables a solver to find a large number of solutions, however such instances are limited and the current implementation is not powerful enough to make it possible to solve instances that can not be handled without simplification. Figure 10 has a narrower horizontal range than the other figures. This is because non-blocking solvers exhibit quite similar performance and they can not be distinguished otherwise. It is surprising that BT is almost as efficient as the other elaborated backtracking methods. Decision level-based scheme is equal to or more efficient than sublevel-based scheme. Figure 11 shows that non-blocking procedure is clearly better as an underlying solver in which caching mechanism is embedded, while there is almost no difference between caching methods.

Figure 8. Cactus plot of representative solvers with respect to running time.
Figure 9. Cactus plot of blocking solvers.
Figure 10. Cactus plot of non-blocking solvers, where the horizontal scale is narrowed to make the difference clear.
Figure 11. Cactus plot of formula-BDD caching solvers.

5.4. Comparison of Maximum Memory Usage.

Figure 12 shows a cactus plot of the maximum memory usage. Solved instances are in turn ranked with respect to the maximum memory usage. Each point then represents a solved instance with its rank (the horizontal coordinate) and the required memory (the vertical coordinate). In terms of memory consumption, the formula-BDD caching solver is the worst, while the non-blocking solver and clasp exhibit a stable performance. The rapid increase in the curves of the non-blocking solver and clasp is due to large CNF formulae. Although the formula-BDD caching solver consumes much memory, these days it is not unusual that even laptop computers have several giga bytes of RAM, and advantage of the formula-BDD caching solver is not impaired so much.

Figure 12. Cactus plot of representative solvers with respect to the maximum memory usage, where the vertical axis is in logarithmic scale.

5.5. Comparison of Scalability in The Number of Solutions.

As shown in Table 1, each representative solver has the following limit in the number of solutions within 600 seconds time limit .

  • Blocking: one million solutions.

  • Clasp: one hundred million solutions.

  • Non-Blocking: ten billion solutions.

  • BDD: more than one quadrillion solutions.

Blocking clasp NonBlocking BDD
38 36 39 37
11 11 11 11
417 417 417 416
682 682 682 682
408 408 408 408
82 134 133 134
0 86 82 86
0 56 87 91
0 0 68 83
0 0 23 92
0 0 0 83
0 0 0 44
0 0 0 29
0 0 0 19
0 0 0 8
total 1,638 1,830 1,950 2,223
Table 1. Distribution of solved instances with respect to the number of solutions.

5.6. Distribution of Solved Instances over Instance Series.

Table 2 shows the distribution of solved instances over all instance series. In almost all series, differences in the number of solved instances between solvers can be explained by their scalability.

Exceptions are the sc14 instances. They are clearly harder instances. All solvers were unable to find even one solution in many instances. For sc14-crafted series, solved instances are those of less than 10 solutions with a few exceptions of BDD. Table 3 shows the distribution of all sc14 instances including unsolved instances. Although clasp could find relatively many solutions, the other solvers could only find less than 10 solutions for a majority of instances. In terms of the ability to find solutions for sc14 instances, we could say that the best solver is clasp; the blocking solvers and the non-blocking solvers are a tie, both ranked at the second; the formula-BDD caching solver is no match for those instances, which is due to fixed variable ordering.

The favorite ranges of instances for representative solvers are illustrated in Fig. 13 according to the two factors: hardness of instances and the numbers of solutions instances have. Each solver is placed in such a way that the vertical position corresponds to the ability to find solutions for sc14 instances, where the blocking solver is above the non-blocking solver because of more benefit from a SAT solver, and the horizontal position corresponds to the scalability in the number of solutions. Hence, each indicated range refers to instances leftward or downward from it as well as those within it. It should be noted that shrinking the vertical axis would be more suitable for real performance, because in reality all solvers, on the whole, exhibits a poor performance over hard instances.

Blocking clasp NonBlocking BDD
ais (4) 4 4 4 4
bmc (13) 0 0 0 1
bw (7) 7 7 7 6
Flat125-301 (100) 4 51 94 100
Flat150-360 (101) 0 12 61 101
Flat175-417 (100) 0 2 15 98
Flat200-479 (100) 0 0 3 77
Flat75-180 (100) 87 100 100 100
flat (1,199) 1,136 1,195 1,199 1,199
gcp (1) 0 0 0 0
hanoi (2) 2 2 2 2
inductive (41) 1 5 9 8
logistics (4) 0 0 0 1
parity (20) 20 20 20 20
qg (10) 10 10 10 10
ssa (4) 0 0 0 0
SW100-8-0 (100) 0 0 0 0
SW100-8-1 (100) 0 0 0 0
SW100-8-2 (100) 0 0 0 0
SW100-8-3 (100) 0 0 0 0
SW100-8-4 (100) 0 1 5 68
SW100-8-5 (100) 55 100 95 100
SW100-8-6 (100) 100 100 100 100
SW100-8-7 (100) 100 100 100 100
SW100-8-8 (100) 100 100 100 100
SW100-8-p0 (1) 1 1 1 1
sc14-app (56) 0 0 0 0
sc14-crafted (65) 5 3 6 5
iscas (39) 6 17 19 22
total (2,867) 1,638 1,830 1,950 2,223
Table 2. Distribution of solved instances over instance series, where the number of instances in each series is enclosed in parenthesis.
Blocking clasp NonBlocking BDD
54 22 58 89
25 24 22 19
8 0 0 0
12 1 0 0
3 5 1 0
4 20 1 0
15 27 2 0
0 21 0 2
0 1 0 0
0 0 14 0
0 0 23 0
0 0 0 0
0 0 0 1
0 0 0 1
0 0 0 0
0 0 0 9
total 121 121 121 121
Table 3. Distribution of all sc14 instances including unsolved instances according to the number of found solutions within time limit.
Figure 13. Favorite ranges of instances for representative solvers.

5.7. Comparisons to Publicly Available #SAT Solvers

We also have a comparison with publicly available #SAT solvers, sharpSAT version 1.1 [52], c2d version 2.20 [16] and relsat version 2.20 [6], to check the performance of our developed ALLSAT solvers for #SAT. In the result for the same 2867 instances, relsat, sharpSAT, c2d and BDD_Cut_NonBlocking_DLevel_BJ solved 2191, 2196, 2196 and 2223 respectively. Since relsat supports solution count, it was used as a solution counter in this comparison. So, even for #SAT, the formula-BDD caching solver shows better performance.

6. Conclusion

We surveyed and discussed major techniques of existing AllSAT solvers. We classified the types of solvers into a blocking solver, a non-blocking solver, and a formula-BDD caching solver. We faithfully implemented and released these solvers publicly so that other researchers can easily develop their solver by modifying our codes and compare it with existing methods. We conducted comprehensive experiments with total 2867 instances taken from SATLIB, SAT competition 2014, and ISCAS benchmarks. Apart from our implemented solvers, we used clasp, one of the few off-the-shelf softwares with solution generation support. The experiments revealed the following solver’s characteristics (600 seconds time limit). See also Fig. 13.

  • The formula-BDD caching solver is the most powerful. It has the most solved instances, including instances with more than one quadrillion solutions. The maximum memory usage amounts to several tens giga bytes in the worst case, though it is controllable by refreshing caches at the cost of a low cache hit rate. They are bad at hard instances due to fixed variable ordering.

  • The non-blocking solver is ranked at the next best, followed by clasp. The non-blocking solver and clasp can handle instances with ten billion solutions and one hundred million solutions with a low maximum memory usage (a few mega bytes to several tens of mega bytes), respectively. Although both solvers exhibit relatively a similar performance, a difference is that clasp is able to find a moderately many number of solutions from even hard instances, though it is not powerful enough to make it possible to solve instances that can not be handled by other means.

  • The blocking solver is limited to instances with one million solutions as blocking clauses deteriorate the performance of unit propagation. However, it can benefit from state-of-the-art techniques of SAT solvers as they are, thereby it is suitable for finding a small number of solutions for hard instances.

From the above, we conclude that the formula-BDD caching solver is the most superior in terms of exact AllSAT solving over various kinds of instances. However, since not all solutions are necessary in some practical applications and duplicated solutions may be allowed, it is recommended to select an appropriate solver in accordance with types of instances and applications.


  • [1] S. B. Akers. Binary decision diagrams. IEEE Trans. Comput., 27(6):509–516, June 1978.
  • [2] Fadi A. Aloul, Igor L. Markov, and Karem A. Sakallah. MINCE: A static global variable-ordering heuristic for SAT search and BDD manipulation. J. UCS, 10(12):1562–1596, 2004.
  • [3] H.R. Andersen, T. Hadzic, J.N. Hooker, and P. Tiedemann. A constraint store based on multivalued decision diagrams. In Christian Bessiére, editor, Principles and Practice of Constraint Programming, volume 4741 of Lecture Notes in Computer Science, pages 118–132. Springer Berlin Heidelberg, 2007.
  • [4] F. Bacchus, S. Dalmao, and T. Pitassi.

    Algorithms and complexity results for #sat and bayesian inference.

    In Foundations of Computer Science, 2003. Proceedings. 44th Annual IEEE Symposium on, pages 340–351, Oct 2003.
  • [5] Roberto J. Bayardo and J. D. Pehoushek. Counting models using connected components. In In Proceedings of the AAAI National Conference, pages 157–162, 2000.
  • [6] Roberto J. Bayardo, Jr. and Robert C. Schrag. Using csp look-back techniques to solve real-world sat instances. In

    Proceedings of the Fourteenth National Conference on Artificial Intelligence and Ninth Conference on Innovative Applications of Artificial Intelligence

    , AAAI’97/IAAI’97, pages 203–208. AAAI Press, 1997.
  • [7] Paul Beame, Russell Impagliazzo, Toniann Pitassi, and Nathan Segerlind. Formula caching in dpll. ACM Trans. Comput. Theory, 1(3):9:1–9:33, March 2010.
  • [8] D. Bergman, A. A. Cire, W.-J. van Hoeve, and J. N. Hooker. Discrete optimization with decision diagrams. INFORMS Journal on Computing. To appear.
  • [9] A. Biere, A. Biere, M. Heule, H. van Maaren, and T. Walsh. Handbook of Satisfiability: Volume 185 Frontiers in Artificial Intelligence and Applications. IOS Press, Amsterdam, The Netherlands, The Netherlands, 2009.
  • [10] Armin Biere. Picosat essentials. JSAT, 4(2-4):75–97, 2008.
  • [11] Jörg Brauer, Andy King, and Jael Kriener. Existential quantification as incremental sat. In Ganesh Gopalakrishnan and Shaz Qadeer, editors, Computer Aided Verification, volume 6806 of Lecture Notes in Computer Science, pages 191–207. Springer Berlin Heidelberg, 2011.
  • [12] R.E. Bryant. Graph-based algorithms for boolean function manipulation. Computers, IEEE Transactions on, C-35(8):677–691, Aug 1986.
  • [13] Xinguang Chen and Peter van Beek. Conflict-directed backjumping revisited. J. Artif. Int. Res., 14(1):53–81, March 2001.
  • [14] Edmund Clarke, Daniel Kroening, Natasha Sharygina, and Karen Yorav. Predicate abstraction of ansi-c programs using sat. Formal Methods in System Design, 25(2-3):105–127, 2004.
  • [15] Y. Crama and P.L. Hammer. Boolean Functions: Theory, Algorithms, and Applications. Encyclopedia of Mathematics and its Applications. Cambridge University Press, 2011.
  • [16] Adnan Darwiche. New advances in compiling CNF into decomposable negation normal form. In Proceedings of the 16th Eureopean Conference on Artificial Intelligence, ECAI’2004, including Prestigious Applicants of Intelligent Systems, PAIS 2004, Valencia, Spain, August 22-27, 2004, pages 328–332, 2004.
  • [17] Rina Dechter and Daniel Frost. Backjump-based backtracking for constraint satisfaction problems. Artificial Intelligence, 136(2):147 – 188, 2002.
  • [18] Thomas Eiter, Kazuhisa Makino, and Georg Gottlob. Computational aspects of monotone dualization: A brief survey. Discrete Applied Mathematics, 156(11):2035 – 2049, 2008. In Memory of Leonid Khachiyan (1952 - 2005 ).
  • [19] Niklas Eén and Niklas Sörensson. An extensible sat-solver. In Enrico Giunchiglia and Armando Tacchella, editors, Theory and Applications of Satisfiability Testing, volume 2919 of Lecture Notes in Computer Science, pages 502–518. Springer Berlin Heidelberg, 2004.
  • [20] M.K. Ganai, A. Gupta, and P. Ashar. Efficient sat-based unbounded symbolic model checking using circuit cofactoring. In Computer Aided Design, 2004. ICCAD-2004. IEEE/ACM International Conference on, pages 510–517, Nov 2004.
  • [21] Martin Gebser, Benjamin Kaufmann, André Neumann, and Torsten Schaub. clasp: A conflict-driven answer set solver. In Chitta Baral, Gerhard Brewka, and John Schlipf, editors, Logic Programming and Nonmonotonic Reasoning, volume 4483 of Lecture Notes in Computer Science, pages 260–265. Springer Berlin Heidelberg, 2007.
  • [22] Martin Gebser, Benjamin Kaufmann, André Neumann, and Torsten Schaub. Conflict-driven answer set enumeration. In Chitta Baral, Gerhard Brewka, and John Schlipf, editors, Logic Programming and Nonmonotonic Reasoning, volume 4483 of Lecture Notes in Computer Science, pages 136–148. Springer Berlin Heidelberg, 2007.
  • [23] Orna Grumberg, Assaf Schuster, and Avi Yadgar. Memory efficient all-solutions sat solver and its application for reachability analysis. In AlanJ. Hu and AndrewK. Martin, editors, Formal Methods in Computer-Aided Design, volume 3312 of Lecture Notes in Computer Science, pages 275–289. Springer Berlin Heidelberg, 2004.
  • [24] Tias Guns, Siegfried Nijssen, and Luc De Raedt. Itemset mining: A constraint programming perspective. Artificial Intelligence, 175(12–13):1951 – 1983, 2011.
  • [25] Aarti Gupta, Zijiang Yang, Pranav Ashar, and Anubhav Gupta. Sat-based image computation with application in reachability analysis. In Jr. Hunt, WarrenA. and StevenD. Johnson, editors, Formal Methods in Computer-Aided Design, volume 1954 of Lecture Notes in Computer Science, pages 391–408. Springer Berlin Heidelberg, 2000.
  • [26] Jiawei Han, Hong Cheng, Dong Xin, and Xifeng Yan. Frequent pattern mining: current status and future directions. Data Mining and Knowledge Discovery, 15(1):55–86, 2007.
  • [27] Jinbo Huang and Adnan Darwiche. Using dpll for efficient obdd construction. In HolgerH. Hoos and DavidG. Mitchell, editors, Theory and Applications of Satisfiability Testing, volume 3542 of Lecture Notes in Computer Science, pages 157–172. Springer Berlin Heidelberg, 2005.
  • [28] Jinbo Huang and Adnan Darwiche. The language of search. J. Artif. Intell. Res. (JAIR), 29:191–219, 2007.
  • [29] Saïd Jabbour, Jerry Lonlac, Lakhdar Sais, and Yakoub Salhi. Extending modern SAT solvers for models enumeration. In Proceedings of the 15th IEEE International Conference on Information Reuse and Integration, IRI 2014, Redwood City, CA, USA, August 13-15, 2014, pages 803–810, 2014.
  • [30] Said Jabbour, Joao Marques-Silva, Lakhdar Sais, and Yakoub Salhi. Enumerating prime implicants of propositional formulae in conjunctive normal form. In Eduardo Fermé and João Leite, editors, Logics in Artificial Intelligence, volume 8761 of Lecture Notes in Computer Science, pages 152–165. Springer International Publishing, 2014.
  • [31] Said Jabbour, Lakhdar Sais, and Yakoub Salhi. Boolean satisfiability for sequence mining. In Proceedings of the 22Nd ACM International Conference on Conference on Information; Knowledge Management, CIKM ’13, pages 649–658, New York, NY, USA, 2013. ACM.
  • [32] HoonSang Jin, HyoJung Han, and Fabio Somenzi. Efficient conflict analysis for finding all satisfying assignments of a boolean circuit. In Nicolas Halbwachs and LenoreD. Zuck, editors, Tools and Algorithms for the Construction and Analysis of Systems, volume 3440 of Lecture Notes in Computer Science, pages 287–300. Springer Berlin Heidelberg, 2005.
  • [33] HoonSang Jin and Fabio Somenzi. Prime clauses for fast enumeration of satisfying assignments to boolean circuits. In Proceedings of the 42Nd Annual Design Automation Conference, DAC ’05, pages 750–753, New York, NY, USA, 2005. ACM.
  • [34] Hyeong-Ju Kang and In-Cheol Park. Sat-based unbounded symbolic model checking. Computer-Aided Design of Integrated Circuits and Systems, IEEE Transactions on, 24(2):129–140, Feb 2005.
  • [35] Donald E. Knuth. The Art of Computer Programming, Volume 4, Fascicle 1: Bitwise Tricks & Techniques; Binary Decision Diagrams. Addison-Wesley Professional, 12th edition, 2009.
  • [36] C. Y. Lee. Representation of switching circuits by binary-decision programs. Bell System Technical Journal, 38(4):985–999, 1959.
  • [37] Bin Li, Michael S. Hsiao, and Shuo Sheng. A novel SAT all-solutions solver for efficient preimage computation. In 2004 Design, Automation and Test in Europe Conference and Exposition (DATE 2004), 16-20 February 2004, Paris, France, pages 272–279, 2004.
  • [38] Nuno Lopes, Nikolaj Bjorner, Patrice Godefroid, and George Varghese. Network verification in the light of program verification. Technical report, September 2013.
  • [39] Stephen M. Majercik and Michael L. Littman. Using caching to solve larger probabilistic planning problems. In Proceedings of the Fifteenth National Conference on Artificial Intelligence and Tenth Innovative Applications of Artificial Intelligence Conference, AAAI 98, IAAI 98, July 26-30, 1998, Madison, Wisconsin, USA., pages 954–959, 1998.
  • [40] Sharad Malik and Lintao Zhang. Boolean satisfiability from theoretical hardness to practical success. Commun. ACM, 52(8):76–82, August 2009.
  • [41] Joao Marques-Silva, Mikoláš Janota, and Inês Lynce. On computing backbones of propositional theories. In Proceedings of the 2010 Conference on ECAI 2010: 19th European Conference on Artificial Intelligence, pages 15–20, Amsterdam, The Netherlands, The Netherlands, 2010. IOS Press.
  • [42] J.P. Marques-Silva and K.A. Sakallah. Grasp: a search algorithm for propositional satisfiability. Computers, IEEE Transactions on, 48(5):506–521, May 1999.
  • [43] KenL. McMillan. Applying sat methods in unbounded symbolic model checking. In Ed Brinksma and KimGuldstrand Larsen, editors, Computer Aided Verification, volume 2404 of Lecture Notes in Computer Science, pages 250–264. Springer Berlin Heidelberg, 2002.
  • [44] A. Morgado and J. Marques-Silva. Good learning and implicit model enumeration. In Tools with Artificial Intelligence, 2005. ICTAI 05. 17th IEEE International Conference on, pages 6 pp.–136, Nov 2005.
  • [45] DoRonB. Motter and IgorL. Markov. A compressed breadth-first search for satisfiability. In DavidM. Mount and Clifford Stein, editors, Algorithm Engineering and Experiments, volume 2409 of Lecture Notes in Computer Science, pages 29–42. Springer Berlin Heidelberg, 2002.
  • [46] Keisuke Murakami and Takeaki Uno. Efficient algorithms for dualizing large-scale hypergraphs. Discrete Applied Mathematics, 170:83 – 94, 2014.
  • [47] Knot Pipatsrisawat and Adnan Darwiche. A lightweight component caching scheme for satisfiability solvers. In Theory and Applications of Satisfiability Testing - SAT 2007, 10th International Conference, Lisbon, Portugal, May 28-31, 2007, Proceedings, pages 294–299, 2007.
  • [48] Patrick Prosser. Hybrid algorithms for the constraint satisfaction problem. Computational Intelligence, 9:268–299, 1993.
  • [49] J. Qadir and O. Hasan. Applying formal methods to networking: Theory, techniques, and applications. Communications Surveys Tutorials, IEEE, 17(1):256–291, Firstquarter 2015.
  • [50] Kavita Ravi and Fabio Somenzi. Minimal assignments for bounded model checking. In Kurt Jensen and Andreas Podelski, editors, Tools and Algorithms for the Construction and Analysis of Systems, volume 2988 of Lecture Notes in Computer Science, pages 31–45. Springer Berlin Heidelberg, 2004.
  • [51] Shuo Sheng and Michael Hsiao. Efficient preimage computation using a novel success-driven atpg. In Proceedings of the Conference on Design, Automation and Test in Europe - Volume 1, DATE ’03, pages 10822–, Washington, DC, USA, 2003. IEEE Computer Society.
  • [52] Marc Thurley. sharpsat – counting models with advanced component caching and implicit bcp. In Armin Biere and CarlaP. Gomes, editors, Theory and Applications of Satisfiability Testing - SAT 2006, volume 4121 of Lecture Notes in Computer Science, pages 424–429. Springer Berlin Heidelberg, 2006.
  • [53] Takahisa Toda. Hypergraph transversal computation with binary decision diagrams. In Vincenzo Bonifaci, Camil Demetrescu, and Alberto Marchetti-Spaccamela, editors, Experimental Algorithms, volume 7933 of Lecture Notes in Computer Science, pages 91–102. Springer Berlin Heidelberg, 2013.
  • [54] Takahisa Toda. Dualization of boolean functions using ternary decision diagrams. In International Symposium on Artificial Intelligence and Mathematics, ISAIM 2014, Fort Lauderdale, FL, USA, January 6-8, 2014, 2014.
  • [55] Takahisa Toda and Koji Tsuda. Bdd construction for all solutions sat and efficient caching mechanism. In Proceedings of the 30th Annual ACM Symposium on Applied Computing, SAC ’15, pages 1880–1886, New York, NY, USA, 2015. ACM.
  • [56] Frank van Harmelen, Frank van Harmelen, Vladimir Lifschitz, and Bruce Porter. Handbook of Knowledge Representation. Elsevier Science, San Diego, USA, 2007.
  • [57] Yinlei Yu, P. Subramanyan, N. Tsiskaridze, and S. Malik. All-sat using minimal blocking clauses. In VLSI Design and 2014 13th International Conference on Embedded Systems, 2014 27th International Conference on, pages 86–91, Jan 2014.
  • [58] Shuyuan Zhang, Sharad Malik, and Rick McGeer. Verification of computer switching networks: An overview. In Supratik Chakraborty and Madhavan Mukund, editors, Automated Technology for Verification and Analysis, Lecture Notes in Computer Science, pages 1–16. Springer Berlin Heidelberg, 2012.