A Probabilistic Approach to Satisfiability of Propositional Logic Formulae

by   Reazul Hasan Russel, et al.
University of New Hampshire

We propose a version of WalkSAT algorithm, named as BetaWalkSAT. This method uses probabilistic reasoning for biasing the starting state of the local search algorithm. Beta distribution is used to model the belief over boolean values of the literals. Our results suggest that, the proposed BetaWalkSAT algorithm can outperform other uninformed local search approaches for complex boolean satisfiability problems.



There are no comments yet.


page 6


Farsighted Probabilistic Sampling based Local Search for (Weighted) Partial MaxSAT

Partial MaxSAT (PMS) and Weighted Partial MaxSAT (WPMS) are both practic...

Local Search Trajectories over S-box Space

The study of S-box properties relations is an interesting problem. In th...

Structure-Based Local Search Heuristics for Circuit-Level Boolean Satisfiability

This work focuses on improving state-of-the-art in stochastic local sear...

When Gravity Fails: Local Search Topology

Local search algorithms for combinatorial search problems frequently enc...

Generalizing Boolean Satisfiability III: Implementation

This is the third of three papers describing ZAP, a satisfiability engin...

NLocalSAT: Boosting Local Search with Solution Prediction

The boolean satisfiability problem is a famous NP-complete problem in co...

An Efficient Implementation for WalkSAT

Stochastic local search (SLS) algorithms have exhibited great effectiven...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Abstract

We propose a version of WalkSAT algorithm, named as BetaWalkSAT. This method uses probabilistic reasoning for biasing the starting state of the local search algorithm. Beta distribution is used to model the belief over boolean values of the literals. Our results suggest that, the proposed BetaWalkSAT algorithm can outperform other uninformed local search approaches for complex boolean satisfiability problems.

2 Introduction

A statement that has either a true or false value is called a proposition. Propositional logic studies the idea of building complicated propositions using simple propositions or statements connected by logical connectives. The true or false value of a compound proposition depends on the values of underlying simple propositions and their structure of connectives.

Two popular algorithms for determining the satisfiability of propositional logic are: Davis–Logemann–Loveland (DLL) algorithm and WalkSAT algorithm. DLL is a backtrack based complete algorithm and WalkSAT is an incomplete local search algorithm for determining the boolean satisfiability of the propositional logic formulae. These algorithms are unable to apply any prior belief about the assignments of the propositions. The WalkSAT

algorithm always starts and continues with a uniform distribution and is unable to update it’s belief about the variable assignment from an unsatisfiable situation, rather it repeatedly samples from the uniform distribution on each trial. The current algorithms don’t employ the belief update or probabilistic reasoning about the binary assignment of the variables. This paper addresses the problem to make the

WalkSAT algorithm smarter by updating the belief of variable assignments with each unsatisfied attempt, so that it can obtain a positive bias towards convergence and can find a satisfiable or unsatisfiable situation more efficiently.

The paper is organized as follows: section 2 discusses some of the relevant works. Section 3 presents the approach and the proposed algorithm. Section 4 presents and discusses the results obtained. Section 5 concludes the paper with some probable future works.

3 Related works

Search strategy can be divided broadly into two groups: complete and heuristic

[2]. A complete strategy considers all possible options to find a solution whereas a heuristic based strategy prunes out the less promising options to quickly reach the goal. Heuristic strategies can be deterministic or non-deterministic. A deterministic strategy (e.g. different hill climbing algorithms

) yields the same solution under the same condition and can get stuck into the local maxima or minima. A non-deterministic strategy is able to produce different solutions under the same condition and can escape from local minima or maxima by the strength of it’s stochasticity. Evolutionary algorithms, Genetic algorithms, Simulated annealing etc. utilizes the non-deterministic strategy. But these non-deterministic strategy approaches have some shortcomings like: they are not guaranteed to always find an optimal solution, can be very slow and complex, depends on many parameters that require manual tuning and finding the optimal parameters itself yields an optimization problem to solve

[6, 3].

Estimation distribution algorithms (EDA) biases the movement of the search towards the more promising candidate options by maintaining an explicit probabilistic model. EDA is an evolutionary algorithm also known as probabilistic model-building genetic algorithm that treats the optimization problem as a sequence of incremental update of probabilistic model starting from a prior belief model (e.g. uniform) [4]. The incremental bias towards the goal makes EDA a good candidate to solve complex optimization problems. There are several papers available on EDA containing its application in bioinformatics, genomics, DNA microarray classification, proteomics and so on [1]. A mixed random walk strategy is proposed in [5] for escaping from the local minima for satisfiability testing of CNF formulas. [7] presents different conflict driven learning techniques used in various SAT solvers and proposes several new learning strategies. To the best of our knowledge, we did not find a paper that applies the idea of Estimation distribution algorithms for boolean satisfiability of propositional logic, maintaining the explicit probabilistic model as a beta distribution of the binary literals.

4 Approach

In probability theory,

Bernoulli distribution

is the probability distribution of a random variable that has a value of

true with probability p and false with probability 1-p. The WalkSAT algorithm uniformly samples a true value for a proposition with probability 0.5 and a false value with probability 0.5. When a specific assignment to variables lead to an unsatisfiable situation, there is a reason to change the belief that the value of a currently assigned proposition is uniformly distributed. If a variable is assigned to true and the final outcome is unsatisfiable, we may change our belief a little that a false assignment to this proposition might have a better chance to lead to satisfiability.

1:procedure ProbabilisticWalkSAT
2:Input: clauses, a set of clauses in propositional logic
3:    p, a probability to choose random walk
4:    maxTries, number of maximum tries
5:    maxFlips, number of maximum flips
6:Output: A solution if consistent, false otherwise
8:     for 1 to maxTries do
9:         assign all variables sampling from the beta distribution
10:    of the explicit probabilistic model of EAD
11:         for 1 to maxFlips do
12:              randomly choose an unsatisfied clause c
13:              if one or more of c’s variables can be flipped while breaking
14:     nothing then
15:                  randomly chose among those
16:              else
17:                  with probability p
18:        randomly chose one of c’s variables
19:                  else
20:        randomly choose among those of c’s variables that
21:        minimize breaks.               
22:              flip the variable
23:              if formula satisfied, terminate          end for
24:         update explicit probabilistic model of beta distribution
25:   of the Random variables (literals) based on EAD      end for
Algorithm 1 Beta-WalkSAT algorithm applying Estimation distribution algorithms strategy with positive bias for starting state using explicit probabilistic model with Beta distribution

Beta distribution is a very convenient and powerful choice of priors for Bernoulli distribution. The beta distribution presents a continuous probability distribution of random variables on the interval [0,1]. Beta distribution is parametrized by two positive parameters and that controls the shape of the distribution.

The algorithm presented here assumes the literals are independent of each other, they are random variables with binary choices and models the belief of the assignment of each literal with a beta distribution. Initially it represents the prior belief about the assignment which is uniform. After each unsuccessful trial, the belief for each literal is updated with the beta distribution. Sampling for the next trial is then done from this distribution, biasing the chance towards faster convergence.

4.1 Beta-WalkSat algorithm

The Beta-WalkSat algorithm described in algorithm 1 is almost same as of the normal WalkSAT algorithm except that it maintains a beta distribution for each literal and utilizes that distribution for biasing the starting state of each trial. Lines 2-5 describes the input to the algorithm and line 6 describes the returned output.

The algorithm starts at line 8 and iterates for maxTries number of times. At the start of each trial, an initial value is assigned to each literal sampling from the beta distribution of that literal in lines 9-10. The for loop at line 11 runs for maxFlip number of times, picks a random unsatisfied clause in line 12, tries to find a literal that doesn’t break any other clause and flips that in lines 13-15. If no such literal is found, it flips a random literal with probability p in line 18, or with probability 1-p chooses a random literal that minimizes breaks. If all the clauses are satisfied, the result is returned in line 23. If not, it updates the beta distribution of each literal by increasing the value of alpha or beta based on the current assignment of the literal in lines 24-25 and proceeds for the next trial.

5 Result

The result comparison is done among four different versions of WalkSAT algorithms. The BetaWalkSAT algorithm is the one that’s described above. The 5BestWalkSat and AllWalkSAT algorithms are introduced for comparison purpose to see how it behaves compared to some random variations of the general WalkSAT algorithm. The 5BestWalkSat algorithm stores the variable assignments of 5 best previous trials and samples the initial assignment of next trial from this information. The AllWalkSAT version works same as 5BestWalkSat, but it keeps track of all previous literal assignments instead of 5.

Figure 1: number of solved problems vs. time

Figure 1 presents the number of problems solved in X-axis and the time in Y-axis. The presented BetaWalkSAT algorithm has to sample from Beta

distribution of each variable, which involves calculating the inverse of Cumulative Distribution Function (CDF). This is an expensive operation making the

BetaWalkSAT algorithm a little slower. The WalkSat and 5BestWalkSat algorithms doesn’t involve any complex calculation and thus works faster in simple solvable problems. The graph could be presented as number of problems solved vs. number of flips required, but as the number of tries are very few for the simple satisfiable problems and number of flips highly varies randomly, that seems not a reasonable way to express the result.

Figure 2: average number of tries and flips per solution

Figure 2 shows the average number of tries and flips per solution. The available CNF problem generators generate problems that are either easily satisfiable or are not satisfiable at all. They generally do not generate problems that are satisfiable, but are complex and difficult to solve. Though thousands of CNF problems are generated for experimental purposes, the satisfiable problems among them are very easily solvable yielding very few number of tries per solution. As the intelligence of the BetaWalkSAT algorithm comes into play when sampling for the starting state is performed for a new trial, the generated samples using the CNF generators doesn’t help much in evaluating the BetaWalkSAT algorithm.

Figure 3: solving percentage of each algorithm for a hard satisfiable problem

Figure 3 presents the percentage of satisfiable runs of each algorithm for a complex satisfiable problem. BetaWalkSAT performs better in this case bolstering the idea of giving positive bias to the initial assignment of literals at the start of each trial. As the problem considered here is complex, it requires many trials to find a solution and BetaWalkSAT algorithm is able to apply its reasoning for the positive bias of the starting state.

6 Conclusion

The starting state is critically important for any local search algorithm. Finding a global minima or maxima can extensively be affected depending on where the search started. WalkSAT, the existing local search algorithm for determining the satisfiability of propositional logic formulae is not able to bias the starting state towards more promising states. The proposed BetaWalkSAT algorithm improves the starting state over consecutive trials biasing towards convergence. The results presented in the paper bolsters the claim showing that, biasing starting state by maintaining Beta distribution for each literal does help in determining satisfiability. It outperforms for harder problems requiring many trials, though the difference is not significant enough. Applying BetaWalkSAT in more complex solvable problems would help to better understand its performance. In future, we will work more to formulate complex solvable problems to better quantify the performance improvement of our algorithm. The idea of biasing starting state is general and is supposed to work for other local search algorithms as well. In future, we will work to see how it behaves for other local search algorithms.

7 Acknowledgment

Special thanks to professor Wheeler Ruml for the awesome AI course and for all the great helps and suggestions regarding this project.


  • [1] R. Armañanzas, I. Inza, R. Santana, Y. Saeys, J. L. Flores, J. A. Lozano, Y. Van de Peer, R. Blanco, V. Robles, C. Bielza, and P. Larrañaga (2008) A review of estimation of distribution algorithms in bioinformatics.. BioData mining 1 (1), pp. 6. External Links: Document, ISBN 1756-0381, ISSN 1756-0381, Link Cited by: §3.
  • [2] E. Bengoetxea (2003) Inexact Graph Matching Using Estimation of Distribution Algorithms.. PhD Thesis. Ecole Nationale Supérieure des Télécommunications, Paris, France. Last Version: March 2003, pp. 1–17. Cited by: §3.
  • [3] K. D. Boese, A. B. Kahng, and S. Muddu (1994) A new adaptive multi-start technique for combinatorial global optimizations. Operations Research Letters 16 (2), pp. 101–113. External Links: Document, ISSN 01676377 Cited by: §3.
  • [4] M. Pelikan, M. W. Hauschild, and F. G. Lobo (2012) Introduction to Estimation of Distribution Algorithms. (2012003), pp. 42. Cited by: §3.
  • [5] B. Selman, H. Kautz, and B. Cohen (1995) Local Search Strategies for Satissability Testing. DIMACS Series in Discrete Mathematics and Theoretical Computer Science 00. Cited by: §3.
  • [6] M. Tabassum (2014) a Genetic Algorithm Analysis Towards Optimization Solutions. International Journal of Digital Information and Wireless Communications 4 (1), pp. 124–142. External Links: Document, ISSN 2225-658X, Link Cited by: §3.
  • [7] L. Zhang, C. F. Madigan, M. H. Moskewicz, and S. Malik (2001) Efficient Conflict Driven Learning in a Boolean Satisfiability Solver. Proceedings of the 2001 IEEE/ACM International Conference on Computer-aided Design, pp. 279–285. External Links: Document, ISBN 0-7803-7249-2, ISSN 1092-3152, Link Cited by: §3.