We propose a version of WalkSAT algorithm, named as BetaWalkSAT. This method uses probabilistic reasoning for biasing the starting state of the local search algorithm. Beta distribution is used to model the belief over boolean values of the literals. Our results suggest that, the proposed BetaWalkSAT algorithm can outperform other uninformed local search approaches for complex boolean satisfiability problems.
A statement that has either a true or false value is called a proposition. Propositional logic studies the idea of building complicated propositions using simple propositions or statements connected by logical connectives. The true or false value of a compound proposition depends on the values of underlying simple propositions and their structure of connectives.
Two popular algorithms for determining the satisfiability of propositional logic are: Davis–Logemann–Loveland (DLL) algorithm and WalkSAT algorithm. DLL is a backtrack based complete algorithm and WalkSAT is an incomplete local search algorithm for determining the boolean satisfiability of the propositional logic formulae. These algorithms are unable to apply any prior belief about the assignments of the propositions. The WalkSAT
algorithm always starts and continues with a uniform distribution and is unable to update it’s belief about the variable assignment from an unsatisfiable situation, rather it repeatedly samples from the uniform distribution on each trial. The current algorithms don’t employ the belief update or probabilistic reasoning about the binary assignment of the variables. This paper addresses the problem to make theWalkSAT algorithm smarter by updating the belief of variable assignments with each unsatisfied attempt, so that it can obtain a positive bias towards convergence and can find a satisfiable or unsatisfiable situation more efficiently.
The paper is organized as follows: section 2 discusses some of the relevant works. Section 3 presents the approach and the proposed algorithm. Section 4 presents and discusses the results obtained. Section 5 concludes the paper with some probable future works.
3 Related works
Search strategy can be divided broadly into two groups: complete and heuristic. A complete strategy considers all possible options to find a solution whereas a heuristic based strategy prunes out the less promising options to quickly reach the goal. Heuristic strategies can be deterministic or non-deterministic. A deterministic strategy (e.g. different hill climbing algorithms
) yields the same solution under the same condition and can get stuck into the local maxima or minima. A non-deterministic strategy is able to produce different solutions under the same condition and can escape from local minima or maxima by the strength of it’s stochasticity. Evolutionary algorithms, Genetic algorithms, Simulated annealing etc. utilizes the non-deterministic strategy. But these non-deterministic strategy approaches have some shortcomings like: they are not guaranteed to always find an optimal solution, can be very slow and complex, depends on many parameters that require manual tuning and finding the optimal parameters itself yields an optimization problem to solve[6, 3].
Estimation distribution algorithms (EDA) biases the movement of the search towards the more promising candidate options by maintaining an explicit probabilistic model. EDA is an evolutionary algorithm also known as probabilistic model-building genetic algorithm that treats the optimization problem as a sequence of incremental update of probabilistic model starting from a prior belief model (e.g. uniform) . The incremental bias towards the goal makes EDA a good candidate to solve complex optimization problems. There are several papers available on EDA containing its application in bioinformatics, genomics, DNA microarray classification, proteomics and so on . A mixed random walk strategy is proposed in  for escaping from the local minima for satisfiability testing of CNF formulas.  presents different conflict driven learning techniques used in various SAT solvers and proposes several new learning strategies. To the best of our knowledge, we did not find a paper that applies the idea of Estimation distribution algorithms for boolean satisfiability of propositional logic, maintaining the explicit probabilistic model as a beta distribution of the binary literals.
Beta distribution is a very convenient and powerful choice of priors for Bernoulli distribution. The beta distribution presents a continuous probability distribution of random variables on the interval [0,1]. Beta distribution is parametrized by two positive parameters and that controls the shape of the distribution.
The algorithm presented here assumes the literals are independent of each other, they are random variables with binary choices and models the belief of the assignment of each literal with a beta distribution. Initially it represents the prior belief about the assignment which is uniform. After each unsuccessful trial, the belief for each literal is updated with the beta distribution. Sampling for the next trial is then done from this distribution, biasing the chance towards faster convergence.
4.1 Beta-WalkSat algorithm
The Beta-WalkSat algorithm described in algorithm 1 is almost same as of the normal WalkSAT algorithm except that it maintains a beta distribution for each literal and utilizes that distribution for biasing the starting state of each trial. Lines 2-5 describes the input to the algorithm and line 6 describes the returned output.
The algorithm starts at line 8 and iterates for maxTries number of times. At the start of each trial, an initial value is assigned to each literal sampling from the beta distribution of that literal in lines 9-10. The for loop at line 11 runs for maxFlip number of times, picks a random unsatisfied clause in line 12, tries to find a literal that doesn’t break any other clause and flips that in lines 13-15. If no such literal is found, it flips a random literal with probability p in line 18, or with probability 1-p chooses a random literal that minimizes breaks. If all the clauses are satisfied, the result is returned in line 23. If not, it updates the beta distribution of each literal by increasing the value of alpha or beta based on the current assignment of the literal in lines 24-25 and proceeds for the next trial.
The result comparison is done among four different versions of WalkSAT algorithms. The BetaWalkSAT algorithm is the one that’s described above. The 5BestWalkSat and AllWalkSAT algorithms are introduced for comparison purpose to see how it behaves compared to some random variations of the general WalkSAT algorithm. The 5BestWalkSat algorithm stores the variable assignments of 5 best previous trials and samples the initial assignment of next trial from this information. The AllWalkSAT version works same as 5BestWalkSat, but it keeps track of all previous literal assignments instead of 5.
Figure 1 presents the number of problems solved in X-axis and the time in Y-axis. The presented BetaWalkSAT algorithm has to sample from Beta
distribution of each variable, which involves calculating the inverse of Cumulative Distribution Function (CDF). This is an expensive operation making theBetaWalkSAT algorithm a little slower. The WalkSat and 5BestWalkSat algorithms doesn’t involve any complex calculation and thus works faster in simple solvable problems. The graph could be presented as number of problems solved vs. number of flips required, but as the number of tries are very few for the simple satisfiable problems and number of flips highly varies randomly, that seems not a reasonable way to express the result.
Figure 2 shows the average number of tries and flips per solution. The available CNF problem generators generate problems that are either easily satisfiable or are not satisfiable at all. They generally do not generate problems that are satisfiable, but are complex and difficult to solve. Though thousands of CNF problems are generated for experimental purposes, the satisfiable problems among them are very easily solvable yielding very few number of tries per solution. As the intelligence of the BetaWalkSAT algorithm comes into play when sampling for the starting state is performed for a new trial, the generated samples using the CNF generators doesn’t help much in evaluating the BetaWalkSAT algorithm.
Figure 3 presents the percentage of satisfiable runs of each algorithm for a complex satisfiable problem. BetaWalkSAT performs better in this case bolstering the idea of giving positive bias to the initial assignment of literals at the start of each trial. As the problem considered here is complex, it requires many trials to find a solution and BetaWalkSAT algorithm is able to apply its reasoning for the positive bias of the starting state.
The starting state is critically important for any local search algorithm. Finding a global minima or maxima can extensively be affected depending on where the search started. WalkSAT, the existing local search algorithm for determining the satisfiability of propositional logic formulae is not able to bias the starting state towards more promising states. The proposed BetaWalkSAT algorithm improves the starting state over consecutive trials biasing towards convergence. The results presented in the paper bolsters the claim showing that, biasing starting state by maintaining Beta distribution for each literal does help in determining satisfiability. It outperforms for harder problems requiring many trials, though the difference is not significant enough. Applying BetaWalkSAT in more complex solvable problems would help to better understand its performance. In future, we will work more to formulate complex solvable problems to better quantify the performance improvement of our algorithm. The idea of biasing starting state is general and is supposed to work for other local search algorithms as well. In future, we will work to see how it behaves for other local search algorithms.
Special thanks to professor Wheeler Ruml for the awesome AI course and for all the great helps and suggestions regarding this project.
-  (2008) A review of estimation of distribution algorithms in bioinformatics.. BioData mining 1 (1), pp. 6. External Links: Cited by: §3.
-  (2003) Inexact Graph Matching Using Estimation of Distribution Algorithms.. PhD Thesis. Ecole Nationale Supérieure des Télécommunications, Paris, France. Last Version: March 2003, pp. 1–17. Cited by: §3.
-  (1994) A new adaptive multi-start technique for combinatorial global optimizations. Operations Research Letters 16 (2), pp. 101–113. External Links: Cited by: §3.
-  (2012) Introduction to Estimation of Distribution Algorithms. (2012003), pp. 42. Cited by: §3.
-  (1995) Local Search Strategies for Satissability Testing. DIMACS Series in Discrete Mathematics and Theoretical Computer Science 00. Cited by: §3.
-  (2014) a Genetic Algorithm Analysis Towards Optimization Solutions. International Journal of Digital Information and Wireless Communications 4 (1), pp. 124–142. External Links: Cited by: §3.
-  (2001) Efficient Conflict Driven Learning in a Boolean Satisfiability Solver. Proceedings of the 2001 IEEE/ACM International Conference on Computer-aided Design, pp. 279–285. External Links: Cited by: §3.