Counterexample Guided Inductive Optimization

04/11/2017
by   Rodrigo F. Araújo, et al.
0

This paper describes three variants of a counterexample guided inductive optimization (CEGIO) approach based on Satisfiability Modulo Theories (SMT) solvers. In particular, CEGIO relies on iterative executions to constrain a verification procedure, in order to perform inductive generalization, based on counterexamples extracted from SMT solvers. CEGIO is able to successfully optimize a wide range of functions, including non-linear and non-convex optimization problems based on SMT solvers, in which data provided by counterexamples are employed to guide the verification engine, thus reducing the optimization domain. The present algorithms are evaluated using a large set of benchmarks typically employed for evaluating optimization techniques. Experimental results show the efficiency and effectiveness of the proposed algorithms, which find the optimal solution in all evaluated benchmarks, while traditional techniques are usually trapped by local minima.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

08/14/2017

Counterexample Guided Inductive Optimization Applied to Mobile Robots Path Planning (Extended Version)

We describe and evaluate a novel optimization-based off-line path planni...
02/07/2012

Optimization in SMT with LA(Q) Cost Functions

In the contexts of automated reasoning and formal verification, importan...
07/21/2020

Automated and Sound Synthesis of Lyapunov Functions with SMT Solvers

In this paper we employ SMT solvers to soundly synthesise Lyapunov funct...
07/28/2021

Satisfiability and Synthesis Modulo Oracles

In classic program synthesis algorithms, such as counterexample-guided i...
07/21/2018

Delta-Decision Procedures for Exists-Forall Problems over the Reals

Solving nonlinear SMT problems over real numbers has wide applications i...
01/11/2018

Formal verification of an interior point algorithm instanciation

With the increasing power of computers, real-time algorithms tends to be...
05/22/2019

A CDCL-style calculus for solving non-linear constraints

In this paper we propose a novel approach for checking satisfiability of...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Optimization is an important research topic in many fields, especially in computer science and engineering deb2004optimization . Commonly, scientists and engineers have to find parameters, which optimize the behavior of a given system or the value of a given function (i.e., an optimal solution). Optimization characterizes and distinguishes the engineering gaze over a problem; for this particular reason, previous studies showed that optimization is one of the main differences between engineering design and technological design Gattie2007 .

Computer science and optimization maintain a symbiotic relationship. Many important advances of computer science are based on optimization theory. As example, planning and decidability problems (e.g.

, game theory 

Shoham2008 ), resource allocation problems (e.g., hardware/software co-design Teich2012

), and computational estimation and approximation (

e.g., numerical analysis kowalski1995selected ) represent important optimization applications. Conversely, computer science plays an important role in recent optimization studies, developing efficient algorithms and providing respective tools for supporting model management and results analysis derigs2009optimization .

There are many optimization techniques described in the literature (e.g., simplex garfinkel1972integer , gradient descent Bartholomew–Biggs2008

, and genetic algorithms 

goldberg1989genetic ), which are suitable for different classes of optimization problems (e.g., linear or non-linear, continuous or discrete, convex or non-convex, and single- or multi-objective). These techniques are usually split into two main groups: deterministic and stochastic optimization. Deterministic optimization is the classic approach for optimization algorithms, which is based on calculus and algebra operators, e.g., gradients and Hessians cavazzuti2012optimization . Stochastic optimization employs randomness in the optima search procedure cavazzuti2012optimization . This paper presents a novel class of search-based optimization algorithm that employs non-deterministic representation of decision variables and constrains the state-space search based on counterexamples produced by an SMT solver, in order to ensure the complete global optimization without employing randomness. This class of techniques is defined here as counterexample guided inductive optimization (CEGIO), which is inspired by the syntax-guided synthesis (SyGuS) to perform inductive generalization based on counterexamples provided by a verification oracle AlurSyGus2013 .

Particularly, a continuous non-convex optimization problem is one of the most complex problems. As a result, several traditional methods (e.g., Newton-Raphson deb2004optimization and Gradient Descent Bartholomew–Biggs2008 ) are inefficient to solve that specific class of problems deb2004optimization

. Various heuristics are developed for obtaining approximated solutions to those problems; heuristics methods (

e.g., ant colony Dorigo2006 and genetic algorithms goldberg1989genetic ) offer faster solutions for complex problems, but they sacrifice the system’s correctness and are easily trapped by local optimal solutions.

This paper presents a novel counterexample guided inductive optimization technique based on SMT solvers, which is suitable for a wide variety of functions, even for non-linear and non-convex functions, since most real-world optimization problems are non-convex. The function evaluation and the search for the optimal solution is performed by means of an iterative execution of successive verifications based on counterexamples extracted from SMT solvers. The counterexample provides new domain boundaries and new optima candidates. In contrast to other heuristic methods (e.g., genetic algorithms), which are usually employed for optimizing this class of function, the present approaches always find the global optimal point.

This study extends the previous work of Araújo et al. Araujo2016 and presents three variants of a counterexample guided inductive optimization approach based on SMT solvers, which improve the technique performance for specific class of functions. Furthermore, the experimental evaluation is largely expanded, since the algorithms are executed for additional optimization problems and the performance of each proposed algorithm is compared to six well-known optimization techniques. The present CEGIO approaches are able to find the correct global minima for 100% of the benchmarks, while other techniques are usually trapped by local minima, thus leading to incorrect solutions.

1.1 Contributions

Our main original contributions are:

  • Novel counterexample guided inductive optimization approach. This work describes three novel variants of a counterexample guided inductive optimization approach based on SMT solvers: generalized, simplified, and fast algorithms. The generalized algorithm can be used for any constrained optimization problem and presents minor improvements w.r.t. Araújo et al. Araujo2016 . The simplified algorithm is faster than the generalized one and can be employed if information about the minima location is provided, e.g., the cost function is semi-definite positive. The fast algorithm presents a significant speed-up if compared to the generalized and simplified ones, but it can only be employed for convex functions.

  • Convergence Proofs. This paper presents proofs of convergence and completeness (omitted in Araújo et al. Araujo2016 ) for the proposed counterexample guided inductive optimization algorithms.

  • SMT solvers performance comparison. The experiments are performed with three different SMT solvers: Z3 z3 , Boolector boolector , and MathSAT mathsat5 . The experimental results show that the solver choice can heavily influence the method performance.

  • Additional benchmarks. The benchmark suite is expanded to optimization functions extracted from the literature functionslist .

  • Comparison with existing techniques. The proposed technique is compared to genetic algorithm goldberg1989genetic , particle swarm pswarmref:olsson2011particle , pattern search patternsref:alberto2004pattern , simulated annealing saref:Laarhoven:1987:SAT:59580 , and nonlinear programming npref:byrd2000trust , which are traditional optimization techniques employed for non-convex functions.

1.2 Availability of Data and Tools

Our experiments are based on a set of publicly available benchmarks. All tools, benchmarks, and results of our evaluation are available on a supplementary web page http://esbmc.org/benchmarks/jscp2017.zip.

1.3 Outline

Section 2 discusses related studies. Section 3 provides an overview of optimization problems and techniques, and describes background on software model checking. Section 4 describes the ANSI-C model developed for optimization problems that is suitable for the counterexample guided inductive optimization procedure. Section 5 describes the generalized and simplified optimization algorithms and respective completeness proofs. Section 6 describes the fast optimization algorithm and respective completeness proof. Section 7 reports the experimental results for evaluating all proposed optimization algorithms, while Section 8 concludes this work and proposes further studies.

2 Related Work

SMT solvers have been widely applied to solve several types of verification, synthesis, and optimization problems. They are typically used to check the satisfiability of a logical formula, returning assignments to variables that evaluate the formula to true, if it is satisfiable; otherwise, the formula is said to be unsatisfiable. Nieuwenhuis and Oliveras Nieuwenhuis2006 presented the first research about the application of SMT to solve optimization problems. Since then, SMT solvers have been used to solve different optimization problems, e.g., minimize errors in linear fixed-point arithmetic computations in embedded control software Eldib2014 ; reduce the number of gates in FPGA digital circuits Estrada2003 ; hardware/software partition in embedded systems to decide the most efficient system implementation TrindadeSBESC2015 ; TrindadeSBESC2013 ; TrindadeDAES2016 ; and schedule applications for a multi-processor platform Cotton2011 . All those previous studies use SMT-based optimization over a Boolean domain to find the best system configuration given a set of metrics. In particular, in Cotton et al. Cotton2011 the problem is formulated as a multi-objective optimization problem. Recently, Shoukry et al. CDC2016 proposed a scalable solution for synthesizing a digital controller and motion planning for under-actuated robots from LTL specifications. Such solution is more flexible and allows solving a wider variety of problems, but they are focused on optimization problems that can be split into a Boolean part and other convex part.

In addition, there were advances in the development of different specialized SMT solvers that employ generic optimization techniques to accelerate SMT solving, e.g., the ABsolver ABsolver , which is used for automatic analysis and verification of hybrid-system and control-system. The ABsolver uses a non-linear optimization tool for Boolean and polynomial arithmetic and a lazy SMT procedure to perform a faster satisfiability checking. Similarly, CalCs Nuzzo2010 is also an SMT solver that combines convex optimization and lazy SMT to determine the satisfiability of conjunctions of convex non-linear constraints. Recently, Shoukry et al. SMC show that a particular class of logic formulas (named SMC formulas) generalizes a wide range of formulas over Boolean and nonlinear real arithmetic, and propose the Satisfiability Modulo Convex Optimization to solve satisfiability problems over SMC formulas. Our work differs from those previous studies ABsolver ; Nuzzo2010 ; SMC since it does not focus on speeding up SMT solvers, but it employs an SMT-based model-checking tool to guide (via counterexample) an optimization search procedure in order to ensure the global optimization.

Recently,  Bjorner2015 extends the SMT solver Z3 for linear optimization problems; Li et al. proposed the SYMBA algorithm Symba , which is an SMT-based symbolic optimization algorithm that uses the theory of linear real arithmetic and SMT solver as black box. Sebastiani and Trentin SebastianiCAV2015 present OptiMathSat, which is an optimization tool that extends MathSAT5 SMT solver to allow solving linear functions in the Boolean, rational, and integer domains or a combination of them; in Sebastiani and Tomasi SebastianiTransactions2015 , the authors used a combination of SMT and LP techniques to minimize rational functions; the related work SebastianiTACAS2015 extends their work with linear arithmetic on the mixed integer/rational domain, thus combining SMT, LP, and ILP techniques.

As an application example, Pavlinovic et al. Pavlinovic0W15 propose an approach which considers all possible compiler error sources for statically typed functional programming languages and reports the most useful one subject to some usefulness criterion. The authors formulate this approach as an optimization problem related to SMT and use Z to compute an optimal error source in a given ill-typed program. The approach described by Pavlinovic et al., which uses MaxSMT solver Z, shows a significant performance improvement if compared to previous SMT encodings and localization algorithms.

Most previous studies related to SMT-based optimization can only solve linear problems over integer, rational, and Boolean domains in specific cases, leading to limitations in practical engineering applications. Only a few studies CDC2016 are able to solve non-linear problems, but they are also constrained to convex functions. In contrast, this paper proposes a novel counterexample guided inductive optimization method based on SMT solvers to minimize functions, linear or non-linear, convex or non-convex, continuous or discontinuous. As a result, the proposed methods are able to solve optimization problems directly on the rational domain with adjustable precision, without using any other technique to assist the state-space search. Furthermore, our proposed methods employ a model-checking tool to generate automatically SMT formulas from an ANSI-C model of the optimization problem, which makes the representation of problems for SMT solving easy for engineers.

3 Preliminaries

3.1 Optimization Problems Overview

Let be a cost function, such that

represents the decision variables vector

and . Let be a subset settled by a set of constraints.

Definition 1.

A multi-variable optimization problem consists in finding an optimal vector x, which minimizes in .

According to Definition 1, an optimization problem can be written as

(1)

In particular, this optimization problem can be classified in different ways w.r.t. constraints, decision variables domain, and nature of cost function

. All optimization problems considered here are constrained, i.e., decision variables are constrained by the subset . The optimization problem domain that contains can be the set of , , , or . Depending on the domain and constraints, the optimization search-space can be small or large, which influences the optimization algorithms performance.

The cost function can be classified as linear or non-linear; continuous, discontinuous or discrete; convex or non-convex. Depending on the cost function nature, the optimization problem can be hard to solve, given the time and memory resources Galperin19911 . Particularly, non-convex optimization problems are the most difficult ones w.r.t. the cost function nature. A non-convex cost function is a function whose epigraph is a non-convex set and consequently presents various inflexion points that can trap the optimization algorithm to a sub-optimal solution. A non-convex problem is necessarily a non-linear problem and it can also be discontinuous. Depending on that classification, some optimization techniques are unable to solve the optimization problem, and some algorithms usually point to suboptimal solutions, i.e., a solution that is not a global minimum of , but it only locally minimizes . Global optimal solutions of the function , aforementioned, can be defined as

Definition 2.

A vector is a global optimal solution of in iff .

3.2 Optimization Techniques

Different optimization problems offer different difficulties to their particular solutions. Such complexity is mainly related to the ruggedness (e.g., continuity, differentiability, and smoothness) and dimensionality of the problem (i.e., the dimension, and for the finite case, the number of elements of ). Depending on these factors, different optimization techniques can be more efficient to solve a particular optimization problem. Generally, traditional optimization techniques can be divided into two groups: deterministic and stochastic optimization.

The deterministic techniques employ a search engine, where each step is directly and deterministically related to the previous steps refdet:floudas2000deterministic . In summary, deterministic techniques can be gradient-based or enumerative search-based. Gradient-based techniques search for points, where the gradient of cost function is null (), e.g., gradient-descent snyman2005practical and Newton’s optimization deb2004optimization . Although they are fast and efficient, those techniques are unable to solve non-convex or non-differentiable problems. Enumerative search-based optimization consists in scanning the search-space by enumerating all possible points and comparing cost function with best previous values, e.g., dynamic programming, branch and bound scholz2011deterministic , and pattern search FINDLER198741 .

Stochastic techniques employ randomness to avoid the local minima and to ensure the global optimization; such techniques are usually based on meta-heuristics refstc:marti2005stochastic . This class of techniques has become very popular in the last decades and has been used in all types of optimization problems. Among those stochastic techniques, simulated annealing saref:Laarhoven:1987:SAT:59580 , particle swarm pswarmref:olsson2011particle

, and evolutionary algorithms (

e.g., genetic algorithms goldberg1989genetic ) are usually employed in practice.

Recently, optimization techniques and tools that employ SMT solvers and non-deterministic variables were applied to solve optimization problems CDC2016 ; ABsolver ; Nuzzo2010 ; Gao2013 ; Bjorner2015 ; Symba ; SebastianiCAV2015 ; SebastianiTransactions2015 ; SebastianiTACAS2015 , which searches for the global optima in a search-space that is symbolically defined and uses counterexamples produced by SMT solvers to further constrain the search-space. The global optima is the set of values for the decision variables that makes an optimization proposition satisfiable. The technique presented here is the first optimization method based on SMT solvers and inductive generalization described in the literature, which is able to solve non-convex problems over .

3.3 Model Checking

Model checking is an automated verification procedure to exhaustively check all (reachable) system’s states Clarke1999 . The model checking procedure typically consists of three steps: modeling, specification, and verification.

Modeling is the first step, where it converts the system to a formalism that is accepted by a verifier. The modeling step usually requires the use of an abstraction to eliminate irrelevant (or less) important system details Baier2008 . The second step is the specification, which describes the system’s behavior and the property to be checked. An important issue in the specification is the correctness. Model checking provides ways to check whether a given specification satisfies a system’s property, but it is difficult to determine whether such specification covers all properties in which the system should satisfy.

Finally, the verification step checks whether a given property is satisfied w.r.t. a given model, i.e., all relevant system states are checked to search for any state that violates the verified property. In case of a property violation, the verifier reports the system’s execution trace (counterexample), which contains all steps from the (initial) state to the (bad) state that lead to the property violation. Errors could also occur due to incorrect system modeling or inadequate specification, thus generating false verification results.

3.3.1 Bounded Model Checking (BMC)

BMC is an important verification technique, which has presented attractive results over the last years Beyer2016 . BMC techniques based on Boolean Satisfiability (SAT) handbook09 or Satisfiability Modulo Theories (SMT) BarrettSST09 have been successfully applied to verify single- and multi-threaded programs, and also to find subtle bugs in real programs esbmc ; CBMC . BMC checks the negation of a given property at a given depth over a transition system .

Definition 3.

handbook09 – Given a transition system , a property , and a bound ; BMC unrolls the system times and translates it into a verification condition (VC) , which is satisfiable iff has a counterexample of depth less than or equal to .

In this study, the ESBMC tool MorseRCN014 is used as verification engine, as it represents one of the most efficient BMC tools that participated in the last software verification competitions Beyer2016

. ESBMC finds property violations such as pointer safety, array bounds, atomicity, overflows, deadlocks, data race, and memory leaks in single- and multi-threaded C/C++ software. It also verifies programs that make use of bit-level, pointers, structures, unions, fixed- and floating-point arithmetic. Inside ESBMC, the associated problem is formulated by constructing the following logical formula

(2)

where is a property and is a set of initial states of , and is the transition relation of between time steps and . Hence, represents the executions of a transition system of length . The above VC can be satisfied if and only if, for some there exists a reachable state at time step in which is violated. If the logical formula (2) is satisfiable (i.e., returns ), then the SMT solver provides a satisfying assignment (counterexample).

Definition 4.

A counterexample for a property is a sequence of states with , , and for that makes (2) satisfiable. If it is unsatisfiable (i.e., returns false), then one can conclude that there is no error state in steps or less.

In addition to software verification, ESBMC has been applied to ensure correctness of digital filters and controllers Abreu2016 ; Bessa2016 ; esbmc_controller_SBESC . Recently, ESBMC has been applied to optimize HW/SW co-design TrindadeSBESC2015 ; TrindadeSBESC2013 ; TrindadeDAES2016 .

4 Verification Model for Counterexample Guided Inductive Optimization

4.1 Modeling Optimization Problems using a Software Model Checker

There are two important directives in the C/C++ programming language, which can be used for modeling and controlling a verification process: ASSUME and ASSERT. The ASSUME directive can define constraints over (non-deterministic) variables, and the ASSERT directive is used to check system’s correctness w.r.t. a given property. Using these two statements, any off-the-shelf C/C++ model checker (e.g., CBMC CBMC , CPAChecker Beyer2011 , and ESBMC MorseRCN014 ) can be applied to check specific constraints in optimization problems, as described by Eq. (1).

Here, the verification process is iteratively repeated to solve an optimization problem using intrinsic functions available in ESBMC (e.g., __ESBMC_assume and __ESBMC_assert). We apply incremental BMC to efficiently prune the state-space search based on counterexamples produced by an SMT solver. Note that completeness is not an issue here (cf. Definitions 1 and 2) since our optimization problems are represented by loop-free programs Gadelha2015 .

4.2 Illustrative Example

The Ursem03’s function is employed to illustrate the present SMT-based optimization method for non-convex optimization problems functionslist . The Ursem03’s function is represented by a two-variables function with only one global minimum in , and has four regularly spaced local minima positioned in a circumference, with the global minimum in the center. Ursem03’s function is defined by Eq. (3); Fig. 1 shows its respective graphic.

(3)
Figure 1: Ursem03’s function

4.3 Modeling

The modeling process defines constraints, i.e., boundaries (cf. Section 3.1). This step is important for reducing the state-space search and consequently for avoiding the state-space explosion by the underlying model-checking procedure. Our verification engine is not efficient for unconstrained optimization; fortunately, the verification time can be drastically reduced by means of a suitable constraint choice. Consider the optimization problem given by Eq. (4), which is related to the Ursem03’s function given in Eq. (3):

(4)

Note that inequalities and are pruning the state-space search to the first quadrant; however, even so it produces a (huge) state-space to be explored since and can assume values with very high modules. The optimization problem given by Eq. (4) can be properly rewritten as Eq. (5) by introducing new constraints. The boundaries are chosen based on Jamil and Yang functionslist , which define the domain in which the optimization algorithms can evaluate the benchmark functions.

(5)

From the optimization problem definition given by Eq. (5), the modeling step can be encoded, where decision variables are declared as non-deterministic variables constrained by the ASSUME directive. In this case, and . Fig. 2 shows the respective C code for modeling Eq. (5).

1#include "math2.h"
2float nondet_float();
3int main() {
4  //define decision variables
5  float x1 = nondet_float();
6  float x2 = nondet_float();
7  //constrain the state-space search
8  __ESBMC_assume((x1>=-3) && (x1<=3));
9  __ESBMC_assume((x2>=-2) && (x2<=2));
10  //computing Ursem’s function
11  float fobj;
12  fobj= -sin(2.2*pi*x1-pi/2)*(2-abs(x1))(3-abs(x1))/4
13  -sin(2.2*pi*x2-pi/2)*(2-abs(x2))(3-abs(x2))/4;
14  return 0;
15}
Figure 2: C Code for the optimization problem given by Eq. (5).

Note that in Figure 2, the decision variables and are declared as floating-point numbers initialized with non-deterministic values; we then constraint the state-space search using assume statements. The objective function of Ursem‘s function is then declared as described by Eq. 3.

4.4 Specification

The next step of the proposed methodology is the specification, where the system behavior and the property to be checked are described. For the Ursem03’s function, the result of the specification step is the C program shown in Fig. 3, which is iteratively checked by the underlying verifier. Note that the decision variables are declared as integer type and their initialization depends on a given precision , which is iteratively adjusted once the counterexample is produced by the SMT solver. Indeed, the C program shown in Fig. 2 leads the verifier to produce a considerably large state-space exploration, if the decision variables are declared as non-deterministic floating-point type. In this study, decision variables are defined as non-deterministic integers, thus discretizing and reducing the state-space exploration; however, this also reduces the optimization process precision.

To trade-off both precision and verification time, and also to maintain convergence to an optimal solution, the underlying model-checking procedure has to be iteratively invoked, in order to increase its precision for each successive execution. An integer variable is created and iteratively adjusted, such that is the amount of decimal places related to the decision variables. Additionally, a new constraint is inserted; in particular, the new value of the objective function at the -th must not be greater than the value obtained in the previous iteration . Initially, all elements in the state-space search are candidates for optimal points, and this constraint cutoffs several candidates on each iteration.

In addition, a property has to be specified to ensure convergence to the minimum point on each iteration. This property specification is stated by means of an assertion, which checks whether the literal given in Eq. (6) is satisfiable for every optimal candidate remaining in the state-space search (i.e., traversed from lowest to highest).

(6)

The verification procedure stops when the literal is not satisfiable, i.e., if there is any for which ; a counterexample shows such , approaching iteratively from the optimal . Fig. 3 shows the initial specification for the optimization problem given by Eq. (5). The initial value of the objective function can be randomly initialized. For the example in Fig. 3, is arbitrarily initialized to , but the present optimization algorithm works for any initial state.

1#include "math2.h"
2#define p 1 //precision variable
3int nondet_int();
4float nondet_float();
5int main() {
6  float f_i = 100;  //previous objective function value
7  int lim_inf_x1 = -2*p;
8  int lim_sup_x1 = 2*p;
9  int lim_inf_x2 = -2*p;
10  int lim_sup_x2 = 2*p;
11  int X1 = nondet_int();
12  int X2 = nondet_int();
13  float x1 = float nondet_float();
14  float x2 = float nondet_float();
15  __ESBMC_assume( (X1>=lim_inf_x1) && (X1<=lim_sup_x1) );
16  __ESBMC_assume( (X2>=lim_inf_x2) && (X2<=lim_sup_x2) );
17  __ESBMC_assume( x1 = (float) X1/p );
18  __ESBMC_assume( x2 = (float) X2/p );
19  float fobj;
20  fobj= -sin(2.2*pi*x1-pi/2)*(2-abs(x1))(3-abs(x1))/4
21  -sin(2.2*pi*x2-pi/2)*(2-abs(x2))(3-abs(x2))/4;
22  //constrain to exclude fobj>f_i
23  __ESBMC_assume( fobj < f_i );
24  assert( fobj > f_i );
25  return 0;
26}
Figure 3: C code after the specification of Eq. (5).

4.5 Verification

Finally, in the verification step, the C program shown in Fig. 3 is checked by the verifier and a counterexample is returned with a set of decision variables x, for which the objective function value converges to the optimal value. A specified C program only returns a successful verification result if the previous function value is the optimal point for that specific precision (defined by ), i.e., . For the example shown in Fig. 3, the verifier shows a counterexample with the following decision variables: and . These decision variable are used to compute a new minimum candidate, note that , which is the new minimum candidate solution provided by this verification step. Naturally, it is less than the initial value (100), and this verification can be repeated with the new value of , in order to obtain an objective function value that is close to the optimal point on each iteration. Note that the data provided by the counterexample is crucial for the algorithm convergence and for the state-space search reduction.

5 Counterexample Guided Inductive Optimization of Non-convex Functions

This section presents two variants of the Counterexample Guided Inductive Optimization (CEGIO) algorithm for global constrained optimization. A generalized CEGIO algorithm is explained in Subsection 5.1, together with a convergence proof in Subsection 5.2, while Subsection 5.3 presents a simplified version of that algorithm.

5.1 CEGIO: the Generalized Algorithm (CEGIO-G)

The generalized SMT-based optimization algorithm previously presented by Araújo et al. Araujo2016 is able to find the global optima for any optimization problem that can be modeled with the methodology presented in Section 4. The execution time of that algorithm depends on how the state-space search is restricted and on the number of the solution decimal places. Specifically, the algorithm presents a fixed-point solution with adjustable precision, i.e., the number of decimal places can be defined. Naturally, for integer optimal points, this algorithm returns the correct solution quickly. However, this algorithm might take longer for achieving the optimal solution of unconstrained optimization problems with non-integer solutions since it depends on the required precision. Although this algorithm frequently produces a longer execution time than other traditional techniques, its error rate is typically lower than other existing methods, once it is based on a complete and sound verification procedure. Alg. 1 shows an improved version of the algorithm presented by Araújo et al. Araujo2016 ; this algorithm is denoted here as ”Generalized CEGIO algorithm” (CEGIO-G).

input : A cost function , the space for constraint set , and a desired precision
output : The optimal decision variable vector , and the optimal value of function
1 Initialize randomly and Initialize the precision variable with Declare the auxiliary variables x as non-deterministic integer variables while  do
2        Define bounds for x with the ASSUME directive, such that Describe a model for do
3               Constrain with the ASSUME directive Verify the satisfiability of given by Eq. (6) with the ASSERT directive Update and based on the counterexample Do 
4       while  is satisfiableUpdate the precision variable p
5 end while
and return and
Algorithm 1 CEGIO: the generalized algorithm.

Alg. 1 repeats the specification and verification steps, described in Section 4, until the optimal solution is found. The precision of optimal solution defines the desired precision variable . An unitary value of results in integer solutions. Solution with one decimal place is obtained for , two decimal places are achieved for , i.e., the number of decimal places for the solution is calculated by means of the equation

(7)

After the variable initialization and declaration (lines 1-3 of Alg. 1), the search domain is specified in line 5, which is defined by lower and upper bounds of the x variable, and in line 6, the model for function, , is defined. The specification step (line 8) is executed for each iteration until the desired precision is achieved. In this specific step, the search-space is remodelled for the -th precision and it employs previous results of the optimization process, i.e., . The verification step is performed in lines 9-10, where the candidate function , i.e., is analyzed by means of the satisfiability check of . If there is a that violates the ASSERT directive, then the candidate function is updated and the algorithm returns to the specification step (line 8) to remodel the state-space again. If the ASSERT directive is not violated, the last candidate is the minimum value with the precision variable (initially equal to 1), thus is multiplied by , adding a decimal place to the optimization solution, and the outer loop (while) is repeated.

Note that Alg. 1 contains two nested loops, the outer (while) loop is related to the desired precision and the inner (do-while) loop is related to the specification and verification steps. This configuration speeds-up the optimization problem due to the complexity reduction if compared to the algorithm originally presented in Araújo et al. Araujo2016 . The generalized CEGIO algorithm uses the manipulation of fixed-point number precision to ensure the optimization convergence.

5.2 Proof of Convergence

A generic optimization problem described in the previous section is formalized as follow: given a set , determine , such that, is the lowest value of the function , i.e., , where is the image set of (i.e., ). Our approach solves the optimization problem with decimal places, i.e., the solution is an element of the rational domain such that , where , i.e., is composed by rationals with decimal places in (e.g., ). Thus, is the minima of function in .

Lemma 1.

Let be a finite set composed by all values , where is any minimum candidate and . The literal (Eq. 6) is UNSAT iff holds the lowest values in ; otherwise, is SAT iff there exists any such that .

Theorem 1.

Let be the -th image set of the optimization problem constrained by , where , and . There exists an , such that , and .

Proof.

Initially, the minimum candidate is chosen randomly from . Considering Lemma 1, if is SAT, any (from the counterexample) is adopted as next candidate solution (i.e., , and every element from is less than . Similarly in the next iterations, while is SAT, , and every element from is less than , consequently, the number of elements of is always less than that of . Since is finite, in the -th iteration, will be empty and the is UNSAT, which leads to (Lemma 1) . ∎

Theorem 1 provides sufficient conditions for the global minimization over a finite set; it solves the optimization problem defined at the beginning of this section, iff the search domain is finite. It is indeed finite, once it is defined as an intersection between a bounded set () and a discrete set (). Thus, the CEGIO-G algorithm will always provide the minimum with decimal places (i.e., ).

5.2.1 Avoiding the Local Minima

As previously mentioned, an important feature of this proposed CEGIO method is always to find the global minimum (cf. Theorem 1). Many optimization algorithms might be trapped by local minima and they might incorrectly solve optimization problems. However, the present technique ensures the avoidance of those local minima, through the satisfiability checking, which is performed by successive SMT queries. This property is maintained for any class of functions and for any initial state.

Figures 4 and 5 show the aforementioned property of this algorithm, comparing its performance to the genetic algorithm. In those figures, Ursem03’s function is adapted for a single-variable problem over , i.e., is considered fixed and equals to , and the respective function is reduced to a plane crossing the global optimum in . The partial results after each iteration are illustrated by the various marks in these graphs. Note that the present method does not present continuous trajectory from the initial point to the optimal point; however, it always achieves the correct solution. Fig. 4 shows that both techniques (GA and SMT) achieve the global optimum. However, Fig. 5 shows that GA might be trapped by the local minimum for a different initial point. In contrast, the proposed CEGIO method can be initialized further away from the global minimum and as a result it can find the global minimum after some iterations, as shown in Figures 4 and 5.

Figure 4: Optimization trajectory of GA and SMT for a Ursem03’s plane in . Both methods obtain the correct answer.
Figure 5: Optimization trajectory of GA and SMT for a Ursem03’s plane in . GA is trapped by an local minimum, but SMT obtains the correct answer.

5.3 A Simplified Algorithm for CEGIO (CEGIO-S)

Alg. 1 is suitable for any class of functions, but there are some particular functions that contain further knowledge about their behaviour (e.g., positive semi-definite functions such as ). Using that knowledge, Alg. 1 is slightly modified for handling this particular class of functions. This algorithm is named here as “Simplified CEGIO algorithm” (CEGIO-S) and it is presented in Alg. 2.

Note that Alg. 2 contains three nested loops after the variable initialization and declaration (lines 1-4), which is similar to the algorithm presented in Araujo2016 . In each execution of the outer loop while (lines 5-25), the bounds and precision are updated accordingly. The main difference in this algorithm w.r.t the Alg. 1 is the presence of the condition in line , i.e., it is not necessary to generate new checks if that condition does not hold, since the solution is already at the minimum limit, i.e., .

Furthermore, there is another inner loop while (lines 12-15), which is responsible for generating multiple VCs through the ASSERT directive, using the interval between and . Note that this loop generates VCs through the step defined by in line 8.

input : A cost function , the space for constraint set , a desired precision , and a learning rate
output : The optimal decision variable vector , and the optimal value of function
1 Initialize Initialize randomly and Initialize the precision variable with Declare the auxiliary variables x as non-deterministic integer variables while  do
2        Define bounds for x with the ASSUME directive, such that Describe a model for Declare if  then
3               do
4                      Constraint with the ASSUME directive while  do
5                             Verify the satisfiability of given by Eq. (6) with the ASSERT directive Do 
6                      end while
7                     Update and based on the counterexample Do 
8              while  is satisfiable
9        end if
10       else
11              break
12        end if
13       Update the precision variable
14 end while
and return and
Algorithm 2 CEGIO: a simplified algorithm.

These modifications allow Alg. 2 to converge faster than Alg. 1 for the positive semi-definite functions, since the chance of a check failure is higher due to the larger number of properties. However, if represents a large number, then the respective algorithm would produce many VCs, which could cause the opposite effect and even lead the verification process to exhaust the memory.

6 Counterexample Guided Inductive Optimization of Convex Problems

This section presents the fast CEGIO algorithm for convex optimization problems. Subsection 6.1 presents the convex optimization problems, while the fast SMT algorithm is explained in Subsection 6.2. Additionally, a convergence proof of the CEGIO convex problem is described in Subsection 6.3.

6.1 Convex Optimization Problems

Convex functions are an important class of functions commonly found in many areas of mathematics, physics, and engineering li2015selected . A convex optimization problem is similar to Eq. (1), where is a convex function, which satisfies Eq. (8) as

(8)

for all , with and all , with , , .

Theorem 2 is an important theorem for convex optimization, which is used by most convex optimization algorithms.

Theorem 2.

A local minimum of a convex function , on a convex subset, is always a global minimum of Boyd:2004:CO:993483 .

Here, Theorem 2 is used to ensure convergence of the CEGIO convex optimization algorithm presented in Subsection 6.2.

6.2 Fast CEGIO (CEGIO-F)

Alg. 1 aforementioned evolves by increasing the precision of the decision variables, i.e., in the first execution of its while loop, the obtained global minimum is integer since , called . Alg. 3 is an improved algorithm of that Alg. 1 for application in convex functions. It will be denoted here as fast CEGIO algorithm. Note that, the only difference of Alg. 1 is the insertion of line , which updates before of . For each execution of the while loop, the solution is optimal for precision . A new search domain is obtained from a CEGIO process over , defining as follows: , where is the solution with decimal places.

input : A cost function , the space for constraint set , and a desired precision
output : The optimal decision variable vector , and the optimal value of function
1 Initialize randomly and Initialize the precision variable with Declare the auxiliary variables x as non-deterministic integer variables while  do
2        Define bounds for x with the ASSUME directive, such that Describe a model for do
3               Constrain with the ASSUME directive Verify the satisfiability of given by Eq. (6) with the ASSERT directive Update and based on the counterexample Do 
4       while  is satisfiableUpdate set Update the precision variable p
5 end while
and return and
Algorithm 3 Fast CEGIO.

6.3 Proof of Convergence for the Fast CEGIO Algorithm

The fast CEGIO algorithm computes iteratively for every . Theorem 1 ensures the global minimization for any finite . The global convergence of the fast CEGIO algorithm is ensured iff the minima of any is inside . It holds for the generalized algorithm since . However, the fast CEGIO algorithm modifies boundaries using the -th solution.

Lemma 2.

Let be a convex function, as is a finite set, Theorem  1 ensures that the minimum, in is a local minimum for precision , where . In addition, as is a convex function, any element outside has its image ensured by Eq. (8).

Lemma 2 ensures that the solution is a local minimum of , and Theorem 2 ensures that it is a global minimum. As a result, bounds of can be updated on each execution of the outer while loop; this modification considerably reduces the state-space searched by the verifier, which consequently decreases the algorithm execution time.

7 Experimental Evaluation

This section describes the experiments design, execution, and analysis for the proposed CEGIO algorithms. We use the ESBMC tool as verification engine to find the optimal solution for a particular class of functions. We also compare the present approaches to other exisiting techniques, including genetic algorithm, particle swarm, pattern search, simulated annealing, and nonlinear programming. Preliminary results allowed us to improve the experimental evaluation as follows.

  1. There are functions with multiplication operations and large inputs, which lead to overflow in some particular benchmarks. Thus, the data-type float is replaced by double in some particular functions to avoid overflow.

  2. ESBMC uses different SMT solvers to perform program verification. Depending on the selected solver, the results, verification time, and counterexamples can be different. This is observed in several studies TrindadeDAES2016 ; Abreu2016 ; Bessa2016 ; Pereira:2016:VCP:2851613.2851830 ; as a result, our evaluation here is also carried out using different SMT solvers such as Boolector boolector , Z3 z3 , and MathSAT mathsat5 , in order to check whether a particular solver heavily influences the performance of the CEGIO algorithms.

  3. There are functions that present properties which permits the formulation of invariants to prune the state-space search, e.g., functions that use absolute value operators (or polynomial functions with even degree); those functions will always present positive values. As a result, the optimization processes can be simplified, reducing the search domain to positive regions only. Such approach led to the development of Algorithm 2, which aims to reduce the verification time.

All experiments are conducted on a otherwise idle computer equipped with Intel Core i7-4790 CPU 3.60 GHz, with 16 GB of RAM, and Linux OS Ubuntu 14.10. All presented execution times are CPU times, i.e., only time periods spent in allocated CPUs, which were measured with the times system call (POSIX system).

7.1 Experimental Objectives

The experiments aim to answer two research questions:

  1. (sanity check) what results do the proposed CEGIO algorithms obtain when searching for the functions optimal solution?

  2. (performance)

    what is the proposed CEGIO algorithms performance if compared to genetic algorithm, particle swarm, pattern search, simulated annealing, and non-linear programming?

7.2 Description of Benchmarks

In order to answer these research questions, we consider reference functions of global optimization problems extracted from the literature DBLP:journals/corr/JamilY13 ; all reference functions are multivariable with two decision variables. Those functions present different formats, e.g., polynomials, sine, cosine, floor, sum, square root; and can be continuous, differentiable, separable, non-separable, scalable, non-scalable, uni-modal, and multi-modal. The employed benchmark suite is described in Table 1 as follows: benchmark name, domain, and global minimum, respectively.

# Benchmark Domain Global Minima
1 Alpine 1
2 Booth
3 Chung
4 Cube
5 Dixon & Price
6 Egg Crate
7 Himmeblau
8 Leon
9 Power Sum
10 Price 4
11 Engvall
12 Schumer
13 Tsoulos
14 Branin RCOS
15 Schuwefel 2.25
16 Sphere
17 Step 2
18 Scahffer 4
19 Sum Square
20 Wayburn Seader 2
21 Adjiman
22 Cosine
23 S2
24 Matyas
25 Rotated Ellipse
26 Styblinski Tang
27 Trecanni
28 Ursem 1
29 Zettl
30 Zirilli
Table 1: Benchmark Suite for Global Optimization Problems.

In order to perform the experiments with three different CEGIO algorithms, generalized (Alg. 1), simplified (Alg. 2), and fast (Alg. 3), a set of programs were developed for each function, taking into account each algorithm and varying the solver and the data-type accordingly. For the experiment with the generalized algorithm, all benchmarks are employed; for the simplified algorithm, functions are selected from the benchmark suite. By previous observation, we can affirm that those functions are semi-definite positive; lastly, we selected convex functions from the benchmark suite to evaluate the fast algorithm.

For the experiments execution with the proposed algorithms, random values are generated, belonging to the solutions space of each function, and they are used as initialization of the proposed algorithms, as described in Section 5. The other optimization techniques used for comparison, had all benchmarks performed by means of the Optimization Toolbox in MATLAB 2016b optoolboxmanual with the entire benchmark suite. The time presented in the following tables are related to the average of executions for each benchmark; the measuring unit is always in seconds based on the CPU time.

7.3 Experimental Results

In the next subsections, we evaluate the proposed CEGIO algorithms performance; we also compare them to other traditional techniques.

7.3.1 Generalized Algorithm (CEGIO-G) Evaluation

The experimental results presented in Table 2 are related to the performance evaluation of the Generalized Algorithm (CEGIO-G) (cf. Alg. 1). Here, the CPU time is measured in seconds to find the global minimum using the ESBMC tool with a particular SMT solver. Each column of Table 2 is described as follows: columns 1 and 5 are related to functions of the benchmark suite; columns 2 and 6 are related to the configuration of ESBMC with Boolector; columns 3 and 7 are related to ESBMC with Z3; and columns 4 and 8 are related to ESBMC with MathSAT.

# Boolector Z3 MathSAT # Boolector Z3 MathSAT
1 537 6788 590 16 1 1 2
2 2660 972 5 17 3 1 11
3 839* 5812* 2 18 6785 14738 33897
4 170779* 77684* 5 19 41 1 3
5 36337* 22626* 8 20 33794* 36324 37
6 5770 3565 500 21 665 2969 19313
7 4495 11320 10 22 393 2358 3678
8 269 1254 4 23 32 13 10
9 3 40 4 24 5945 5267 23
10 16049* 110591 6 25 1210 2741 16
11 1020 3653 662 26 1330 19620 438
12 445 20 4 27 76 269 2876
13 305 9023 2865 28 808 645 11737
14 17458 25941 3245 29 271 611 11
15 2972 5489 7 30 383 720 662
Table 2: Experimental Times Results with the Generic Algorithm (in seconds).

All benchmarks are employed for evaluating the generalized algorithm performance. The correct global minima is found in all benchmarks using different SMT solvers: MathSAT, Z3, and Boolector. For all evaluated benchmarks, MathSAT is times faster than Z3, although there are benchmarks in which MathSAT took longer than Z3, e.g., in Adjiman and Cosine functions. If we compare Boolector performance to other SMT solvers, we can also observe that it is routinely faster than both Z3 and MathSAT.

Initially, all experiments were performed using float-type variables, but we noticed that there was either overflow or underflow in some particular benchmarks, e.g., the Cube functions. It occurs due to truncation in some arithmetic operations and series, e.g., sines and cosines, once the verification engine employs fixed-point for computations. This might lead to a serious problem if there are several operations being performed with very large inputs, in a way that causes errors that can be propagated; those errors thus lead to incorrect results. For this specific reason, we decided to use double-type variables for these particular benchmarks to increase precision. We observed that the global minimum value is always found using double precision, but it takes longer than using float-type variables. The cells with asterisks in Table 2 identify the benchmarks that we use double- instead of float-type.

Additionally, we observed that when the function has more than one global minimum, e.g., Wayburn Seader 2 with the decision variables , the algorithm first finds the global minimum with the decision variables of less precision, then in this case . Analyzing Alg. 1, when an overall minimum value is found, the condition in line is not satisfied since there is no candidate function with a value less than the current one found; on line the precision is updated and the outer loop starts again. Even if there is another overall minimum in this new precision, it will not be considered by the ASSUME directive in line since the decision variables define a candidate function with the same value as the current function , and not less than the defined in Eq. 6. In order to find the other global minimum, it would be necessary to limit it with the ASSUME directive, disregarding the previous minimum.

7.3.2 Simplified Algorithm (CEGIO-S) Evaluation

The simplified algorithm (CEGIO-S) is applied to functions that contain invariants about the global minimum, e.g., semi-definite positive functions, where it is not needed to search for their minimum in the negative values. For instance, the leon function presented in Eq. (9) has the global minimum at as follows

(9)

By inpesction it is possible to claim that there are no negative values for . Therefore, in order to effectively evaluate Algorithm 2, benchmarks are selected, which have modules or exponential pair, i.e., the lowest possible value to global minimum is a non-negative value. The experiments are performed using the float data-type, and double as needed to avoid overflow, using the same solvers as described in Subsection 7.3.1. According to the experimental results shown in Table 3, we confirmed that all obtained results match those described in the literature DBLP:journals/corr/JamilY13 .

# CEGIO-S CEGIO-G
Boolector Z3 MathSAT Boolector Z3 MathSAT
1 74 2 413 537 6788 590
2 <1 <1 1 2660 972 5
3 <1 <1 <1 839* 5812* 2
4 <1 <1 2 170779* 77684* 5
5 14 2 6 36337* 22626* 8
6 34 2 240 5770 3565 500
7 1 1 6 4495 11320 10
8 1 <1 2 269 1254 4
9 <1 <1 2 3 40 4
10 <1 <1 5 16049* 110591 6
12 <1 <1 1 445 20 4
15 1 2 5 2972 5489 7
16 <1 <1 <1 1 1 2
19 <1 <1 <1 41 1 3
20 215 2446 30 33794* 36324 37
Table 3: Experimental Results with the Simplified Algorithm (in seconds).

Additionally, we can see that the simplified algorithm reduces the optimization time considerably, with particular benchmarks reaching less than second. However, the reduction with the MathSAT solver is less expressive since it models float-type variables using floating-point arithmetic in both CEGIO-S and CEGIO-G algorithms, while Boolector and Z3 uses fixed-point arithmetic. We conclude that either our fast algorithm is suitable for fixed-point architectures or MathSAT implements more aggressive simplifications than Boolector and Z3

The purpose of this algorithm is to find the global minimum to reduce the verification time, for functions that have invariants about the global minimum. However, the simplified algorithm run-time might be longer than the generalized one since it requires parameter settings according to the function. As described in Subsection 5.3, in line of Algorithm 2, we have the variable that defines the state-space search segmentation; is obtained by the difference of the current and the boundary that we know, divided by the variable (previously established). If we have a very large absolute value for , then we would have additional checks, thus creating many more properties to be checked by the verifier (and thus leading it to longer verification times).

If we analyze function S2 in Eq. (10), then we can easily inspect that there is no less than ; in this case, therefore, in line of Algorithm 2, one can establish with the value . This slightly change in the initialization of in Algorithm 2 prunes the state-space search and the verification time accordingly.

(10)

7.3.3 Fast Algorithm (CEGIO-F) Evaluation

The experimental results for the fast algorithm (CEGIO-F) are presented in Table 4. This algorithm is applied to convex functions, where there is only a global minimum; in particular, the state-space is reduced on each iteration of the while-loop in Algorithm 3, ensuring that the global minimum is in the new (delimited) space, and then it performs a new search in that space to reduce the overall optimization time.

In order to evaluate the effectiveness of Algorithm 3, we selected approximately convex functions of the benchmark suite; we also compare the fast algorithm (CEGIO-F) results with the generalized one (CEGIO-G). We observed that there are significant performance improvements if we compare CEGIO-F to CEGIO-G for convex function benchmarks, i.e., CEGIO-F algorithm is times faster using the SMT solver Boolector and times faster using the SMT solver Z3 than the (original) CEGIO-G algorithm, as shown in Table 4.

# CEGIO-F CEGIO-G
Boolector Z3 Boolector Z3
2 <1 <1 2660 972
3 33* 26* 839* 5812*
4 43* 25 170779* 77684*
5 59* 10* 36337* 22626*
9 1 10 3 40
12 1 2 445 20
19 1 <1 41 1
24 7 2 5945 5267
25 2 1* 1210 2741
29 63* 76 271 611
Table 4: Experimental Results with the Fast Algorithm (in seconds).

7.3.4 Comparison to Other Traditional Techniques

In this section, our CEGIO algorithms are compared to other traditional optimization techniques: genetic algorithm (GA), particle swarm (ParSwarm), pattern search (PatSearch), simulated annealing (SA), and nonlinear programming (NLP).

Table 5 describes the hit rates and the mean time for each function w.r.t. our proposal (ESBMC) and other existing techniques (GA, ParSwarm, PatSearch, SA, and NLP). An identification for each algorithm is defined: (1) Generic, (2) Simplified, and (3) Fast. All traditional optimization techniques are executed times using MATLAB, for obtaining the correctness rate and the mean time for each function.

Our hit rate is omitted for the sake of space, but our algorithms have found the correct global minima in 100% of the experiments. The experiments show that our hit rate is superior than any other optimization technique, although the optimization time is usually longer.

The other optimization techniques are very sensitive to non-convexity; for this reason, they are usually trapped by local minima. The other optimization techniques presented better performance in convex functions. Specifically, they converge faster to the response and there are no local minimums that could lead to incorrect results, whereas with the non-convex functions, their hit rate is lower, precisely because there are local minimums.

# ESBMC GA ParSwarm PatSearch SA NLP
T R T R T R T R% T R% T
1 25 1 45 3 10 4 50 1 0 9
2 100 10 100 2 100 6 95 1 100 2
3 100 9 100 1 100 4 90 1 100 5
4 20 1 30 3 0 8 10 2 100 7
5 0 9 0 2 0 3 0 1 0 2
6 100 9 100 1 70 3 100 1 25 2
7 60 9 50 1 25 3 15 1 35 2
8 90 1 75 2 0 7 10 1 100 4
9 100 9 100 1 100 3 50 1 100 2
10 0 9 10 2 0 7 0 4 50 2
11 90 1 100 2 90 3 95 1 100 7
12 100 9 100 1 100 4 75 1 100 4
13 100 9 95 1 100 3 75 9 0 6
14 100 8 100 9 100 4 75 8 0 5
15 100 1 95 1 100 3 100 1 100 2
16 100 10 100 7 100 4 100 1 100 2
17 0 9 0 1 0 2 0 8 0 1
18 30 1 15 0 0 2 0
19 100 9 100 1 100 4 100 1 100 2
20 45 10 45 2 0 8 50 2 45 6
21 0 10 100 1 0 4 80 2 95 2
22 100 9 100 1 95 3 95 2 15 2
23 65 100 100 85 1 100
24 100 9 100 1 100 8 10 1 100 2
25 100 9 100 2 100 7 100 1 100 2
26 100 9 100 1 50 3 100 1 35 2
27 0 9 0 1 0 3 0 1 0 2
28 100 9 100 1 100 3 80 1 65 2
29 100 9 100 1 100 4 100 1 100 3
30 100 9 100 1 100 3 60 1 75 2
Table 5: Experimental Results with the Traditional Techniques and our Best CEGIO Algorithm (in seconds).

8 Conclusions

This paper presented three variants of a counterexample-guided inductive optimization approach for optimizing a wide range of functions based on counterexamples extracted from SMT solvers. In particular, this work proposed algorithms to perform inductive generalization based on counterexamples provided by a verification oracle for optimizing convex and non-convex functions and also presented respective proofs for global convergence. Furthermore, the present study provided an analysis about the influence of the solver and data-types in the performance of the proposed algorithms.

All proposed algorithms were exhaustively evaluated using a large set of public available benchmarks. We also evaluated the present algorithms performance using different SMT solvers and compared them to other state-of-art optimization techniques (genetic algorithm, particle swarm, pattern search, nonlinear programming, and simulated annealing). The counterexample-guided inductive optimization algorithms are able to find the global optima in 100% of the benchmarks, and the optimization time is significantly reduced if compared to Araújo et al. Araujo2016 . Traditional optimization techniques are typically trapped by local minima and are unable to ensure the global optimization, although they still present lower optimization times than the proposed algorithms.

In contrast to previous optimization techniques, the present approaches are suitable for every class of functions; they are also complete, providing an improved accuracy compared to other existing traditional techniques. Future studies include the application of the present approach to autonomous vehicles navigation systems, enhancements in the model-checking procedure for reducing the verification time by means of multi-core verification TrindadeSBESC2015 and invariant generation Rocha2015 ; Gadelha2015 . We also intend to improve Alg. 2 by implementing a dynamic learning rate since it is currently fixed in the proposed algorithm. Finally, we intend to extend all present approaches for multi-objective optimization problems.

References

References

  • (1) K. Deb, Optimization for Engineering Design: Algorithms and Examples, Prentice-Hall of India, 2004.
  • (2) D. K. Gattie, R. C. Wicklein, Curricular value and instructional needs for infusing engineering design into k-12 technology education, Journal of Technology Education 19 (1) (2007) 13. doi:10.21061/jte.v19i1.a.1.
  • (3) Y. Shoham, Computer science and game theory, Commun. ACM 51 (8) (2008) 74–79. doi:10.1145/1378704.1378721.
  • (4) J. Teich, Hardware/software codesign: The past, the present, and predicting the future, Proceedings of the IEEE 100 (Special Centennial Issue) (2012) 1411–1430. doi:10.1109/JPROC.2011.2182009.
  • (5) M. Kowalski, C. Sikorski, F. Stenger, Selected Topics in Approximation and Computation, Oxford University Press, 1995.
  • (6) U. Derigs, Optimization and Operations Research – Volume I:, 2009.
  • (7) R. Garfinkel, G. Nemhauser, Integer programming, Series in decision and control, Wiley, 1972.
  • (8) M. Bartholomew-Biggs, The Steepest Descent Method, Springer US, Boston, MA, 2008, pp. 1–8. doi:10.1007/978-0-387-78723-7_7.
  • (9)

    D. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Artificial Intelligence, Addison-Wesley Publishing Company, 1989.

  • (10) M. Cavazzuti, Optimization Methods: From Theory to Design Scientific and Technological Aspects in Mechanics, Springer Berlin Heidelberg, 2012.
  • (11) R. Alur, R. Bodik, G. Juniwal, M. M. K. Martin, M. Raghothaman, S. A. Seshia, R. Singh, A. Solar-Lezama, E. Torlak, A. Udupa, Syntax-guided synthesis, in: 2013 Formal Methods in Computer-Aided Design, 2013, pp. 1–8. doi:10.1109/FMCAD.2013.6679385.
  • (12) M. Dorigo, M. Birattari, T. Stutzle, Ant colony optimization, IEEE Computat. Intell. Mag. 1 (4) (2006) 28–39. doi:10.1109/MCI.2006.329691.
  • (13) R. Araújo, I. Bessa, L. Cordeiro, J. E. C. Filho, SMt-based verification applied to non-convex optimization problems, in: Proceedings of VI Brazilian Symposium on Computing Systems Engineering, 2016, pp. 1–8. doi:10.1109/SBESC.2016.010.
  • (14) L. De Moura, N. Bjørner, Z3: An efficient SMT solver, in: TACAS, Springer-Verlag, Berlin, Heidelberg, 2008, pp. 337–340.
  • (15) R. Brummayer, A. Biere, Boolector: An Efficient SMT Solver for Bit-Vectors and Arrays, in: Tools and Algorithms for the Construction and Analysis of Systems (TACAS), 2009, pp. 174–177.
  • (16) A. Cimatti, A. Griggio, B. Schaafsma, R. Sebastiani, The mathSAT5 SMT solver, in: Tools and Algorithms for the Construction and Analysis of Systems, 2013, pp. 93–107. doi:10.1007/978-3-642-36742-7.
  • (17) M. Jamil, X.-S. Yang, A literature survey of benchmark functions for global optimisation problems, IJMMNO 4 (2) (2013) 150–194. doi:10.1504/IJMMNO.2013.055204.
  • (18)

    A. Olsson, Particle Swarm Optimization: Theory, Techniques and Applications, Engineering tools, techniques and tables, Nova Science Publishers, 2011.

  • (19) P. Alberto, F. Nogueira, H. Rocha, L. N. Vicente, Pattern search methods for user-provided points: Application to molecular geometry problems, SIAM Journal on Optimization 14 (4) (2004) 1216–1236.
  • (20) P. J. M. Laarhoven, E. H. L. Aarts (Eds.), Simulated Annealing: Theory and Applications, Kluwer Academic Publishers, Norwell, MA, USA, 1987.
  • (21) R. H. Byrd, J. C. Gilbert, J. Nocedal, A trust region method based on interior point techniques for nonlinear programming, Mathematical Programming 89 (1) (2000) 149–185.
  • (22) R. Nieuwenhuis, A. Oliveras, On SAT Modulo Theories and Optimization Problems, Springer Berlin Heidelberg, Berlin, Heidelberg, 2006, pp. 156–169. doi:10.1007/11814948_18.
  • (23) H. Eldib, C. Wang, An SMT based method for optimizing arithmetic computations in embedded software code, IEEE CAD 33 (11) (2014) 1611–1622. doi:10.1109/TCAD.2014.2341931.
  • (24) G. G. Estrada, A note on designing logical circuits using SAT, in: ICES, Springer Berlin Heidelberg, 2003, pp. 410–421. doi:10.1007/3-540-36553-2_37.
  • (25) A. Trindade, H. Ismail, L. Cordeiro, Applying multi-core model checking to hardware-software partitioning in embedded systems, in: SBESC, 2015, pp. 102–105. doi:10.1109/SBESC.2015.26.
  • (26) A. Trindade, L. Cordeiro, Aplicando verifica o de modelos para o particionamento de hardware/software, in: SBESC, 2014, p. 6.
    URL http://sbesc.lisha.ufsc.br/sbesc2014/dl185
  • (27) A. Trindade, L. Cordeiro, Applying SMT-based verification to hardware/software partitioning in embedded systems, DES AUTOM EMBED SYST 20 (1) (2016) 1–19. doi:10.1007/s10617-015-9163-z.
  • (28) S. Cotton, O. Maler, J. Legriel, S. Saidi, Multi-criteria optimization for mapping programs to multi-processors, in: SIES, 2011, pp. 9–17.
  • (29) Y. Shoukry, P. Nuzzo, I. Saha, A. L. Sangiovanni-Vincentelli, S. A. Seshia, G. J. Pappas, P. Tabuada, Scalable lazy SMT-based motion planning, in: 55th IEEE Conference on Decision and Control, CDC 2016, Las Vegas, NV, USA, December 12-14, 2016, 2016, pp. 6683–6688. doi:10.1109/CDC.2016.7799298.
  • (30) M. Pister, M. Tautschnig, A. Bauer, Tool-support for the analysis of hybrid systems and models, 2007 10th Design, Automation and Test in Europe Conference and Exhibition 00 (2007) 172. doi:10.1109/DATE.2007.364411.
  • (31) P. Nuzzo, A. A. A. Puggelli, S. A. Seshia, A. L. Sangiovanni-Vincentelli, CalCS: SMT solving for non-linear convex constraints, Tech. Rep. UCB/EECS-2010-100, EECS Department, University of California, Berkeley (Jun 2010).
  • (32) Y. Shoukry, P. Nuzzo, A. L. Sangiovanni-Vincentelli, S. A. Seshia, G. J. Pappas, P. Tabuada, SMC: Satisfiability modulo convex optimization, in: Proceedings of the 20th ACM International Conference on Hybrid Systems: Computation and Control, HSCC ’17, ACM, New York, NY, USA, 2017, p. 10. doi:10.1145/3049797.3049819.
  • (33) N. Bjørner, A.-D. Phan, L. Fleckenstein, Z - an optimizing SMT solver, in: TACAS, Springer Berlin Heidelberg, 2015, pp. 194–199. doi:10.1007/978-3-662-46681-0_14.
  • (34) Y. Li, A. Albarghouthi, Z. Kincaid, A. Gurfinkel, M. Chechik, Symbolic optimization with SMT solvers, in: Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’14, ACM, New York, NY, USA, 2014, pp. 607–618. doi:10.1145/2535838.2535857.
  • (35) R. Sebastiani, P. Trentin, OptiMathSAT: A tool for optimization modulo theories, in: CAV, Springer International Publishing, 2015, pp. 447–454. doi:10.1007/978-3-319-21690-4_27.
  • (36) R. Sebastiani, S. Tomasi, Optimization modulo theories with linear rational costs, ACM TOCL 16 (2) (2015) 12:1–12:43. doi:10.1145/2699915.
  • (37) R. Sebastiani, P. Trentin, Pushing the envelope of optimization modulo theories with linear-arithmetic cost functions, in: TACAS, Springer Berlin Heidelberg, 2015, pp. 335–349. doi:10.1007/978-3-662-46681-0_27.
  • (38) Z. Pavlinovic, T. King, T. Wies, Practical SMT-based type error localization, in: ICFP, 2015, pp. 412–423.
  • (39) E. A. Galperin, Problem-method classification in optimization and control, Computers & Mathematics with Applications 21 (6?7) (1991) 1 – 6. doi:10.1016/0898-1221(91)90155-W.
  • (40) C. Floudas, Deterministic Global Optimization, Nonconvex Optimization and Its Applications, Springer, 2000.
  • (41)

    J. Snyman, Practical Mathematical Optimization: An Introduction to Basic Optimization Theory and Classical and New Gradient-Based Algorithms, Applied Optimization, Springer, 2005.

  • (42) D. Scholz, Deterministic Global Optimization: Geometric Branch-and-bound Methods and their Applications, Springer Optimization and Its Applications, Springer New York, 2011.
  • (43) N. V. Findler, C. Lo, R. Lo, Pattern search for optimization, Mathematics and Computers in Simulation 29 (1) (1987) 41 – 50. doi:10.1016/0378-4754(87)90065-6.
  • (44) K. Marti, Stochastic Optimization Methods, Springer, 2005.
  • (45) S. Gao, S. Kong, E. M. Clarke, dReal: An SMT Solver for Nonlinear Theories over the Reals, Springer Berlin Heidelberg, Berlin, Heidelberg, 2013, pp. 208–214. doi:10.1007/978-3-642-38574-2_14.
  • (46) E. M. Clarke, Jr., O. Grumberg, D. A. Peled, Model Checking, MIT Press, Cambridge, MA, USA, 1999.
  • (47) C. Baier, J.-P. Katoen, Principles of Model Checking (Representation and Mind Series), The MIT Press, 2008.
  • (48) D. Beyer, Reliable and Reproducible Competition Results with BenchExec and Witnesses (Report on SV-COMP 2016), Springer Berlin Heidelberg, 2016, pp. 887–904. doi:10.1007/978-3-662-49674-9_55.
  • (49) A. Biere, Bounded model checking, in: Handbook of Satisfiability, IOS Press, 2009, pp. 457–481.
  • (50) C. W. Barrett, R. Sebastiani, S. A. Seshia, C. Tinelli, Satisfiability modulo theories, in: Handbook of Satisfiability, IOS Press, 2009, pp. 825–885.
  • (51) L. Cordeiro, B. Fischer, J. Marques-Silva, SMT-based bounded model checking for embedded ANSI-C software, IEEE TSE 38 (4) (2012) 957–974. doi:10.1109/TSE.2011.59.
  • (52) D. Kroening, M. Tautschnig, CBMC – c bounded model checker (2014) 389–391doi:10.1007/978-3-642-54862-8_26.
  • (53) J. Morse, M. Ramalho, L. Cordeiro, D. Nicole, B. Fischer, ESBMC 1.22 - (competition contribution), in: TACAS, 2014, pp. 405–407.
  • (54) B. R. Abreu, Y. M. R. Gadelha, C. L. Cordeiro, B. E. de Lima Filho, S. W. da Silva, Bounded model checking for fixed-point digital filters, JBCS 22 (1) (2016) 1–20. doi:10.1186/s13173-016-0041-8.
  • (55) I. V. Bessa, H. I. Ismail, L. C. Cordeiro, J. E. C. Filho, Verification of fixed-point digital controllers using direct and delta forms realizations, DES AUTOM EMBED SYST 20 (2) (2016) 95–126. doi:10.1007/s10617-016-9173-5.
  • (56) I. Bessa, H. Ibrahim, L. Cordeiro, J. E. Filho, Verification of Delta Form Realization in Fixed-Point Digital Controllers Using Bounded Model Checking, in: SBESC, 2014.
  • (57) D. Beyer, M. E. Keremoglu, CPAchecker: A tool for configurable software verification, in: CAV, Springer Berlin Heidelberg, 2011, pp. 184–190. doi:10.1007/978-3-642-22110-1_16.
  • (58) M. Y. R. Gadelha, H. I. Ismail, L. C. Cordeiro, Handling loops in bounded model checking of C programs via k-induction, STTT 19 (1) (2017) 97–114. doi:10.1007/s10009-015-0407-9.
    URL http://dx.doi.org/10.1007/s10009-015-0407-9
  • (59) L. Li, Selected Applications of Convex Optimization, Springer Optimization and Its Applications, Springer Berlin Heidelberg, 2015.
  • (60) S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge University Press, New York, NY, USA, 2004.
  • (61) P. Pereira, H. Albuquerque, H. Marques, I. Silva, C. Carvalho, L. Cordeiro, V. Santos, R. Ferreira, Verifying CUDA programs using SMT-based context-bounded model checking, in: Proceedings of the 31st Annual ACM Symposium on Applied Computing, SAC ’16, ACM, New York, NY, USA, 2016, pp. 1648–1653. doi:10.1145/2851613.2851830.
    URL http://doi.acm.org/10.1145/2851613.2851830
  • (62) M. Jamil, X. Yang, A literature survey of benchmark functions for global optimization problems, CoRR abs/1308.4008.
    URL http://arxiv.org/abs/1308.4008
  • (63) The Mathworks, Inc., Matlab Optimization Toolbox User’s Guide (2016).
  • (64) H. Rocha, H. Ismail, L. Cordeiro, R. Barreto, Model checking embedded c software using k-induction and invariants, in: Proceedings of VI Brazilian Symposium on Computing Systems Engineering, 2015, pp. 90–95. doi:10.1109/SBESC.2015.24.