The genetic algorithm (GA) proposed by J. Holland 
is a randomized heuristic search method, based on analogy with the genetic mechanisms observed in nature and employing a population of tentative solutions. Different modifications of GA are widely used in the areas of operations research pattern recognition, artificial intelligence etc. (see e.g.[13, 16]). Despite of numerous experimental investigations of these algorithms, their theoretical analysis is still at an early stage .
In this paper, the genetic algorithms are studied from the prospective of local search for combinatorial optimization problems, and the NP optimization problems in particular. The major attention is payed to identification of the situations where the GA finds a local optimum in polynomially bounded time on average. Here and below we assume that the randomness is generated only by the randomized operators of selection, crossover and mutation within the GA. In what follows, we call a value polynomially bounded, if there exists a polynomial in the length of the problem input, which bounds the value from above. Throughout the paper we use the terms efficient algorithm or polynomial-time algorithm for an algorithm with polynomially bonded running time. A problem which is solved by such an algorithm is polynomially solvable.
1 Standard definitions and algorithm description
NP Optimization Problems.
In what follows, the standard definition of an NP optimization problem is used (see e.g. ). By we denote the set of all strings with symbols from and arbitrary string length. For a string , the symbol will denote its length. In what follows denotes the set of positive integers, and given a string , the symbol denotes the length of the string .
An NP optimization problem
is a triple , where
is the set of instances of and:
1. The relation is computable in polynomial time.
2. Given an instance , is the set of feasible solutions
of , where stands for the dimension of the search
space . Given and , the decision whether
may be done in polynomial time, and for some polynomial poly.
3. Given an instance , is the objective function (computable in polynomial time) to be maximized if is an NP maximization problem or to be minimized if is an NP minimization problem.
Without loss of generality we will consider in our analysis only the maximization problems. The results will hold for the minimization problems as well. The symbol of problem instance may often be skipped in the notation, when it is clear what instance is meant from the context.
A combinatorial optimization problem is polynomially bounded, if there exists a polynomial in , which bounds the objective values , from above.
An algorithm for an NP maximization problem has a guaranteed approximation ratio , , if for any instance it delivers a feasible solution , such that .
Neighborhoods and local optima.
Let a neighborhood be defined for every . The mapping is called the neighborhood mapping. This mapping is supposed to be efficiently computable (see e.g. ).
If the inequality holds for all neighbors of a solution , then is called a local optimum w.r.t. the neighborhood mapping .
Suppose is a metric on . The neighborhood mapping
is called a neighborhood mapping of radius defined by metric .
A local search method starts from some feasible solution . Each iteration of the algorithm consists in moving from the current solution to a new solution in its neighborhood, such that the value of objective function is increased. The way to choose an improving neighbor, if there are several of them, will not matter in this paper. The algorithm continues until it will reach a local optimum.
The simple GA proposed in  has been intensively studied and exploited over four decades. A plenty of variants of GA have been developed since publication of the simple GA, sharing the basic ideas, but using different population management strategies, selection, crossover and mutation operators .
The GA operates with populations , which consist of genotypes. In terms of the present paper the genotypes are strings from . For convenience we assume that the number of genotypes is even.
In a selection operator , each parent is independently drawn from the previous population where each individual in
is assigned a selection probability depending on itsfitness . Below we assume the following natural form of the fitness function:
if then its fitness is defined by some penalty function, such that
In this paper we consider the tournament selection operator : draw individuals uniformly at random from (with replacement) and choose the best of them as a parent.
A pair of offspring genotypes is created using the randomized operators of crossover and mutation . In general, we assume that operators and are efficiently computable randomized routines. We also assume that there exists a positive constant which does not depend on , such that the fitness of at least one of the genotypes resulting from crossover is not less than the fitness of the parents with probability at least , i.e.
for any .
When a population of offspring is constructed, the GA proceeds to the next iteration . An initial population is generated randomly. One of the ways of initialization consists in independent choice of all bits in genotypes.
To simplify the notation below, will always denote
the non-elitist genetic algorithm with the following
Generate the initial population , assign
While termination condition is not met do:
For from 1 to do:
Tournament selection: ,
The population size and tournament size , in general
may depend on problem instance . The termination condition may
be required to stop a genetic algorithm when a solution of
sufficient quality is obtained or the computing time is limited,
or because the population is ”trapped” in some unpromising area
and it is preferable to restart the search. In theoretical
analysis of the it is often assumed that the
termination condition is never met. In order to incorporate the
possibility of restarting the search, we will also consider
the iterated , which has the following outline.
Generate the initial population , assign
While termination condition is not met do:
For from 1 to do:
Tournament selection: ,
Examples of mutation and crossover operators.
As examples of crossover and mutation we can consider the well-known operators of bitwise mutation and single-point crossover from the simple GA .
The crossover operator computes , given such that with probability ,
where the random number is chosen uniformly from 1 to . With probability both parent individuals are copied without any changes, i.e. .
The bitwise mutation operator computes a genotype , where independently of other bits, each bit , is assigned a value with probability and with probability it keeps the value .
2 Expected Hitting Time of a Local Optimum
Suppose an NP maximization problem is given and a neighborhood mapping is defined. Let denote the number of all non-optimal values of objective function , i.e. . Then starting from any feasible solution the local search method finds a local optimum within at most steps. Let us compare this process to the computation of a .
Let be a lower bound on the probability that the mutation operator transforms a given solution into a specific neighbor , i.e.
The greater the value , the more consistent is the mutation with the neighborhood mapping . Let the size of population , the tournament size and the bound be considered as functions of the input data . The symbol denotes the base of the natural logarithm.
If contains a feasible solution, , , , and
then the visits a local optimum until iteration with probability at least .
Proof. Note that in the initial population, the individual of greatest fitness is a feasible solution. Let an event , , consist in fulfilment of the following three conditions:
An individual of greatest fitness in population is selected at least once when the -th pair of offspring is computed;
Mutation operator applied to performs the best improving move within the neighborhood , i.e. .
When the crossover operator is applied for computing the -th pair of offsprings, at least one of its outputs has the fitness not less than ;
Let denote the probability of union of the events . In what follows we construct a lower bound , which holds for any population containing a feasible solution. According to the outline of the , . Let us denote this probability by . Given a population , the events are independent, so . Now may be bounded from below:
Note that . Therefore
To bound probability from below, we first note that for any holds
Assume . Then in view of inequality (4), , and consequently,
Now the right-hand side expression from (6) may be used as a lower bound .
Let us now consider a sequence of populations . Note that is a lower bound for the probability to reach a local optimum in a series of at most iterations, where that at each iteration the best found solution is improved, until a local optimum is found. Indeed, suppose . Then
In view of condition (2), we find a lower bound for the probability to reach a local optimum in a sequence of at most iterations where the best found solution is improved in each iteration:
Many well-known NP optimization problems, such as the Maximum Satisfiability Problem and the Maximum Cut Problem have a set of feasible solutions equal to the whole search space . The following proposition applies to the problems with such property.
If for all and the conditions of Lemma 1 hold, then a local optimum is reached in at most iterations of the on average.
Proof. Consider a sequence of series of the iterations, where the length of each series is iterations. Suppose, denotes an event of absence of local optima in the population throughout the -th series. The probability of each event is at most according to Lemma 1. Analogously to the bound (7) we obtain the inequality
denote the random variable, equal to the number of the first run where a local optimum was obtained. By the properties of expectation (see e.g.),
Consequently, the average number of iterations until a local
optimum is first obtained is at most .
Suppose the termination condition in the iterated is . Then execution of this algorithm may be viewed as a sequence of independent runs of the , where the length of each run is iterations.
Let denote rounding up. In conditions of Lemma 1, given the parameters
the probability that finds a local optimum during the first iterations is . So the total number of populations computed in the iterated until it first visits a local optimum is at most .
The operators Mut and Cross are supposed to be efficiently computable and the tournament selection requires time. Therefore the time complexity of computing a pair of offspring in the is polynomially bounded and the following theorem holds.
If problem and the function are polynomially bounded and population contains a feasible solution at every run, then the iterated with suitable choice of parameters first visits a local optimum on average in polynomially bounded time.
Note that a slight modification of the proof of Theorem 4 from  yields the result of the above theorem in the case when the crossover operator is not used. The proof in , however, is based on a more complex method of drift analysis.
Often the neighborhood mappings for NP optimization problems are polynomially bounded, i.e. the cardinality is a polynomially bounded value . In such cases there exists a mutation operator
that generates a uniform distribution over the set, and the condition on function in Theorem 1 is satisfied.
Let denote the Hamming distance between and .
 Suppose is an NP optimization problem. A neighborhood mapping is called -bounded, if for any and holds , where is a constant.
The bitwise mutation operator with probability outputs a string , given a string . Note that probability , as a function of , , attains its minimum at . The following proposition gives a lower bound for the probability , which is valid for any , assuming that .
Suppose the neighborhood mapping is -bounded, and . Then for any and any holds
Proof. For any and we have
since . Now for , and besides that,
as . Therefore , which
implies the required inequality.
3 Analysis of Guaranteed Local Optima Problems
In this section Theorem 1
is used to estimate the GA capacity of finding the solutions with guaranteed approximation ratio.
 A polynomially bounded NP optimization problem belongs to the class GLO of Guaranteed Local Optima problems, if the following two conditions hold:
1) At least one feasible solution is efficiently computable for every instance ;
2) A -bounded neighborhood mapping exists, such that for every instance , any local optimum of with respect to has a constant guaranteed approximation ratio.
The class GLO contains such well-known NP optimization problems as Maximum Staisfiablity, Maximum Cut and the following problems on graphs with bounded vertex degree: Independent Set Problem, Dominating Set Problem and Vertex Cover .
If a problem belongs to GLO and then in view of Proposition 2, for any and , the bitwise mutation operator with satisfies the condition , where poly is some polynomial. If , then probability is bounded from below by a positive constant. Therefore, Theorem 1 implies the following
If and population at every run contains a feasible solution, then given suitable values of parameters, the iterated with bitwise mutation first visits a solution with a constant guaranteed approximation ratio in polynomially bounded time on average.
The obtained results indicate that if a local optimum is efficiently computable by the local search method, it is also computable in expected polynomial time by the iterated GA with tournament selection. The same applies to the GA without restarts, if the set of feasible solutions is the whole search space. Besides that, given suitable parameters, the iterated GA with tournament selection and bitwise mutation approximates any problem from GLO class within a constant ratio in polynomial time on average.
Supported by Russian Foundation for Basic Research grants 12-01-00122 and 13-01-00862.
-  Aarts E.H.L., Lenstra J. K. Introduction // Local Search in Combinatorial Optimization. Ed. by Aarts E.H.L., Lenstra J.K. – New York: John Wiley & Sons Ltd., 1997. – P. 1–19.
-  Ausiello G., Protasi M. Local search, reducibility and approximability of NP-optimization problems // Information Processing Letters. – 1995. – V. 54. – P. 73–79.
-  Ausiello G., Crescenzi P., Gambosi G. et al. Complexity and approximation: combinatorial optimization problems and their approximability properties. – Berlin: Springer-Verlag, 1999.
-  Balas E., Niehaus W. Optimized crossover-based genetic algorithms for the maximum cardinality and maximum weight clique problems // Jour. of Heurist. – 1998. – V. 4, N 2, – P. 107–122.
Beyer H.-G., Schwefel H.-P., Wegener I.
How to analyse evolutionary algorithms // Theor. Comp. Sci. – 2002. – Vol. 287. – P. 101–130.
-  Droste S., Jansen T., Wegener I. Upper and lower bounds for randomized search heuristics in black-box optimization // Theory Comput. Syst. – 2006. – V. 39, N 4. – P. 525–544.
Eremeev A. V.
On complexity of optimal recombination for binary representations of solutions // Evolutionary Computation. – 2008. – Vol. 16. – N 1. – P. 127–147.
-  Gnedenko B.V. Theory of probability. Gordon and Breach, 1997.
-  Goldberg D.E. A note on Boltzmann tournament selection for genetic algorithms and population-oriented simulated annealing // Complex Syst. – V. 4. – P. 445–460.
-  Holland J. Adaptation in natural and artificial systems. – Ann Arbor: University of Michigan Press. 1975.
-  Krishnakumar K. Micro-genetic algorithms for stationary and non-stationary function optimization // In SPIE Proceedings: Intelligent Control and Adaptive Systems. – 1989. pp. 289-296.
-  Lehre P. K. Fitness-levels for non-elitist populations // Proc. 13th Conf. on Genetic and Evolutionary Computation. – 2011. pp. 2075–2082.
-  Reeves C.R. Genetic algorithms for the operations researcher // INFORMS Journ. on Comput. – 1997. – V. 9, N 3. – P.231–250.
-  Reeves C. R., Rowe J. E. Genetic algorithms: principles and perspectives. – Norwell, MA: Kluwer Acad. Pbs., 2002.
-  Rödl V., Tovey C. Multiple optima in local search // Journ. of Algorithms. – 1987. – V. 8. – P. 250–259.
-  Yankovskaya A.E. Test pattern recognition with the use of genetic algorithms // Pattern Recognition and Image Analysis. – 1999. – Vol. 9. – N 1. – P. 121–123.