Genetic Stereo Matching Algorithm with Fuzzy Fitness

10/09/2014 ∙ by Haythem Ghazouani, et al. ∙ 0

This paper presents a genetic stereo matching algorithm with fuzzy evaluation function. The proposed algorithm presents a new encoding scheme in which a chromosome is represented by a disparity matrix. Evolution is controlled by a fuzzy fitness function able to deal with noise and uncertain camera measurements, and uses classical evolutionary operators. The result of the algorithm is accurate dense disparity maps obtained in a reasonable computational time suitable for real-time applications as shown in experimental results.



There are no comments yet.


page 2

page 3

page 4

page 6

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Evolutionary algorithms try to solve complex problems by imitating the Darwinian evolution process. In an EA, artificial critters are created to search for a solution over the space problem. The artificial critters, known as individuals are constantly competing with each other to discover optimal areas of the search space. Each individual is defined by a fixed encoding scheme representing a single possible solution to the problem. The EA is started by creating an initial population of size including randomly generated individuals [8]. An evaluation score, known as fitness, is then assigned to each individual. Fitness is generated so that the individual approaches a potential solution as well as possible. After this initial step, the evolutionary algorithm goes into the main iterative cycle. It produces children from the individuals in current population. The children are produced using perturbation of individual encoding (mutation) and recombination between two or more individuals (crossover). All individuals (children) newly created are assigned fitness scores. A second generation of population of individuals is formed from individuals in the current population and the children. The same iterative cycle is applied to the new generation and all the generations that will follow. During the whole cycle, an adaptation pressure is applied to individuals. That is, evolutionary approach of the survival of the fittest is applied and individuals try to outrace each other. The adaptation pressure is done by selection, with the fitter individuals more likely to be chosen for the next generation. The selection pressure is applied when designating individuals for crossover or when electing individuals to build a new population [8].

Genetic algorithms (GAs) are adaptive search heuristics that belong to the family of evolutionary algorithms. As such, they use natural selection and genetics to perform a random search in order to solve optimization problems. Although based on random processes, genetic algorithms are not entirely arbitrary, instead they use fitness scores to guide the search towards areas of better performance within the search space. The key techniques of GAs are performed to mimic the natural evolutionary processes, particularly those adopt the Darwinian rule of ”survival of the fittest”. Since in nature, the competition between individuals to achieve results in terms of scarce resources results in the fittest individuals surviving at the expense of the weaker ones [16].

The application of artificial genetic algorithms in various fields such as image processing, pattern recognition or machine learning has yielded encouraging results [1, 9]. In particular, several genetic approaches have been used to solve the problem of stereo matching [2, 3, 4]. To obtain a disparity map, these methods use fitness functions defined from similarity and disparity smoothness constraints. They start by generating an initial population of individuals (chromosomes). Each individual encodes a possible match with respect to local constraints. After that, the evolutionary process is launched to reach a solution for which the pairing is as compatible as possible with respect to matching constraints. Genetic stereo matching algorithm provides good results, but its major drawback is the computational effort required to achieve a satisfactory solution. This disadvantage can be explained by the fact that classical genetic approaches use binary encodings for individuals, which leads to some matching ambiguities [5]. Furthermore, a binary encoding requires more storage space and more computing time. To get around these limitations, we propose a new encoding for individuals, more compact than binary encoding and requiring much less space. Reducing the storage space has the immediate effect of reducing the computational time and allows the search algorithm to explore the search space more efficiently [5]. Thus, the convergence time is considerably improved. The proposed algorithm uses also a fuzzy fitness, to overcome noise and uncertain camera measurement problems. In the next section, we present the proposed algorithm.

Ii The genetic fuzzy stereo-matching algorithm

In stereo-vision problems, we have a pair of images depicting the same scene, taken from two different angles of view. The goal is to determine the corresponding pixels (the projections of the same punctual area on both images). The first idea that comes to the mind is comparing the areas around the two pixels to have a similarity score. Once the similarity score calculated, the result can be improved by including restrictions and calculating the matching that maximizes the global similarity [6, 14, 15]. The epipolar restriction is used to reduce the search space [6].

Genetic algorithms are adaptive search methods based on the principles of evolution theory, i.e. crossover, mating, mutation, natural selection, fitness. These principles are adapted to be applied in the context of stereo matching.

Before defining genetic operators, we give assumptions on which our work is based: The input images used in this work are rectified to have a horizontal epipolar line. Consequently, the disparity depends only on the column index of the pixel: the pixel in the reference image is paired to the pixel in the target image. A disparity space is defined. Dimension of the disparity space are , and to designate respectively row, column and disparity. Each element of the disparity space is projected to the pixel in the reference image and in the target image. The element refers to the pairing of the pixel of the reference image and the pixel of the target image. Let be the image size.

Ii-a New encoding scheme

In genetic algorithms, a population of individuals representing potential solutions is preserved within the space of search. Each individuals is represented by a finite length vector of components, or variables, by means of some symbols, mostly the binary symbols


To continue the genetic analogy these individuals are likened to chromosomes and the variables are analogous to genes. Thus a chromosome (potential solution) is composed of several genes (variables) [16].

State of-the-art genetic stereo algorithms provide good matching results, but their major limitation is the computational time. This disadvantage can be explained by the fact that classical genetic approaches use binary encodings for individuals, which leads to some matching ambiguities. Furthermore, a binary encoding requires more storage space and more computing time [5]. To get around these limitations, we propose a new encoding for individuals, more compact than binary encoding and requiring much less space.

In stereo matching problems, the solution is a disparity map. A chromosome is an encoding scheme of a potential solution. Consequently, we define a chromosome as a matrix with the same size as the input image. The value of the cell of the chromosome matrix represents the disparity between the pixel in the reference image (the right image) and the corresponding pixel in the target image.

Fig. 1: A chromosome representing a 8x8 image, the value of the element (5,4) is 3 which means that pixel (5,4) in the reference image corresponds to the pixel (5,7) in the target image.

Ii-B Fuzzy fitness

In stereo matching, corresponding pixels are identified using several constraints such as similarity constraint and disparity smoothness. Similarity assumption assumes that the projections of the same punctual area of the scene have nearly the same intensities.

Disparity smoothness constraint assumes that the disparities are smooth in a local neighbourhood. Accordingly, the evaluation function is formulated based on similarity and smoothness constraints.

Similarity assumption is commonly used by stereo vision approaches to define a static fitness function by calculating the difference of intensities between two neighbourhood of the candidate pixels. A static fitness is not robust to noise caused by changes of illumination conditions, sampling, scanning… In this work, we propose to calculate a fuzzy quantification of the similarity assumption which can model more efficiently non ergodic phenomena. For this purpose, a grey scale classification of pixels is defined [14]. This classification is based on three classes; black pixels, white pixels and average pixels. We define membership functions (Eq.1) of theses classes as Gaussian centered in 0, 127.5 and 255.


is the intensity at the pixel , and

are respectively the center and the standard deviation of the class under consideration. Based on this classification and using similarity assumption, we can assume that the matching of two pixels

and projections of the same punctual area M on the stereo images, is ”possible” if the two pixels are in the same grey class, that means ( is black AND is black) OR ( is white AND is white) OR ( is average AND is average). Using classical fuzzy operators, we define a fuzzy matching possibility metric: (given by Eq. 2). is a measure of co-membership to a same grey class. It reflects how much it is ”possible” to have and as corresponding pixels.


is the degree of membership of the pixel to the class under consideration. The possibility of matching ranges between and . Thereafter, we will use the notation = with and .

During the evolutionary process, the fittest individual is determined by a fitness function, also known as the evaluation function. This function aims to direct the selection by assigning a fitness score to each chromosome of each generation.

As classical approaches, our fitness function uses intensity similarity and disparity smoothness. But instead of using the difference of intensity measurements, which can be easily affected by noise, we use the matching possibilities. That makes the proposed fuzzy fitness more robust to noise, change of view point, occlusions… The Fitness of an individual matrix C is given by Eq. 3.


is the disparity value of the cell within the chromosome matrix . is a neighbouring introduced to have a discriminating comparison between the projections. and are Sobel gradient norms respectively on reference pixel and target pixel . That is intended to penalize pixels which project onto uniform regions, i.e. less significant pixels.

Ii-C Genetic operators

Some genetic operators are intended to preserve diversity and to avoid local optima such as mutation. Other operators are used to combine best individuals into others such as crossover [7]. Offspring is generated from two chosen individuals from the current population by exchanging a part of the matrices (Figure 2). The child chromosome acquires some characteristics from each parent. The crossover line is chosen randomly in the interval for a image.

Fig. 2: Example of a crossover of two chromosomes in a random dot matrices

Mutation is randomly applied to individuals to produce a varied offspring. This operator randomly changes one or several cells (a neighbouring N of uniform cells) in an individual (Figure 3). Offspring may, thus, inherit several characteristics from their parents. Mutation prevents falling into local optima and increases the chance to find global optima. The mutation rate is set to 40% and affects matrix cells with lower matching possibilities.

Fig. 3: Example of a chromosome mutation in a random dot matrices

Ii-D Selection

Once the choice of the chromosome encoding is fixed, genetic algorithm starts by generating the initial population. Subsequently, a varied offspring is produced using crossover and mutation operators. All the individuals in a given generation are evaluated by the proposed fuzzy fitness function. Based on this evaluation, some chromosomes are selected to play the role of parents in the next generation. This iterative process is repeated until a stop condition is reached.

In our algorithm, we use an elitist deterministic selection. Chromosomes are ordered according to their evaluation score and only the fittest individuals are retained (around 40%). The fitness used to evaluate all individuals is given by Eq. (3). The fitness of an individual represents his survival probability to be selected for the next generation. Sigmoid or rank functions are commonly used to convert from fitness to a survival probability. Selection is controlled by a random function which generates values between 0 and 1 for each selection. If the fitness of the current individuals is greater than the generated probability, then this individual will be chosen for the next generation. The evolution process is ended when the fittest individual of the current generation satisfies the stop condition.

Iii Experimental results

In this work, we acquire images using a pre-calibrated stereo rig. We also consider that the reference and target images are rectified so that epipolar lines are horizontal and have the same line number in the reference and target images. We assume that the rectification is correct, which means that the disparity depends only on the column index of the pixel: the pixel in the reference image is matched to the pixel in the target image.

The performance of our approach was assessed on the Middlebury benchmark site [12], which is well recognized and widely used by the computer vision community. The site contains standard datasets for researchers to experiment with using an on-line interface. Also, it contains results for the most current algorithms and their relative ranking.

First pair of stereo images in Figure 4 shows the Teddy stereo pair and the ground truth in (A), the disparity maps obtained with different parameters in (B).

Fig. 4: A) Reinforce image of Teddy (left) and Ground truth map (right), B) Disparity map after 10 generations with initial population of 40 individuals (Left), Disparity map after 100 generations with initial population of 70 individuals (Right).

Figure 5 below presents some results in test images. The top two images (A) are the reference images of Sawtooth and Venus, the two images in (B) are ground truth (of the disparity map) and the bottom two images correspond to the disparity maps obtained (on the Middlebury site) by our approach after 80 generations. All of the performance evaluations were conducted on-line on the Middlebury evaluation site [12], according to the benchmark protocol and the test datasets (of 4 stereo pairs) agreed upon. The error rates obtained in this evaluation were 15.3%. Note that all disparity maps were obtained with an initial population of 70 individuals and a mutation rate of 40%, used in all of the accuracy evaluation test data on the Middlebury site.

Fig. 5: A) Reference images of the Sawtooth (left) and Venus (right) stereo pair, B) Ground truth disparity maps, C) Disparity maps after 100 generations

A main evaluation criterion used is the number of incorrect pixels, i.e., pixels for which the absolute disparity difference between the solutions provided and the ground truth is greater than 1. Table I shows the percentage of incorrect pixels obtained via the Middlebury evaluation site for our approach.

Tsukuba Venus Teddy Cones
error rate 2,74% 2,5% 14,7% 7,78%
TABLE I: Error rate, according to the Middlebury evaluation site for our approach

To check the efficiency of our algorithm, we performed a number of tests using the sawtooth image. Three genetic stereo matching algorithms (Table II) are considered for the comparison. The experiment is made on a dual core with 3GHz and 1Go of RAM. Table II illustrates the computational time for sawtooth with a half-size image (217x190). We have used an initial population of 50 individuals and the maximum number of generations is set to 10 in this test. Table II shows the improvement in computational time of our algorithm compared to state of-the-art algorithms. Results show that our algorithm is suitable for real-time application.

Algorithms Execution time
Our algorithm 3.4 s
Han’s algorithm [11] 9.7 s
Dong’s algorithm [10] 5 s
Nguyen’s algorithm [13] 2.7 s
TABLE II: Comparison of execution time in seconds with existing stereo matching algorithms for sawtooth image with a half-size image (217x190) after 10 generations.

Iv Conclusion

We presented in this paper a genetic fuzzy algorithm for the stereo matching problem. In this algorithm a new encoding scheme was proposed. A fuzzy formulation of the similarity assumption is used to propose a matching possibility metric. A fuzzy fitness function using matching possibilities and disparity smoothness is defined to evaluate individuals. According to the proposed encoding scheme, genetic operators were adapted. Experimental results show that the presented algorithm yields a significant improvement in accuracy and computational time relative to state of-the-art genetic stereo matching algorithm.


  • [1] D. Snyers and Y. Pétillot, ”Image processing optimization by genetic algorithm with a new coding scheme”, Pattern Recognition Letters, 843-848, 1995.
  • [2] Z. Zhang, C. Hou and N. J. Yang, ”A Stereo Matching Algorithm based on Genetic Algorithm with Propagation Stratagem”, International Workshop on Intelligent Systems and Applications, pp. 1-4, Wuhan, May 2009.
  • [3] D-J Lee, J. D. Anderson and J. K. Archibald, ”Hardware Implementation of a Spline-Based Genetic Algorithm for Embedded Stereo Vision Sensor Providing Real-Time Visual Guidance to the Visually Impaired”, EURASIP J. Adv. Sig. Proc. 2008.
  • [4] A. Arranz, M. Alvar , J. Boal , A. Sánchez and A. de la Escalera, ”Genetic algorithm for stereo correspondence with a novel fitness function and occlusion handling”, 8th International Conference on Computer Vision Theory and Applications -VISAPP 2013. Barcelone, Spain. 21-24 February, 2013.
  • [5] Y. Ruichek, H. Issa and J. G. Postaire, ”An Evolutionary-Based Stereo Matching Method with a Multilevel Searching Strategy”, Soft Computing, Vol. 10, No. 12 pp. 1145-1159, 2006.
  • [6] M. D. Medeiros, L. M. G. Gonçalves and A. C. Frery, ”Using fuzzy logic to enhance stereo matching in multiresolution images”, Sensors, Vol. 10, No 2, pp. 1093-1118, 2010.
  • [7] M. Kumar, M. Husian , N. Upreti and D. Gupta, ”Genetic algorithm: review and application”, International Journal of Information Technology and Knowledge Management, Vol. 2, No. 2, pp. 451-454, July-December 2010.
  • [8] G. Jones, ”Genetic and Evolutionary Algorithms” Encyclopedia of Computational Chemistry, (Schleyer, P.v.R. et al., eds), pp. 1127-1136, John Wiley & Sons, 1998.
  • [9] V. Bevilacqua, G. Mastronardi, F. Menolascina and D. Nitti, ”Stereo-Matching Techniques Optimisation Using Evolutionary Algorithms”, Lecture Notes in Computer Science, Vol. 4113, pp 612-621, 2006.
  • [10] N. Dong-Hu and E. Harbin, ”Stereo Matching Algorithm Using Population-Based Incremental Learning on GPU”, International Workshop on Intelligent Systems and Applications, pp. 1-4 Wuhan, 2009.
  • [11] K-P. Han, T. Bae and Y-H Ha, ”Hybrid Stereo Matching with a New Relaxation Scheme of Preserving Disparity Discontinuity”, Pattern Recognition, Vol.33, No.5, pp.767-785, 2000.
  • [12] Middlebury stereo vision page Available Online:
  • [13] V. D. Nguyen, T. T. Nguyen, D. D. Nguyen, SangJun Lee and J. W. Jeon, ”A Fast Evolutionary Algorithm for Real-Time Vehicle Detection”, IEEE T. Vehicular Technology Vol. 62, No. 6, pp. 2453-2468, 2013.
  • [14] H. Ghazouani, M. Tagina and R. Zapata, ”Fast and Robust Semi-Local Stereo Matching Using Possibility distributions”, International Journal of Computational Vision and Robotics (IJCVR) - Vol. 2, No.3 pp. 237-253, 2011.
  • [15] H. Ghazouani, ”Navigation visuelle de robots mobiles dans un environnement d’intérieur”, Doctoral dissertation, Université Montpellier II-Sciences et Techniques du Languedoc, 2012.
  • [16] P. Bajpai, & M. Kumar, ”Genetic algorithm - an approach to solve global optimization problems”, Indian Journal of computer science and engineering, Vol. 1, No. 3, 199-206, 2010.