Texture is a visual attribute which is present in most of the nature images. The texture quantification, identification and classification is a very important problem that is defined in computer vision as the study of pixel patterns in an image region. Although this attribute is naturally processed by natural vision and easily comprehended by humans, there is no formal definition for it. Indeed, textures are complex visual patterns formed by arrangements of pixels, regions or even set of patterns formed by other visual attributes, such as shape or color. These patterns can be composed by completely distinct factors, such as pixel organization or even its disorganization. In fact, depending of the context, even the noise can be considered as a sort of texture. These characteristics of the texture attribute make it special and hard to be well defined.
Along the last years, a lot of methods has been developed for the analysis of textures. Such interest in texture analysis methods may be comprehended by the richness of the texture attribute in images analyzed in pattern recognition problems.
Basically, the texture analysis methods can be divided into 4 categories Materka et al. (1998), that is, the structural methods, in which the texture is described as a set of primitives well defined; statistical methods, in which the texture is represented through non-deterministic measures of distribution; spectral methods, based on the analysis in the frequency domain and model-based methods, based on the mathematical and physical modelling of the texture image.
Among the model-based methods, the fractal model has presented a large projection recently in the description of textures in a wide number of problems Quevedo et al. (2008); Tian-Gang, Wang, and Zhao (2007); Millan and Gonzalez-Posada (2005). Most of these methods employs the fractal dimension direct or indirectly for the representation of the texture. In recent years, however, a family of methods Backes, Casanova, and Bruno (2009); Plotze et al. (2005); Bruno et al. (2008); Florindo, De Castro, and Bruno (2010) was developed through extracting a set of features from the fractal modelling, unlike the fractal dimension which is only a unique number. Generically speaking, these methods provide the called fractal descriptors, capable of representing a texture with a higher degree of richness than the simple fractal dimension.
The ability of fractal features in the description of textures is related to the nature of fractality concept. The fractal dimension measures the complexity of a structure, which in its turn corresponds to physical important properties of a material, like the roughness, the reflectance, among others. Finally, such properties are strong stimuli in our visual identification of texture images, allowing the classfication of objects based on their texture aspects. In this way, fractal theory becomes a worthy tool in the automation of this process.
Actually, the use of term “fractal” in such kind of application may bring some controversy, once the analyzed images are not real fractals. Even because fractals are only mathematical entities lacking any perfect representation in the real world. The interested reader may appreciate this discussion for example in a letter exchange involving Avnir et al. and Mandelbrot Avnir et al. (1998); Mandelbrot (1998). Inasmuch as this debate is not in any way finished, the most acceptable approach in the literature is that presented in Carlin Carlin (2000). There, objects from real world are measured through fractal metrics even when they get away from fractal concept. In this case, the fractal measures acts as a complexity metric of the real world object.
This work proposes a novel technique for the extraction of fractal descriptors based on the Fourier fractal dimension method Russ (1994) from colored textures. The color is an important attribute in texture images, mainly those extracted from natural scenes Manjunath et al. (2001). The method consists in representing the color image in a new space color through the linear transform described in Geusebroek et al. Geusebroek et al. (2000). This transform aims at emphasizing the relation between the pixels color and their spatial distribution. Posteriorly, we apply the method for the calculus of fractal dimension by the Fourier transform. In this method, the dimension is obtained from a curve relating the power spectrum of the Fourier transform and the frequency. Instead of simply using the dimension value, this work proposes the use of the whole curve to provide the descriptors of the texture.
The proposed method takes some important advantages over classical fractal signature techniques like that based on wavelets Muneeswaran et al. (2005) or multifractal Harte (2001). One of such advantages is that the technique here presented gathers information from frequency domain inherently, allowing the capturing of details and patterns which escapes from the conventional spatial analysis. Besides, the space used allows the expression of colors and spatial distribution of pixels as being a related entity. This relation is important in many applications involving color texture. Moreover and not less relevant, the method shows a simple computational implementation and presents a low computational effort.
The performance of the proposed technique is tested in a comparison with other classical and state-of-the-art methods for color texture analysis, namely, chromaticity moments, histogram ratio and multispectral Gabor. The comparison is achieved in the classification of two color texture datasets.
This work is divided into 8 sections. The following addresses the definitions and theoretical aspects of fractal theory. The third section describes the Fourier fractal dimension. The fourth presents the concept of fractal descriptors. The fifth describes the proposed method. The following shows the experiments employed. The seventh section discusses the results and the last one does the conclusions.
The existence of strange objects which do not obey the rules of the traditional euclidian geometry is known from mathematicians from some centuries ago. Nevertheless, the formalization and denomination of such objects is due to Benoit Mandebrot in the 1970 decade Mandelbrot (1975).
A fractal is defined as a set whose Hausdorff-Besicovitch dimension exceeds strictly the topological (Euclidian) dimension. Such fact has as a consequence the fact that the fractals are dynamical systems with infinite complexity. Besides, fractals are self-similar, that is, each part of the object is a similar copy of the whole. It is noticeable that this repetition of patterns along different observation scales is also present in objects found in the nature, like the embranchment of a river, a tree or of the lung alveoli, or still in the nervures of a plant leaf, in a cloud, in a coastline and in many other cases Mandelbrot (1975).
In computational vision problems, we are interested in finding descriptors which characterize the objects in analysis. From the similarities between the aspect of natural objects and the objects studied in fractal theory, researchers started to study the application of a fractal descriptor to objects from the real world Carlin (2000). The most relevant and safe descriptor for this purpose is the fractal dimension.
ii.1 Fractal Dimension
The literature presents several definitions of dimensions which are generally named as fractal dimension Falconer (1986). Among these, we can cite the Hausdorff-Besicovitch dimension, the packing dimension, the Renyi dimension, the box-counting dimension, etc.
A common point among all these methods is that they are based on the idea of measuring at scale. The analyzed object is measured for different values of and at each different value, the details smaller than are neglected. The fractal dimension thus must express the behavior of the measure as . In a fractal object, a measure of a set must generally obey a power law:
where is a constant and is the fractal dimension of . The value of can therefore be obtained from:
In a general way, must be a homogeneous function with degree yielding to the power law
Here, we describe briefly the development of the first and perhaps most important measure of the fractal dimension, e.g., the Hausdorff dimension. For a general set , the Hausdorff dimension is defined by the following expression:
where is the -dimensional measure of , defined through:
Iii Fourier Fractal Dimension
The literature still shows alternative definitions for the fractal dimension. Among these definitions, one of the most important is the Fourier fractal dimension Falconer (1986). For the calculus of this dimension we must initially define the Fourier transform of a mass distribution , that is, a measure on a bounded subset of such that is positive and finite. The transforms are defined by:
where is a generic subset of and is the Fourier space counterpart of .
In the following we employ an analogy from the classical mechanics for the definition of the fractal dimension. We use the concept of s-potential of a mass distribution over a point in , given by:
where is an auxiliary variable.
By still extending the Physics analogy, the potential energy may be obtained through:
in which is a constant dependent on and and is the complex conjugate of . In this way:
From a theorem developed in Falconer Falconer (1986), if there is a mass distribution on the set for which the expression 10 is finite for some value(s) of , so the Hausdorff dimension of has its lower limit in . Particularly, if , for a constant value , then always converges if . The greatest for which there is a mass distribution on is called the Fourier fractal dimension of .
For practical purposes, the method for the calculus of the Fourier fractal dimension described in Russ Russ (1994) is applied to a gray-scale (intensity) image .
In this case, we have a 2D real-valued image with size and the Fourier transform is expressed through:
where and are respectively the horizontal and vertical frequency. The total frequency is given by . Another important measure is the power spectrum , given through .
Russ Russ (1994) demonstrates that there is an exponential relation between the frequency and the power spectrum in the Fourier spectrum of :
He still affirms that the exponentiation parameter
may be used for the estimation of the fractal dimensionof the texture in . In the practice, is calculated as being the slope of the curve . The dimension is easily estimated through:
In digital images applications, the Fourier transform is calculated by classical optimized techniques, like Fast Fourier Trasnform O. Brigham (1974). The Fourier spectrum is divided into radial rings (corresponding to frequency bands). Thus, the variable in the previous expression corresponds to the power spectrum averaged over each ring and the frequency corresponds to the average distance of each ring from the center of the spectrum. The Figure 1 illustrates the process.
The Figure 2 illustrates the dimension calculated by this practical method.
Iv Fractal Descriptors
Although the fractal dimension is a good descriptor for textures, shapes, contours, etc., they become inefficient in tasks which require a greater precision in the description of the object. Fractal descriptors have arrisen with the aim of filling this gap and provide a more precise technique for the characterization of the image. The Figure 3 shows graphically the importance of fractal descriptors.
The first known work applying the concept of fractal descriptor is Manoel et al. Manoel et al. (2002) which uses the technique named Multiscale Fractal Dimension (MFD) to obtain the descriptors. In this approach, the fractal dimension from the object is inferred at different observation scales. In Manoel et al. Manoel et al. (2002), the authors used the Minkowski sausage method for the calculus of the fractal dimension. In this method, the object is dilated by a variable radius and the area of the dilated object (number of pixels) is called the dilation area . The fractal dimension is estimated from the slope of the curve . Instead of simply obtaining the dimension, the authors used the whole curve to compose the descriptors by measures extracted from the curve, like the peaks, the area under the curve, etc.
In its turn, in Plotze at al. Plotze et al. (2005) and Bruno et al. Bruno et al. (2008) we have an application of MFD in which the derivative of is used in order to provide the fractal descriptors. Bruno et al. Backes, Casanova, and Bruno (2009) still applies the MFD to the analysis of textures, mapped onto surfaces and using the volumetric Minkowski sausage method.
V Proposed Method
This work proposes a novel method for the extraction of fractal descriptors from colored texture images.
A lot of methods has been described in the literature for the extraction of features from colored textures with the aim of solving problems like classification and segmentation in different application fields Caelli and Reye (1993); Paschos (1998); Jain and Healey (1998). However, many of such works do not take into account the spatiality of the color, that is, the relation between the color of a pixel and its position in the image or in a specific neghborhood.
In order to deal with this situation, Geusebroek et al. Geusebroek et al. (2000) proposed an interesting method based on physical characteristics of colors. Roughly speaking, the method consists in a linear transform from the original color space into another physical space. In the special case in which the original space is RGB (Red-Green-Blue) like it is our case, the transformation is represented by the simple expression:
where , and are the original color channels and , and are the transformed channels. corresponds to the convolution of color energy (wavelength) with a gaussian. represents the same convolution with the first derivative of a gaussian, while is the same with the second derivative of gaussian filter. In Hoang et al. Hoang, Geusebroek, and Smeulders (2005), the authors apply the classical Gabor filters to the transformed color space obtaining interesting results.
In this work, initially we apply the transform described to the texture image. In the following, for each one of the channels , and we extract fractal descriptors based on the Fourier fractal dimension described in the Section III. For the calculus of these descriptors, instead of simply calculate the fractal dimension by the Equation 13, we use all the values of in the curve, that is, the logarithm of the whole power spectrum. Thus, we appy a multiscale transform to the curve, with the aim of capturing the fractal behavior at different observation scales, in a similar manner to that described in Plotze et al.Plotze et al. (2005).
Essentially, a multiscale transform is a mapping from the original signal onto a function , where is related to the original variable and is the scale parameter. The literature presents several approaches for the calculus of multiscale transform Witkin (2003); da F. Costa and Cesar, Jr. (2000). Based on empirical results we opted for the use of space-scale approach. In this solution, the transform is formally represented through:
in which represents the zero-crossings of and expresses the convolution of with the first derivative of the gaussian , given by:
where denotes the smoothing parameter of gaussian, referenced in the most of textbooks as . The best results were achieved by calculating the derivative through the Fourier property and projecting onto the axis corresponding to . This multiscale representation based on Equations 15 and 16 uses the gaussian kernel concept. It states, using a result from partial derivative theory, that the gaussian filter associated to the first derivative is the unique operation capable of representing an image or signal under different scales without adding any spurious element in the process. More details may be seen in Witkin Witkin (2003)
Finally, we concatenate the fractal descriptors from each channel, generating the final color Fourier fractal descriptor. The steps of the method are depicted in the Figure 4, while the Figure 5 summarizes the algorithm process.
By joining the power of fractal theory and more specifically fractal descriptors in the description of natural textures and the efficiency of the color approach described in Hoang et al. Hoang, Geusebroek, and Smeulders (2005) we obtain a powerful descriptor for natural colored textures. Such descriptor is capable of capturing complex patterns in the texture which are capital for a complete and precise identification of a real scene. The Figure 6 shows in a simple example the potentiality of texture discrimination present in the proposed descriptor which may be seen even visually.
The performance of the proposed technique is tested by the classification of samples from VisTex MIT (2010), a classical dataset of colored textures and USPTex, a dataset developed in the research group of the authors, which is composed by images of natural textures, photographed in high resolution. The Figures 7 and 8 show some image samples from each dataset.
The dimensionality of proposed descriptors is proportional to the dimension of the image. Thus, for the VisTex dataset, the signature are composed by 120 descriptors, while in USPTex we use 132 descriptors. We have not used other resolutions for descriptors, once such approach may compromise the fractality measure, in which all multiscale levels has equal importance for the description of the texture.
The results obtained were compared to the use of other classical methods for descriptors of colored textures, that is, color Gabor Hoang, Geusebroek, and Smeulders (2005), color histogram ratio Paschos and Petrou (2003) and chromaticity moments Paschos (1998)1973), with
(empirically determined) and using a 10-fold cross-validation process. The comparison is done in terms of the correctness rate with its confidence interval and the confusion matrices for each descriptor and dataset.
In a first moment, we tested the classification performance of some metrics extracted from MFD curve, as it is done in the original MFD work Manoel et al. (2002)
. Here, we employed some fractal and statistical metrics, e.g., fractal dimension of channells 1, 2 and 3, mean, standard deviation, kurtosis, skewness, second and third order moments and the combination of these measures. The Table1 shows the correctness rate and associated error for each approach in Vistex dataset.
|FD Channell 1||30.51 0.20|
|FD Channell 2||33.11 0.23|
|FD Channell 3||35.38 0.14|
|Std. Dev.||26.62 0.17|
|2 Moment||26.62 0.30|
|3 Moment||29.87 0.40|
|Whole Curve||95.12 0.10|
Thus, we see that the use of whole MFD curve in the composition of descriptors provided the best result. From now, we show results for the use of the whole curve in the tested datasets.
Initially, we see in the Table 2 the global correctness rate for each compared descriptor for Vistex and USPTex dataset. It is clearly noticeable that the proposed Fourier method presented the best result in the classification of both datasets. The proposed technique presented an advantage of 2.8% in Vistex dataset and 2.7% in USPTex over Gabor method, the second best technique in this experiment.
|Moment||68.83 0.33||32.06 0.05|
|Histogram||78.89 0.21||41.49 0.18|
|Gabor||92.53 0.18||86.47 0.04|
|Fourier||95.12 0.10||88.83 0.07|
In the Table 3 we present the confusion matrices for each descriptor method. In this matrix, each raw (or column) represents a class and the value in raw and column expresses the number of objects of class , but classified as being from class
. The ideal method (with a 100% correctness rate) must present a diagonal confusion matrix.
For the Vistex dataset, the numbers correspond to the following classes (exemplified in the Figure 7): 1-Bark, 2-Fabric, 3-Food, 4-Metal, 5-Sand, 6-Tile and 7-Water. As expected, the greater the correctness rate, more the confusion matrix presents diagonal aspect. Particularly, the proposed descriptor presented its best performance in the classes 3, 4 and 5. Good results are also observed in classes 1 and 6. Specially, in class 6, the proposed technique presented a relevant advantage over Gabor method. Gabor misclassified elements from class 6 as being from class 1, 2 and 5. This is explained by the self-similarity present in these classes. The fractal method, as expected, captured more faithfully the self-similar nuances.
The confusion matrices from USPTex were represented in a different graphical manner, by using surface figures in the Figure 9. In this representation, each position in the matrix is represented by a surface point and the height of the point, according to the legend on the axis, determines the value in that position. Looking at each figure, we observe that the matrix from the Fourier method presented a more continuous diagonal with the highest points, justifying the greater number of samples correctly classified. The Gabor method presented bad results around the class 110.
This work proposed a novel technique for the calculus of descriptors for colored textures. The method uses the Fourier spectral dimension associated to the spatial color transform proposed in Geusebroek et al. Geusebroek et al. (2000). Initially, the transform is applied to the original texture images in RGB space. Following, the classical Fourier transform is applied to the image and the values in the curve are used as descriptors for the texture.
The accuracy of the proposed method was verified by a comparison with other classical techniques in color texture analysis. The results demonstrated the power of the novel technique once Fourier descriptors presented the greater precision in the classification of two complex datasets. Specially, the proposed technique showed great efficiency in capturing self-similar patterns in the texture.
The results suggest strongly that color Fourier fractal descriptors are an interesting alternative to be used in the solution of problems in which the description of an object is primordial, like tasks involving segmentation and classification of objects represented by their texture.
Odemir M. Bruno gratefully acknowledges the financial support of CNPq (National Council for Scientific and Technological Development, Brazil) (Grant #308449/2010-0 and #473893/2010-0) and FAPESP (The State of São Paulo Research Foundation) (Grant # 2011/01523-1). João B. Florindo is grateful to CNPq(National Council for Scientific and Technological Development, Brazil) for his doctorate grant.
- Mandelbrot (1975) B. B. Mandelbrot, The Fractal Geometry of Nature (Freeman, NY, USA, 1975).
- Manoel et al. (2002) E. T. M. Manoel, L. da Fontoura Costa, J. Streicher, and G. B. Müller, “Multiscale fractal characterization of three-dimensional gene expression data,” in SIBGRAPI (IEEE Computer Society, 2002) pp. 269–274.
- Bruno et al. (2008) O. M. Bruno, R. de Oliveira Plotze, M. Falvo, and M. de Castro, “Fractal dimension applied to plant identification,” Information Sciences 178, 2722–2733 (2008).
- Materka et al. (1998) A. Materka, M. Strzelecki, T. Analysis, M. A. Review, A. Materka, and M. Strzelecki, “Texture analysis methods - a review,” Tech. Rep. (Institute of Electronics, Technical University of Lodz, 1998).
- Quevedo et al. (2008) R. Quevedo, F. Mendoza, J. M. Aguilera, J. Chanona, and G. Gutierrez-Lopez, “Determination of Senescent Spotting in Banana (Musa cavendish) Using Fractal Texture Fourier Image,” Journal of Food Engineering 84, 509–515 (2008).
- Tian-Gang, Wang, and Zhao (2007) L. Tian-Gang, S. Wang, and N. Zhao, “Fractal Research of Pathological Tissue Images,” Computerized Medical Imaging and Graphics 31, 665–671 (2007).
- Millan and Gonzalez-Posada (2005) H. Millan and M. Gonzalez-Posada, “Modelling Soil Water Retention Scaling. Comparison of a Classical Fractal Model with a Piecewise Approach,” GEODERMA 125, 25–38 (2005).
Backes, Casanova, and Bruno (2009)
A. R. Backes, D. Casanova, and O. M. Bruno, “Plant leaf identification based on volumetric fractal dimension,” International Journal of Pattern Recognition and Artificial Intelligence (IJPRAI)23, 1145–1160 (2009).
- Plotze et al. (2005) R. O. Plotze, J. G. Padua, M. Falvo, M. L. C. Vieira, G. C. X. Oliveira, and O. M. Bruno, “Leaf shape analysis by the multiscale minkowski fractal dimension, a new morphometric method: a study in passiflora l. (passifloraceae),” Canadian Journal of Botany-Revue Canadienne de Botanique 83, 287–301 (2005).
- Florindo, De Castro, and Bruno (2010) J. B. Florindo, M. De Castro, and O. M. Bruno, “Enhancing Multiscale Fractal Descriptors Using Functional Data Analysis,” International Journal of Bifurcation and Chaos 20, 3443–3460 (2010).
- Avnir et al. (1998) D. Avnir, O. Biham, D. Lidar, and O. Malcai, “Applied Mathematics: Is the Geometry of Nature Fractal?” Science 279, 39–40 (1998).
- Mandelbrot (1998) B. B. Mandelbrot, “Is nature fractal?” Science. 279, 783–785 (1998).
- Carlin (2000) M. Carlin, “Measuring the complexity of non-fractal shapes by a fractal method,” Pattern Recognition Letters 21, 1013–1017 (2000).
- Russ (1994) J. C. Russ, Fractal Surfaces (Plenum Press, New York, 1994).
- Manjunath et al. (2001) B. Manjunath, J. Ohm, V. Vasudevan, and A. Yamada, “Color and Texture Descriptors,” IEEE Transactions on Circuits and Systems for video Technology 11, 703–715 (2001).
- Geusebroek et al. (2000) J.-M. Geusebroek, R. van den Boomgaard, A. W. M. Smeulders, and A. Dev, “Color and scale: The spatial structure of color images,” in Sixth Europian Conference on Computer Vision (ECCV (Springer, 2000) pp. 331–341.
- Muneeswaran et al. (2005) K. Muneeswaran, L. Ganesan, S. Arumugam, and K. R. Soundar, “Texture classification with combined rotation and scale invariant wavelet features,” Pattern Recogn. 38, 1495–1506 (2005).
- Harte (2001) D. Harte, Multifractals: theory and applications (Chapman and Hall/CRC, 2001).
- Falconer (1986) K. J. Falconer, The Geometry of Fractal Sets (Cambridge University Press, New York, NY, USA, 1986).
- O. Brigham (1974) O. Brigham, The Fast Fourier Transform (Prentice-Hall, NY, USA, 1974).
- Caelli and Reye (1993) T. Caelli and D. Reye, “On the classification of image regions by colour, texture and shape,” Pattern Recognition 26, 461 – 470 (1993).
- Paschos (1998) G. Paschos, “Chromatic correlation features for texture recognition,” Pattern Recognition Letters 19, 643 – 650 (1998).
- Jain and Healey (1998) A. Jain and G. Healey, “A Multiscale Representation Including Opponent Color Features for Texture Recognition,” IEEE Transactions on Image Processing 7, 124–128 (1998).
- Hoang, Geusebroek, and Smeulders (2005) M. A. Hoang, J.-M. Geusebroek, and A. W. Smeulders, “Color texture measurement and segmentation,” Signal Processing 85, 265 – 275 (2005).
- Witkin (2003) A. P. Witkin, “Scale space filtering: a new approach to multi-scale descriptions,” in Proceedings…, ICASSP - IEEE International Conference on Acoustics, Speech, and Signal Processing (GRETSI, Saint Martin d’H res, France, 2003) pp. 79–95.
- da F. Costa and Cesar, Jr. (2000) L. da F. Costa and R. M. Cesar, Jr., Shape Analysis and Classification: Theory and Practice (CRC Press, 2000).
- MIT (2010) MIT, “Mit vistex texture database,” (2010).
- Paschos and Petrou (2003) G. Paschos and M. Petrou, “Histogram ratio features for color texture classification,” Pattern Recognition Letters 24, 309 – 314 (2003).
- Duda and Hart (1973) R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis (Wiley, New York, 1973).