1 Introduction
Image segmentation is the process of dividing an image into meaningful regions. For a segmentation technique to be useful for image analysis and interpretation, the separated regions should strongly relate to depicted objects or features of interest. Image segmentation is an important step in various image processing tasks as it transforms a lowlevel image to highlevel image descriptions, in terms of features, objects, and scenes. A wide variety of methods exist in literature, with most of the segmentation algorithms belonging to one of the following broad categories: threshold based, edgedetection based, region based or based on clustering techniques. reviewpaper1 and reviewpaper2 provide excellent reviews of existing stateoftheart image segmentation techniques and applications.
In this paper, a theoretical model from statistical physics, which aims at studying the properties of a population of interacting agents, is applied to the problem of image segmentation. The model described in this paper, and such other models, were originally motivated by the interaction of molecules in fluids, and have been used widely in studying social dynamics. This is the first crossdisciplinary practical use of such a model, according to the best of the knowledge of the author. Apart from using the original model, modifications are also suggested to utilise the spatial and neighbourhood information within the images to impose smoothness.
2 Paper Structure
The rest of the paper is organised as follows: the following Section 3 provides some background of concepts from statistical physics and introduces the DeffuantWeisbuch model, which is the central model of this paper. Modifications to the model are proposed in Section 4, to include spatial and neighbourhood information. Section 5 outlines the experiments performed and their results, and Section 6 concludes the paper and lists some possible future directions.
3 Statistical Physics and Social Dynamics
The development of the kinetic theory of gases, which sought to describe a system by focusing, not on a single particle, but on the system as a whole, gave rise to the field of modern statistical physics StatPhys . Since then, statistical physics has been applied to fields as diverse as medicine, computer science and economics among others StatPhysExample .
One important area of application of the modelling techniques from statistical physics is in the investigation of the dynamics in social phenomena sociophysics , where the end goal is to understand the largescale effects of collective interaction between agents. Each individual, in such a setting, is modelled as a simple entity having an opinion or a set of opinions, and interaction between agents leads to change in these opinions over time. This assumption, although rather simplistic, has been found to be quite robust in largescale settings socialatom . With this assumption, the aim is to study the steady opinion states of a population, and the processes that determine the interactions.
While there are many models which have been used to study opinionchange dynamics, the following subsection briefly summarises a model which has been used in this paper for the purpose of image segmentation. For a more indepth and comprehensive review of models from statistical physics and their use in social dynamics, the papers SocialDynamicsReview and DeffuantOverview may be consulted.
3.1 The DeffuantWeisbuch Model of Consensus Formation
The basic model developed in deffuant considers a population of agents with continuous opinions at time . Two randomly chosen agents meet at every time step and interaction happens if there opinions differ by less than a certain threshold , causing them to affect each other’s opinion. Thus, if two agents and have opinions and , and the difference in opinion , then the opinions for the next time step are adjusted according to:
(1)  
where, is the convergence parameter, which varies between 0 and 0.5.
The reason for the imposition of such a threshold condition is that, the agents only interact with each other if their opinions are ’close enough’. In that case, their opinions will symmetrically get closer to each other, the final result being one or more clusters, depending on the value of the confidence threshold (as shown by clustersconfidence , number of clusters is given by ).
To illustrate, Figure 1 shows the results of computer simulations of the evolution of opinions, all of which were initialised with agents with opinions distributed uniformly between 0 and 1.
The DeffuantWeisbuch model is subject to two explicit parameters, the confidence threshold and the convergence parameter . While mainly influences the convergence time of the model by changing the size of the update, is the main factor determining opinion convergence and stability. Studies, such as fortunato , show that there exists a critical value for the confidence threshold above which the agents’ opinions can reach consensus, and below which consensus is difficult. Researchers have also found that the critical value for the confidence threshold is related to the initial opinion distribution hirscher shang .
3.2 Why the DeffuantWeisbuch Model?
As mentioned earlier, there are many models in the social dynamics literature, out of which the DeffuantWeisbuch model was chosen for the image segmentation experiments. The rationale behind this is the fact that the models which were suggested before the DeffuantWeisbuch model, namely, the Ising Model ising , the Voter Model voter , the Majority Rule Model majority , the Sznajd Model sznajd , are too primitive and/or work on only discrete opinions, whereas a contemporary of this model, the HegselmannKrause Model hkmodel , has a rather long running time to be meaningful in this context.
4 Modifications to the DeffuantWeisbuch Model and Application to Image Segmentation
4.1 Application to Image Segmentation
An image in it’s basic form is a grid of pixels, each of which has a colour property which might be a vector or a scalar. Treating each pixel as an agent, such that it’s colour property is it’s opinion, a natural setting to apply the DeffuantWeisbuch model is arrived at. If the image is grayscale, then the opinion of the pixel agent is a single number and the model can be readily applied, whereas if the pixel value is a vector, i.e., RGB, HSV, YCbCr etc, then the update rule is applied to interacting pixel agents if:
(2) 
Upon convergence, the image will be expected to have a small number of colour centres to which the pixel values will have converged to, similar to another popular algorithm, the Kmeans
kmeans . The number of the final colour centres depends on the confidence threshold parameter .Having discussed about applying the DeffuantWeisbuch model in the scenario of image segmentation, the next two subsections focus on the following:

In Section 4.2, the model is modified to include the neighbourhood information around each pixel, which impose smoothness constraints.

In a typical clustering problem, the parameter to be chosen is the number of clusters . However, in the DeffuantWeisbuch model, the number of clusters is implicitly determined by the choice of the confidence threshold , which is difficult to select since it is only approximately related to the number of clusters (see Section 3.1). In Section 4.3, a simple solution to this problem is suggested.
4.2 Adding Neighbour Information to the DeffuantWeisbuch Model
The DeffuantWeisbuch model works with interactions among individual agents, and does not take into consideration any information about the spatial neighbourhood of the agent. Since, in an image, spatially adjacent pixels presumably have a stronger connection than pixels which are far away from each other, it is important to take into account the local commonality of location while updating the opinion of the pixel agents. In this paper, two different mechanisms of opinionupdate with locational information have been experimented with.
4.2.1 Using the Distance Between Interacting Agents
During the updating of the opinions in the DeffuantWeisbuch model, as described in Equation 1, two pixel agents and are chosen at random, and their opinions are updated if they satisfy Equation 2. Let the coordinates of the pixels in the image be and , and the coordinates of the topleft and bottomright corners of the image be and respectively. Then the distance between them is calculated as the normalised minkowski distance:
(3) 
where,
for , and with as the dimensionality.
4.2.2 Using the Neighbourhood of Interacting Agents
Instead of using the opinion of a pixel agent, during the opinion update, the average opinion of ’likeminded’ agents in the neighbourhood of that pixel agent in used.
The average neighbourhood opinion is given by:
(5) 
where, denotes the 4 or 8neighbourhood of including the point itself, and is an binary indicator variable which is 1 when the condition given in Equation 2 is met for and , and 0 otherwise. Then using the above equation in the update, the following is arrived at:
(6)  
4.3 Iterative Adjustment of the Confidence Threshold
In the DeffuantWeisbuch model, the number of opinion clusters after convergence is determined by the confidence threshold parameter . In the work of clustersconfidence , it was proved that , which is not an exact relationship between the number of clusters and the confidence threshold, thereby making an initial selection of difficult.
In order to tackle this problem, the process of clustering is started with a small initial value of , which is increased slowly with every run of the model, until the number of clusters at convergence matches a prespecified one.
4.4 Final Algorithms
5 Experiments
5.1 Dataset
In order to provide a quantitative evaluation of this method, experiments are conducted on a small subset of 25 images extracted from the Berkeley Image Segmentation Dataset Berkeley , which contain meaningful object and background entities. Apart from the natural images which are used in the segmentation process as input, the dataset also contains manual border annotation of the images, which are considered the ground truth.
5.2 Pre and PostProcessing
Pre and postprocessing is done in all the experiments. In order to reduce noise, the input images were subjected to bilateral filtering bilateral , which smoothes images but preserves edges. Also, the images after clustering are subjected to morphological smoothing operations to remove small isolated components.
5.3 Features
Since the emphasis of this paper is on the usefulness of the DeffuantWeisbuch model, complex highlevel pixel features have not been extracted to be used in the segmentation process. The raw RGB value of each pixel is used as the opinion vector for the clustering process.
5.4 Benchmarks
5.4.1 KMeans Clustering
The results of the DeffuantWeisbuch scheme are compared with those of the KMeans Clustering algorithm kmeans
. The Kmeans algorithm is chosen as a benchmark because it is a popular wellestablished machine learning algorithm
topten , and as it also clusters based on the distance between the points.5.4.2 Simple Linear Iterative Clustering
The Simple Linear Iterative Clustering algorithm (or SLIC in short) is a special case of the KMeans algorithm, specific to superpixel segmentation slic1 . SLIC clusters pixels based on their coordinates in a fivedimensional space, three of which are the colour coordinates in the CIELAB space, and two are the pixel position coordinates in the image. As shown in slic2 , SLIC can be considered quite stateoftheart.
5.5 Evaluation
The evaluation measures used in this paper are suggested in eval . First, the number of True Positive
(classified as object by both the algorithm and the ground truth), True Negative
(classified as nonobject by both the algorithm and the ground truth), False Positive (nonobject classified as object by both the algorithm and the ground truth) and False Negative (object classified as nonobject by both the algorithm and the ground truth) pixels were counted. Then, the recall, fallout and accuracy are calculated as:(7)  
5.6 Results
Experiments are run with , initial confidence threshold and threshold increment . The results are stated in the following table, where the algorithm name ’Deffuant’ denotes the standard DeffuantWeisbuch model (with the opinion updates given by Equation 1), ’DeffuantDistance’ refers to the modified model with distance information (using update Equation 4), and ’DeffuantNeighbour’ is the modified model with neighbourhood information (using update Equation 6).
Name  Recall  Fallout  Accuracy 

KMeans  73.49%  11.74%  85.98% 
SLIC  79.64%  9.11%  88.61% 
Deffuant  72.54%  3.03%  92.27% 
DeffuantDistance  76.78%  3.00%  92.55% 
DeffuantNeighbour  78.28%  2.89%  92.97% 
Bearing in mind that high values of accuracy and recall are better, while a low value of fallout is desired, it can be quickly observed that even the standard DeffuantWeisbuch model achieves respectable scores relative to both the KMeans and the SLIC algorithm. The modifications to the model bring in further improvements, especially to the recall values, which means that the model gets better at correctly identifying the object pixels.
6 Conclusions and Future Work
In this paper, a popular theoretical model from statistical physics, the DeffuantWeisbuch model, is used for unsupervised image segmentation, with two different modifications to include spatial information. Quantitative evaluation of the algorithm is done by using it to segment images from a small dataset into 2 clusters, and is compared to the results of the wellknown KMeans algorithm and the stateoftheart SLIC algorithm. Results suggest good performance, especially in the algorithm’s ability to correctly identify object pixels in this 2cluster setting.
Possible future work includes testing the performance of the DeffuantWeisbuch model and it’s modifications on images from different domains, such as hyperspectral and medical images, and more sophisticated use of neighbourhood information.
References
 (1) Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels. Tech. rep., Ecole Polytechnique Federale de Lausanne (EPFL), School of Computer and Communication Sciences (2010)
 (2) Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels compared to stateoftheart superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(11), 2274–2282 (2012)
 (3) Bowyer, K., Kranenburg, C., Dougherty, S.: Edge detector evaluation using empirical {ROC} curves. Computer Vision and Image Understanding 84(1), 77–103 (2001)
 (4) Buchanan, M.: The Social Atom: Why the Rich Get Richer, Cheats Get Caught, and Your Neighbor Usually Looks Like You. Marshall Cavendish (2007)
 (5) Carletti, T., Fanelli, D., Grolli, S., Guarino, A.: How to make an efficient propaganda. EPL (Europhysics Letters) 74, 222–228 (2006)
 (6) Castellano, C., Fortunato, S., Loreto, V.: Statistical physics of social dynamics. Reviews of Modern Physics 81, 591–646 (2009)
 (7) Chakrabarti, B.K., Chakraborti, A., Chatterjee, A.: Econophysics and Sociophysics. WileyVCH (2006)
 (8) Clifford, P., Sudbury, A.: A model for spatial conflict. Biometrika 60(3), 581–588 (1973)
 (9) Deffuant, G., Neau, D., Amblard, F., Weisbuch, G.: Mixing beliefs among interacting agents. Advances in Complex Systems 03(01n04), 87–98 (2000)
 (10) Flamm, D.: History and outlook of statistical physics. ArXiv Physics eprints (1998)
 (11) Fortunato, S.: Universality of the threshold for complete consensus for the opinion dynamics of deffuant et al. International Journal of Modern Physics C 15(9), 1301–1307 (2004)
 (12) Fu, K., Mui, J.: A survey on image segmentation. Pattern Recognition 13(1), 3–16 (1981)
 (13) Häggström, O., Hirscher, T.: Further results on consensus formation in the deffuant model (2013)
 (14) Hegselmann, R., Krause, U.: Opinion dynamics and bounded confidence, models, analysis and simulation. Journal of Artificial Societies and Social Simulation 5(3) (2002)
 (15) Hirscher, T.: Consensus formation in the Deffuant model. Department of Mathematical Sciences, Chalmers University of Technology, (2014)
 (16) Jain, A.K.: Data clustering: 50 years beyond kmeans. Pattern Recognition Letters 31(8), 651–666 (2010)
 (17) Liggett, T.M.: Interacting Particle Systems. Springer Berlin Heidelberg (1985)
 (18) Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proc. 8th International Conference on Computer Vision, vol. 2, pp. 416–423 (2001)
 (19) Pal, N.R., Pal, S.K.: A review on image segmentation techniques. Pattern Recognition 26(9), 1277–1294 (1993)
 (20) Shang, Y.: Deffuant model with general opinion distributions: First impression and critical confidence bound. Complexity 19(2), 38–49 (2013)
 (21) Stanley, H.: Exotic statistical physics: Applications to biology, medicine, and economics. Physica A: Statistical Mechanics and its Applications 285(1?2), 1–17 (2000)
 (22) SznajdWeron, K., Sznajd, J.: Opinion evolution in closed community (2000)
 (23) Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the Sixth International Conference on Computer Vision, pp. 839–846. IEEE Computer Society (1998)
 (24) Weidlich, W.: The statistical description of polarization phenomena in society. British Journal of Mathematical and Statistical Psychology 24(2), 251–266 (1971)
 (25) Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Yu, P.S., Zhou, Z.H., Steinbach, M., Hand, D.J., Steinberg, D.: Top 10 algorithms in data mining. Knowl. Inf. Syst. 14(1), 1–37 (2007)
Comments
There are no comments yet.