Unsupervised Image Segmentation using the Deffuant-Weisbuch Model from Social Dynamics

04/15/2016 ∙ by Subhradeep Kayal, et al. ∙ 0

Unsupervised image segmentation algorithms aim at identifying disjoint homogeneous regions in an image, and have been subject to considerable attention in the machine vision community. In this paper, a popular theoretical model with it's origins in statistical physics and social dynamics, known as the Deffuant-Weisbuch model, is applied to the image segmentation problem. The Deffuant-Weisbuch model has been found to be useful in modelling the evolution of a closed system of interacting agents characterised by their opinions or beliefs, leading to the formation of clusters of agents who share a similar opinion or belief at steady state. In the context of image segmentation, this paper considers a pixel as an agent and it's colour property as it's opinion, with opinion updates as per the Deffuant-Weisbuch model. Apart from applying the basic model to image segmentation, this paper incorporates adjacency and neighbourhood information in the model, which factors in the local similarity and smoothness properties of images. Convergence is reached when the number of unique pixel opinions, i.e., the number of colour centres, matches the pre-specified number of clusters. Experiments are performed on a set of images from the Berkeley Image Segmentation Dataset and the results are analysed both qualitatively and quantitatively, which indicate that this simple and intuitive method is promising for image segmentation. To the best of the knowledge of the author, this is the first work where a theoretical model from statistical physics and social dynamics has been successfully applied to image processing.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 5

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Image segmentation is the process of dividing an image into meaningful regions. For a segmentation technique to be useful for image analysis and interpretation, the separated regions should strongly relate to depicted objects or features of interest. Image segmentation is an important step in various image processing tasks as it transforms a low-level image to high-level image descriptions, in terms of features, objects, and scenes. A wide variety of methods exist in literature, with most of the segmentation algorithms belonging to one of the following broad categories: threshold based, edge-detection based, region based or based on clustering techniques. reviewpaper1 and reviewpaper2 provide excellent reviews of existing state-of-the-art image segmentation techniques and applications.

In this paper, a theoretical model from statistical physics, which aims at studying the properties of a population of interacting agents, is applied to the problem of image segmentation. The model described in this paper, and such other models, were originally motivated by the interaction of molecules in fluids, and have been used widely in studying social dynamics. This is the first cross-disciplinary practical use of such a model, according to the best of the knowledge of the author. Apart from using the original model, modifications are also suggested to utilise the spatial and neighbourhood information within the images to impose smoothness.

2 Paper Structure

The rest of the paper is organised as follows: the following Section 3 provides some background of concepts from statistical physics and introduces the Deffuant-Weisbuch model, which is the central model of this paper. Modifications to the model are proposed in Section 4, to include spatial and neighbourhood information. Section 5 outlines the experiments performed and their results, and Section 6 concludes the paper and lists some possible future directions.

3 Statistical Physics and Social Dynamics

The development of the kinetic theory of gases, which sought to describe a system by focusing, not on a single particle, but on the system as a whole, gave rise to the field of modern statistical physics StatPhys . Since then, statistical physics has been applied to fields as diverse as medicine, computer science and economics among others StatPhysExample .

One important area of application of the modelling techniques from statistical physics is in the investigation of the dynamics in social phenomena sociophysics , where the end goal is to understand the large-scale effects of collective interaction between agents. Each individual, in such a setting, is modelled as a simple entity having an opinion or a set of opinions, and interaction between agents leads to change in these opinions over time. This assumption, although rather simplistic, has been found to be quite robust in large-scale settings socialatom . With this assumption, the aim is to study the steady opinion states of a population, and the processes that determine the interactions.

While there are many models which have been used to study opinion-change dynamics, the following subsection briefly summarises a model which has been used in this paper for the purpose of image segmentation. For a more in-depth and comprehensive review of models from statistical physics and their use in social dynamics, the papers SocialDynamicsReview and DeffuantOverview may be consulted.

3.1 The Deffuant-Weisbuch Model of Consensus Formation

The basic model developed in deffuant considers a population of agents with continuous opinions at time . Two randomly chosen agents meet at every time step and interaction happens if there opinions differ by less than a certain threshold , causing them to affect each other’s opinion. Thus, if two agents and have opinions and , and the difference in opinion , then the opinions for the next time step are adjusted according to:

(1)

where, is the convergence parameter, which varies between 0 and 0.5.

The reason for the imposition of such a threshold condition is that, the agents only interact with each other if their opinions are ’close enough’. In that case, their opinions will symmetrically get closer to each other, the final result being one or more clusters, depending on the value of the confidence threshold (as shown by clustersconfidence , number of clusters is given by ).

To illustrate, Figure 1 shows the results of computer simulations of the evolution of opinions, all of which were initialised with agents with opinions distributed uniformly between 0 and 1.

Figure 1: Evolution of opinions for different values of the confidence threshold parameter.

The Deffuant-Weisbuch model is subject to two explicit parameters, the confidence threshold and the convergence parameter . While mainly influences the convergence time of the model by changing the size of the update, is the main factor determining opinion convergence and stability. Studies, such as fortunato , show that there exists a critical value for the confidence threshold above which the agents’ opinions can reach consensus, and below which consensus is difficult. Researchers have also found that the critical value for the confidence threshold is related to the initial opinion distribution hirscher shang .

3.2 Why the Deffuant-Weisbuch Model?

As mentioned earlier, there are many models in the social dynamics literature, out of which the Deffuant-Weisbuch model was chosen for the image segmentation experiments. The rationale behind this is the fact that the models which were suggested before the Deffuant-Weisbuch model, namely, the Ising Model ising , the Voter Model voter , the Majority Rule Model majority , the Sznajd Model sznajd , are too primitive and/or work on only discrete opinions, whereas a contemporary of this model, the Hegselmann-Krause Model hkmodel , has a rather long running time to be meaningful in this context.

4 Modifications to the Deffuant-Weisbuch Model and Application to Image Segmentation

4.1 Application to Image Segmentation

An image in it’s basic form is a grid of pixels, each of which has a colour property which might be a vector or a scalar. Treating each pixel as an agent, such that it’s colour property is it’s opinion, a natural setting to apply the Deffuant-Weisbuch model is arrived at. If the image is grayscale, then the opinion of the pixel agent is a single number and the model can be readily applied, whereas if the pixel value is a vector, i.e., RGB, HSV, YCbCr etc, then the update rule is applied to interacting pixel agents if:

(2)

Upon convergence, the image will be expected to have a small number of colour centres to which the pixel values will have converged to, similar to another popular algorithm, the K-means

kmeans . The number of the final colour centres depends on the confidence threshold parameter .

Having discussed about applying the Deffuant-Weisbuch model in the scenario of image segmentation, the next two subsections focus on the following:

  1. In Section 4.2, the model is modified to include the neighbourhood information around each pixel, which impose smoothness constraints.

  2. In a typical clustering problem, the parameter to be chosen is the number of clusters . However, in the Deffuant-Weisbuch model, the number of clusters is implicitly determined by the choice of the confidence threshold , which is difficult to select since it is only approximately related to the number of clusters (see Section 3.1). In Section 4.3, a simple solution to this problem is suggested.

4.2 Adding Neighbour Information to the Deffuant-Weisbuch Model

The Deffuant-Weisbuch model works with interactions among individual agents, and does not take into consideration any information about the spatial neighbourhood of the agent. Since, in an image, spatially adjacent pixels presumably have a stronger connection than pixels which are far away from each other, it is important to take into account the local commonality of location while updating the opinion of the pixel agents. In this paper, two different mechanisms of opinion-update with locational information have been experimented with.

4.2.1 Using the Distance Between Interacting Agents

During the updating of the opinions in the Deffuant-Weisbuch model, as described in Equation 1, two pixel agents and are chosen at random, and their opinions are updated if they satisfy Equation 2. Let the coordinates of the pixels in the image be and , and the coordinates of the top-left and bottom-right corners of the image be and respectively. Then the distance between them is calculated as the normalised minkowski distance:

(3)

where,

for , and with as the dimensionality.

Using Equations 1 and 3, the new update rule becomes:

(4)

4.2.2 Using the Neighbourhood of Interacting Agents

Instead of using the opinion of a pixel agent, during the opinion update, the average opinion of ’like-minded’ agents in the neighbourhood of that pixel agent in used.

The average neighbourhood opinion is given by:

(5)

where, denotes the 4 or 8-neighbourhood of including the point itself, and is an binary indicator variable which is 1 when the condition given in Equation 2 is met for and , and 0 otherwise. Then using the above equation in the update, the following is arrived at:

(6)

4.3 Iterative Adjustment of the Confidence Threshold

In the Deffuant-Weisbuch model, the number of opinion clusters after convergence is determined by the confidence threshold parameter . In the work of clustersconfidence , it was proved that , which is not an exact relationship between the number of clusters and the confidence threshold, thereby making an initial selection of difficult.

In order to tackle this problem, the process of clustering is started with a small initial value of , which is increased slowly with every run of the model, until the number of clusters at convergence matches a pre-specified one.

4.4 Final Algorithms

With sufficient background knowledge, the algorithm pseudo-codes are stated next.

  • Algorithm 1 outlines the steps required to implement the Deffuant-Weisbuch model.

  • Algorithm 2 builds on Algorithm 1 to adjust the value of confidence parameter to segment the image into a given number of clusters.

Data: Image, confidence threshold
Result: Image with colour centres
1 Let the image at time be denoted by , and the pixels at position and have values as and . Let and the convergence criterion ;
2 do
3       and , where update uses one of Equations 1, 4 or 6;
4       ;
5      
6while ;
Algorithm 1 The Deffuant-Weisbuch Model
Data: Image , the final number of clusters , initial confidence threshold and confidence threshold increment
Result: Image with colour centres
1 Let the image at iteration be denoted by , while the confidence parameter is denoted by . Let be a function which takes in an image and counts the number of cluster centres in the image. The starting conditions are and ;
2 do
3       ;
4       ;
5       ;
6      
7while ;
Algorithm 2 Iterative Updating of Confidence Threshold

5 Experiments

5.1 Dataset

In order to provide a quantitative evaluation of this method, experiments are conducted on a small subset of 25 images extracted from the Berkeley Image Segmentation Dataset Berkeley , which contain meaningful object and background entities. Apart from the natural images which are used in the segmentation process as input, the dataset also contains manual border annotation of the images, which are considered the ground truth.

5.2 Pre and Post-Processing

Pre and post-processing is done in all the experiments. In order to reduce noise, the input images were subjected to bilateral filtering bilateral , which smoothes images but preserves edges. Also, the images after clustering are subjected to morphological smoothing operations to remove small isolated components.

5.3 Features

Since the emphasis of this paper is on the usefulness of the Deffuant-Weisbuch model, complex high-level pixel features have not been extracted to be used in the segmentation process. The raw RGB value of each pixel is used as the opinion vector for the clustering process.

5.4 Benchmarks

5.4.1 K-Means Clustering

The results of the Deffuant-Weisbuch scheme are compared with those of the K-Means Clustering algorithm kmeans

. The K-means algorithm is chosen as a benchmark because it is a popular well-established machine learning algorithm

topten , and as it also clusters based on the distance between the points.

5.4.2 Simple Linear Iterative Clustering

The Simple Linear Iterative Clustering algorithm (or SLIC in short) is a special case of the K-Means algorithm, specific to superpixel segmentation slic1 . SLIC clusters pixels based on their coordinates in a five-dimensional space, three of which are the colour coordinates in the CIELAB space, and two are the pixel position coordinates in the image. As shown in slic2 , SLIC can be considered quite state-of-the-art.

(a)
(a) Original
(b)
(c)
(b) Ground Truth
(c)
(d)
(c) Deffuant-Neighbour
(d)
(e)
(d) SLIC
(e)
(f)
(a) Original
(b)
(c)
(b) Ground Truth
(c)
(d)
(c) Deffuant-Neighbour
(d)
(e)
(d) SLIC
(e)
Figure 2: Example segmentation results with c=2 : (a) Original Images, (b) Ground Truth Segmentation, (c) Segmentation with Deffuant-Weisbuch Model with Neighbourhood information, (d) Segmentation with the SLIC algorithm

5.5 Evaluation

The evaluation measures used in this paper are suggested in eval . First, the number of True Positive

(classified as object by both the algorithm and the ground truth), True Negative

(classified as non-object by both the algorithm and the ground truth), False Positive (non-object classified as object by both the algorithm and the ground truth) and False Negative (object classified as non-object by both the algorithm and the ground truth) pixels were counted. Then, the recall, fallout and accuracy are calculated as:

(7)

5.6 Results

Experiments are run with , initial confidence threshold and threshold increment . The results are stated in the following table, where the algorithm name ’Deffuant’ denotes the standard Deffuant-Weisbuch model (with the opinion updates given by Equation 1), ’Deffuant-Distance’ refers to the modified model with distance information (using update Equation 4), and ’Deffuant-Neighbour’ is the modified model with neighbourhood information (using update Equation 6).

Name Recall Fallout Accuracy
K-Means 73.49% 11.74% 85.98%
SLIC 79.64% 9.11% 88.61%
Deffuant 72.54% 3.03% 92.27%
Deffuant-Distance 76.78% 3.00% 92.55%
Deffuant-Neighbour 78.28% 2.89% 92.97%
Table 1: Segmentation Results

Bearing in mind that high values of accuracy and recall are better, while a low value of fallout is desired, it can be quickly observed that even the standard Deffuant-Weisbuch model achieves respectable scores relative to both the K-Means and the SLIC algorithm. The modifications to the model bring in further improvements, especially to the recall values, which means that the model gets better at correctly identifying the object pixels.

Apart from the results in Table 1, some qualitative results of segmentation are shown in Figure 2.

6 Conclusions and Future Work

In this paper, a popular theoretical model from statistical physics, the Deffuant-Weisbuch model, is used for unsupervised image segmentation, with two different modifications to include spatial information. Quantitative evaluation of the algorithm is done by using it to segment images from a small dataset into 2 clusters, and is compared to the results of the well-known K-Means algorithm and the state-of-the-art SLIC algorithm. Results suggest good performance, especially in the algorithm’s ability to correctly identify object pixels in this 2-cluster setting.

Possible future work includes testing the performance of the Deffuant-Weisbuch model and it’s modifications on images from different domains, such as hyper-spectral and medical images, and more sophisticated use of neighbourhood information.

References

  • (1) Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels. Tech. rep., Ecole Polytechnique Federale de Lausanne (EPFL), School of Computer and Communication Sciences (2010)
  • (2) Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(11), 2274–2282 (2012)
  • (3) Bowyer, K., Kranenburg, C., Dougherty, S.: Edge detector evaluation using empirical {ROC} curves. Computer Vision and Image Understanding 84(1), 77–103 (2001)
  • (4) Buchanan, M.: The Social Atom: Why the Rich Get Richer, Cheats Get Caught, and Your Neighbor Usually Looks Like You. Marshall Cavendish (2007)
  • (5) Carletti, T., Fanelli, D., Grolli, S., Guarino, A.: How to make an efficient propaganda. EPL (Europhysics Letters) 74, 222–228 (2006)
  • (6) Castellano, C., Fortunato, S., Loreto, V.: Statistical physics of social dynamics. Reviews of Modern Physics 81, 591–646 (2009)
  • (7) Chakrabarti, B.K., Chakraborti, A., Chatterjee, A.: Econophysics and Sociophysics. Wiley-VCH (2006)
  • (8) Clifford, P., Sudbury, A.: A model for spatial conflict. Biometrika 60(3), 581–588 (1973)
  • (9) Deffuant, G., Neau, D., Amblard, F., Weisbuch, G.: Mixing beliefs among interacting agents. Advances in Complex Systems 03(01n04), 87–98 (2000)
  • (10) Flamm, D.: History and outlook of statistical physics. ArXiv Physics e-prints (1998)
  • (11) Fortunato, S.: Universality of the threshold for complete consensus for the opinion dynamics of deffuant et al. International Journal of Modern Physics C 15(9), 1301–1307 (2004)
  • (12) Fu, K., Mui, J.: A survey on image segmentation. Pattern Recognition 13(1), 3–16 (1981)
  • (13) Häggström, O., Hirscher, T.: Further results on consensus formation in the deffuant model (2013)
  • (14) Hegselmann, R., Krause, U.: Opinion dynamics and bounded confidence, models, analysis and simulation. Journal of Artificial Societies and Social Simulation 5(3) (2002)
  • (15) Hirscher, T.: Consensus formation in the Deffuant model. Department of Mathematical Sciences, Chalmers University of Technology, (2014)
  • (16) Jain, A.K.: Data clustering: 50 years beyond k-means. Pattern Recognition Letters 31(8), 651–666 (2010)
  • (17) Liggett, T.M.: Interacting Particle Systems. Springer Berlin Heidelberg (1985)
  • (18) Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proc. 8th International Conference on Computer Vision, vol. 2, pp. 416–423 (2001)
  • (19) Pal, N.R., Pal, S.K.: A review on image segmentation techniques. Pattern Recognition 26(9), 1277–1294 (1993)
  • (20) Shang, Y.: Deffuant model with general opinion distributions: First impression and critical confidence bound. Complexity 19(2), 38–49 (2013)
  • (21) Stanley, H.: Exotic statistical physics: Applications to biology, medicine, and economics. Physica A: Statistical Mechanics and its Applications 285(1?2), 1–17 (2000)
  • (22) Sznajd-Weron, K., Sznajd, J.: Opinion evolution in closed community (2000)
  • (23) Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the Sixth International Conference on Computer Vision, pp. 839–846. IEEE Computer Society (1998)
  • (24) Weidlich, W.: The statistical description of polarization phenomena in society. British Journal of Mathematical and Statistical Psychology 24(2), 251–266 (1971)
  • (25) Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Yu, P.S., Zhou, Z.H., Steinbach, M., Hand, D.J., Steinberg, D.: Top 10 algorithms in data mining. Knowl. Inf. Syst. 14(1), 1–37 (2007)