1 The general problem
A basic question that we wish to discuss in this work is whether machine learning and data mining tools may be applied to the analysis of material properties. Specifically, we will review initial efforts to detect, via statistical mechanics and the tools of information science and network analysis, pertinent structures on all scales in general complex systems. We will describe mapping atomic and other configurations onto graphs. As we will explain, patterns found in these graphs via statistical physics methods may inform us about the structure of the investigated materials. These structures can appear on multiple spatial and temporal scales. In comparison to standard procedures, the advantage of such an approach may be significant.
There are numerous classes of complex systems. One prototypical variety is that of glass forming liquids. “Glasses” have been analyzed with disparate tools angell ; fred' ; nakamura ; saida ; sordelet ; wang ; mcgreevy ; keen ; Sheng ; ref:finney ; HA ; BO ; rfot ; ref:lubchenkowolynes ; gilles ; nab . Although they have been known for millennia, structural glasses still remain ill understood. It is just over eighty years since the publication of one of the most famous papers concerning the structure of glasses fred' . Much has been learned since the early days of hand-built plastic models and drawings, yet basic questions persist.
Amorphous systems such as glasses strongly contrast with idealized simple solids. In simple crystals, the structure of an atomic unit cell is replicated to span the entire system. Long before scattering and tunneling technologies, prominent figures such as Robert Hooke, Christiaan Huygens, and their contemporaries in the 17th century proposed the existence of sharp facets in single crystals results from recurrent fundamental unit cell configurations. The many years since have seen numerous breakthroughs (including the advent of quantum mechanics and atomic physics) and witnessed a remarkable understanding as to how the quintessential simple periodic structure of crystals accounts for many of their properties. However, while simple solids form a fundamental pillar of current technology (e.g., the transistor whose invention was made possible by an understanding of the electronic properties of nicely ordered periodic crystals and chemical substitution therein), there are many other complex systems whose understanding is extremely important yet still lacking. The discovery of salient features of these materials across all scales is important for both applied and basic science. The recognized significance of this problem engendered the Materials Genome Initiative wh —a broad effort to develop infrastructure for accelerating materials innovation.
This work discusses a path towards solving this problem in complex amorphous materials. The framework that we will principally suggest is that of multi scale community detection. This approach does not invoke assumptions as to which system properties are important and construct resulting minimal toy models based on the assumptions. The insightful guess-work that is typically required to describe complex materials is, in the work that we review, replaced by a computerized variant of the wisdom of the crowds phenomena wisdom . The key concepts underlying this approach may be applied to general hard problems beyond those concerning the structure of materials or even general data mining. In the next section, we review an “Information theoretic ensemble minimization” method that may be suited for such tasks.
2 Ensemble minimization
Before delving into complex material and network analysis, we first discuss a general strategy for solving hard problems. The concept underlying this approach is perhaps best conveyed by a simple cartoon such as that sketched in Figure 1(a). In this illustration, each sphere corresponds to an individual solver (or “replica”) that explores an energy landscape. On its own, each such sphere might get stuck in a local energy minimum. The collective ensemble of solvers may, however, thwart such situations more readily as compared to the same single solver algorithm grant34 . In Figure 1(b), the individual solvers not only roam the energy landscape but also interact amongst themselves as schematically denoted by springs. If a single solver gets stuck in a false minimum, the other solvers may “pull it out” and explore broader regions of the energy landscape.
This collective evolution of individual solvers is quite natural and has appeared in different guises across many fields. It anthropological contexts, this basic principle is known as “wisdom of the crowds” wisdom . That is, the crowd or ensemble of individuals might do far better than a single solver. Unlike ensemble related approaches such as swarm intelligence ant or genetic genetic algorithms, relevant problems in our context do not focus exclusively on minimizing a given energy function. Rather, we will try to maximize information theory correlations [the effect of the springs in Figure 1(b)] while simultaneously minimizing a cost function ref:rzmultires . If all (or many) solvers agree on a particular candidate solution then that solution may naturally arise in many instances and may be of the high importance regardless of whether or not it is the absolute minimum of the energy. In the physical problems that we will consider—that of finding natural structures in materials—these considerations are pertinent.
The above discussion is admittedly abstract and may, in principle, pertain to any general problem. We next briefly explain the basic mathematical framework—the community detection problem—in which we will later couch the material structure detection endeavor.
3 Community detection and data mining
Community detection pertains to the quest of partitioning a given graph or network into its optimally decoupled subgraphs (or so-called communities), e.g.,fortunato2 ; pnasn ; newman_girvan ; fortunato3 ; fortunato1 ; blondel ; newman_fast ; gudkov ; RosB ; book_comm ; pre3 ; spec ; spec' ; darst' . As the reader may anticipate, given the omnipresence of networks and the generality of this task, this problem appears in disparate arenas including biological systems, computer science, homeland security, and countless others. In what follows, we introduce some of the key elements of community detection. The graphs of interest will be composed of nodes where a node is a fundamental element of an abstracted graph. An edge in the graph is a defined relationship between two nodes. Edges may be weighted or unweighted where the unweighted case is the one most commonly examined. In our applications, we will need to assign weights to the edges in the graph as we will describe. Similarly, in general applications, edges may be either symmetric or directed.
Now we come to a basic ingredient of community detection. A community corresponds to a subset of nodes that are more cohesively linked (or densely connected for unweighted edges) within their own community than they are to other communities. The above definition might seem a bit loose. Indeed, there are numerous formulations of community detection in the literature. As intuitively one may expect, most of these do, more or less, the same thing. When clear community detection solutions exist, all algorithms quantify the structure of large complex networks in terms of the smaller number of the natural cohesive components. Rather general data structures may be cast in terms of abstract networks. Thus, the community detection problem and other network analysis methods can have direct implications across multiple fields. Indeed, we will elaborate how this occurs for image segmentation and material analysis.
In what follows we will briefly review the rudiments of an “Absolute Potts Model” method for community detection ref:rzlocal that avoids a “resolution limit” that an insightful earlier Potts model RB exhibited. To cast things generally, we make a simple observation underlying the “Potts” characterization. Any partition of the numbered nodes into different communities (the ultimate objective of any community detection algorithm) is an assignment where the integer denotes the community number to which node belongs. With a characterization in hand, we next construct an energy functional.
To illustrate the basic premise, we first consider an unweighted graph—one in which the link strength between the two nodes and is if an edge is present between the two nodes and if there is no link. As Figure 2 demonstrates, for each pair of nodes there are four principal cases to consider. That is, either (i) the two nodes belong to the same community and have an “attraction” between them (i.e., ), (ii) two nodes in the same community can have a missing link between them (), (iii) the two nodes may belong to different communities yet nevertheless exhibit cohesion between themselves (), or (iv) nodes and may belong to different communities and have no edge connecting them (). Situations (i) and (iv) agree with the intuitive expectation that nodes in the same community should be connected to one another while those in different communities ought to be disjoint. We may take these four possibilities as the foundation of an energy function. That is, any given pair of nodes may be examined to see which of these categories it belongs to. Thus, a contending cost function is given by the Potts model Hamiltonian
In Eq. (1), is a Kroncker delta (i.e., , ) and is a “resolution parameter” that will play a notable role in our analysis. Before turning to the origin of the name of this parameter, we observe that, subtracting an innocuous additive constant, Eq. (1) is trivially
As Eq. (2) makes clear, by virtue of the Kronecker delta , the sum is local—i.e., the sum only includes intra-community node pairs. The Hamiltonian of Eq. (2) may be minimized by a host of methods. In practice, when the solution of the problem is easy to find, nearly all viable approaches will yield the same answer. Amongst many others, two approaches are afforded by spectral methods [in which the discrete Potts model spins are effectively replaced by continuous spherical model (or large ) spins] and a conceptually more primitive steepest descent type approach.
A simple incarnation of the relatively successful greedy algorithm ref:rzlocal ; ref:rzmultires that extends certain ideas introduced in blondel is given by the following steps: (a) Initially, each node forms its own community [i.e., if there are (numbered) nodes then there will be communities].
(b) A node (whose number is ) is chosen stochastically and then another edge sharing node is picked at random. (c) If it is energetically profitable to move the node together into the group formed by then this is done (otherwise community assignments are unchanged). (d) Yet another node is next chosen and once again it is asked whether moving yet another node into the community of lowers the energy. As earlier mentioned, if this change lowers the energy of Eq. (2), the nodes will be merged. Otherwise no change will be made. (e) In this manner, we cycle through each of the nodes and repeat as necessary. (f) The process stops and a candidate partition is found once all further possible mergers do not lower the energy further. As the reader can appreciate, such a simple simple algorithm lowers the energy until the system becomes trapped in a local minimum. To improve the accuracy (i.e., further lower the energy of candidate solutions), one may repeat the above steps a finite number of times for a finite number of trials—i.e., repeat the above when vertices are chosen in a different random order to see if a lower energy solution may result.
For the wide range of examined problems, the number of trials for each replica of the system is typically on the order of ten or smaller. When approaching the “hard phase” (to be discussed in Section 6) with multiple false minima, an increase in the number of trials may likely further increase the accuracy (this rise in the accuracy was termed the “computational susceptibility” in ref:rzmultires ; phases ). Typically, elsewhere the improvement in the precision due to a further increase in the number of trials is nearly nonexistent (see, e.g., Figure 13 in ref:rzmultires ). Further embellishments of the bare algorithm outlined above, include the acceptance of zero energy moves and other refinements ref:rzlocal . Other illuminating greedy type approaches for the inference of community structure have been advanced, e.g., greed .
4 Multi-scale community detection
We now turn to “multi-scale” community detection, e.g., ref:rzmultires ; KSKK ; local-multi ; jeub ; manilo ; pt ; eytan . In certain notable approaches, e.g., eytan , detection of scale is performed without the resolution parameter but rather by examining the effects of thermal fluctuations in a pure ferromagnetic system (one sans the antiferromagnetic interaction present in the second term of Eq. (2)), and other considerations elsewhere. In what follows, we will build on the ideas introduced in Section 3 that lead to an accurate determination of structure on diverse pertinent scales. To understand the physical content of the resolution parameter (and the origin of its name) in Eq. (2), we consider several trivial limits. First, we focus on the case of . In such a situation, the energy of Eq. (2) is minimized when all nodes belong to a single community. This is the lowest energy solution since each intra-community link lowers the energy [the first term of Eq. (2)], but there is no energy penalty from any missing links between nodes in the same community since the second term in Eq. (2) is trivially zero. Thus, in order to maximize the number of internal links it is profitable to assign all nodes to the same community. In the diametrically opposite limit—that of , the energy penalty diverges unless every pair of nodes belonging to the same community share a link. Thus, in this limit, the lowest energy states are those in which the the system fragments into (typically) a large number of communities where each node is connected to all other nodes in its community. That is, the communities are “perfect cliques.” As is monotonically increased from zero, the ground states of Eq. (2) lead to communities that veer from the extreme global case () to the limit of many disparate densely internally connected local communities (). Putting all of the pieces together, the reader can see why is inherently related to the intra-community edge density and thus is indeed a “resolution parameter”.
At this stage, it is not yet clear which values should be assigned in order to lead to the most physically pertinent solutions. The non-uniqueness of is, actually, a virtue of the Potts model based approach of Eq. (2). That is, in general, there may be several relevant resolution scales that lead to different insightful candidate low energy partitions of this Hamiltonian. This is the situation which is schematically depicted in Figure 3 for a synthetic system that exhibits a hierarchical structure. In such cases as is increased, the minima of Eq. (2) unveil different resolutions in the hierarchy. In practice, the multi-resolution community-detection method ref:rzmultires systematically infers the pertinent scale(s) by information-theory-based correlations info11 ; info2 ; info3 ; ginestra between different independent solvers (or “replicas”, as discussed in Section 2) of the same community detection problem. In most studied systems, the number of replicas used is . As alluded to in Section 3, the lowest energy solution amongst a fixed number of trials is taken for each of the individual replicas. If these solvers (i.e., the replicas) strongly concur with each other about local or global features of the solution ref:rzmultires , then these aspects are likely to be correct. Such an agreement between solvers is manifest in the information correlations. Information theory extrema grant3 ; grant7 ; grant6 then provide all relevant system scales.
Figure 4 shows the results of our analysis as the resolution parameter is varied for the synthetic system of Figure 3. Plotted are three information theory correlations between replicas—the average inter-replica variation of information (VI), the mutual information (I), the normalized mutual information (NMI), the total number of communities () found for different values of , and the Shannon entropy () averaged over different replicas. Transitions between viable solutions are evident as jumps in the number of communities and, most notably, as transitions between crisp information theory measure plateaux. As shown, each of the plateaux in Figure 4 corresponds to a different level of the hierarchy of the synthetic network in Figure 3. Similar to our discussion in Section 3, in practice the replicas differ from one another in the order in which consecutive vertices are picked and moved so as to minimize the energy of Eq. (2). Thus, for any given problem has an ensemble of very similar (or nearly identical) viable solutions associated with it. A detailed summary of this approach appears in ref:rzmultires .
In accord with the above explanation, as is increased, the associated candidate energy minima partition the system into more local, smaller communities (deeper levels of the hierarchy). The inter-replica information theory correlations further afford a measure of the quality of the viable partitions. High NMI values (i.e., of size close to unity) indicate solutions that are likely to be pertinent. In the spirit of Section 2, if the different replicas all agree with one another on a putative partition, then that partition is likely to be physically meaningful. The variation of information measures the disparity between candidate solutions; thus the VI values are high between different NMI plateaux and are low within the NMI plateaux.
5 Image segmentation
Our goal is to identify structure in materials, but before turning to this endeavor, we first illustrate how patterns may, literally, be revealed by community detection. The ideas underlying this objective will elucidate our approach to material genomics. The aim of image segmentation grant6 ; book-image ; Shi ; ling ; abin ; grant21 ; grant21.5 is to divide a given digital image into separate objects (or segments) based on visual characteristics. Two somewhat challenging examples are provided in Figure (5) chal1 ; chal2 .
To transform the problem into that of community detection, we map a digital image into a network as follows. Each pixel in an image is regarded as a node in a graph. (2) The edge weights between nodes in the graph are determined by the degree of similarity between the additive color RGB (i.e., the Red, Green, and Blue) strength of individual pixels or, more generally, of finite size boxes geometrically centered about a given pixel. The bare edge strengths may be embellished and replaced by weights set by the Fourier weights associated with finite size blocks about a given node. Alternatively, we can use exponential weighting of the inter-node edge strength based on the geometric distance between them (the distance between the centers of the finite size blocks about them) grant6 . The edge value assignment is such that if two pixels and (or boxes centered about them) have similar RGB values (or absolute Fourier magnitudes), then a function set by these differences will be small. Analogously, if nodes and (or boxes centered around them) are dissimilar then will become large.
With such functions at hand, a simple generalization of Eq. (2) is given by
Here, is the Heavyside function ( and ) and is an adjustable background value. As the astute reader undoubtedly noticed, the locality constraint imposed by the Kronecker delta in Eq. (2) has been made explicit in Eq. (3) by having only intra-community sums for each of the communities . Details of the construction of the weights are provided in grant6 . Following our more colloquial description here, there are four or five adjustable parameters in Eq. (3): the resolution parameter , the background value , the block size centered about each pixel (or more general rectangular blocks of size ), and the pixel distance over which the pixel interconnection function decays. Once these are set, the earlier community detection algorithm of Section 3 may be applied. The determination of the optimal value(s) of these parameters may be performed using the same procedure outlined in Section 4.
While systems such as the synthetic hierarchical network of Figures 4 exhibit well defined plateaux in the information theory and other measures, we found more generally that the optimal values of parameters correspond to local extrema whereby variations in the parameters do not alter the outcome. That is, if is a measured quantity of interest (e.g., information theory correlations, Shannon entropy, the energy associated with the given Hamiltonian) then optimal parameters are found by the requirement that . These may lead to multiple viable solutions corresponding to very different meaningful partitions.
In practice, we found that in all but the hardest cases, meaningful solutions are found when arbitrarily setting all parameters to a fixed value and that, similar to Section 4, the multi-scale solutions may be found by only varying the resolution parameter . The results of our method are given in Figure (6); these correspond to typical partitions found with the optimal parameter set. The above image analysis ideas may be applied for the detection of the primitive cells in simple Bravais lattices, the inference of domain walls in spin systems, and hierarchical structures in quasi-crystals grant6 . For a complete classification of contending partitions and, most notably, a deeper understanding of whether the found solutions are meaningful or not, it is useful to survey the canonical finite temperature phase diagram associated with Eq. (3) when all of the above parameters, including temperature, are varied. In the current context, by “temperature”, we allude to the finite temperature study of the Hamiltonian of Eq. (2) either analytically or via a thermal bath associated with, e.g., the acceptance of the moves in the algorithm outlined at the end of Section 3 grant3 ; grant6 ; phases ; phases1 ; phases2 .
6 Community detection phase diagram
As the bare edge weights and additional parameters setting the values of in the Hamiltonian of Eq. (3) and temperature are modified, quantities such as the system energy, Shannon entropy, the number of communities, and information theory correlations amongst the found ground states generally attest to the presence of multiple phases. Additional metrics including the “computational susceptibility” (the change in the average inter-replica NMI as the number of trials, see Section 3, is increased ref:rzmultires ; phases ; phases1 ), the time required for convergence (when attainable), and the ergodic/non-ergodic character (“chaotic” type feature) of the dynamics all delineate the very same phase diagram boundaries inferred from each of the examined quantities. Information theory measures have been used to study other specific interesting systems, e.g., information . The observed phases in the community detection problem naturally extend to finite temperatures () when the analysis of the system defined by the Hamiltonian of Eq. (3
) is broadened to include positive temperatures. Finite size systems such as the real networks and images that we discuss cannot exhibit thermodynamic phase transitions and all finite temperature functions are analytic. Nevertheless, practically, sharp changes appear as temperature and other parameters are varied.
Similar to other NP hard cook combinatorial optimization problems hard ; mona ; gangof3 , three prototypical phases were established in general community detection problems with a distribution of varying community sizes phases . Subsequently, these have been beautifully explored in depth in several specific graph types—most notably the so-called “stochastic block models”, in which a graph has equal size communities e.g., decelle ; sly ; NN ; darst" and in other penetrating works, e.g., steeg ; monta1 ; zhang" . Earlier signatures of a bona fide transition in stochastic block and power law distributed models ref:rzlocal ; ref:rzmultires and limits on detectability in the stochastic block model via the cavity approximation were suggested ref:ML . To intuitively highlight the essential character of the prototypical phases with a minimum of jargon, we will colloquially term these the “easily solvable”, the “solvable hard”, and the “unsolvable” phases.
In realistic finite yet very large scale systems phases1 ; phases2 various results can be established and these may be further examined in various limits. Of course, bona fide transitions formally occur only in the thermodynamic limit. A trivial behavior results in infinite size graphs when the average number of nodes per community is of finite size phases1 ; phases2 . As one would expect, typically all community detection problems are either solvable or unsolvable. In NP hard problems, the solvable phase splinters into an “easy” and a “hard phase”. When the edge weights set by are associated with sharp community detection partitions, then finding a natural solution is rather trivial (and nearly all algorithms, not only the Potts model described here, will readily unearth such an answer). On the other hand, if the couplings are sufficiently “noisy” so as to be of, effectively, equally the same strength for edges between nodes in the same putative community as for edges linking nodes belonging to different supposed communities, then no well defined community detection solutions exist. Similarly, at sufficiently high temperatures, in most cases, all traces of structures found in the ground state(s) are lost. The most common variant of the community detection problem has been proven to be NP complete book_comm .
As in disparate NP problems gangof3 , it was found that in broad classes of the community deception problem (and in its image segmentation variant) grant6 ; phases ; phases1 ; phases2 ; decelle ; NN ; steeg ; zhang" , lying between the extremities of the “easy” and “unsolvable” phases there often exists a “hard phase”; in this phase, solutions exist, but due to the plethora of competing states, they may be extremely hard to find. Information theory measures may be used to delineate phase boundaries grant6 ; phases ; phases1 ; phases2 . Using information theory correlations and the global Shannon entropy, we show, in Figure (7) and Figure (8) respectively, the phase diagram associated with the image shown in the upper lefthand side of Figure (9). In the solvable phase(s), typically, all partitions produced by parameters that lie in the same basin, lead to qualitatively similar results. Moderate temperature and/or disorder can lead to order by disorder or annealing effects (similar to those found in other systems, e.g., kirk ; villain1980 ; Henley89 ; Nussinov04 ; funnel ). However, at sufficiently high temperatures and/or the introduction of noise about the initial values, the system will be in the unsolvable phase. By carefully studying the system phase diagram and the character and magnitude of the information theory overlaps or thermodynamic functions such as the internal energy and entropy as well as the dynamics, one may assess whether the perceived community detection solutions may be meaningful. When applied to image segmentation, the consistency of this procedure may be inspected visually and intuitively judged sans complicated analysis.
7 Casting complex materials and physical systems as networks
With all of the above preliminaries, we now finally turn to the ultimate data mining objective of this work: that of the important detection of spatial and temporal structure in complex materials and other systems grant3 ; grant7 ; BODP ; spatial ; holme ; emergence ; temporal ; jack . This problem shares a common conceptual goal with image segmentation yet is, generally, far more daunting for human examination. Similar to the analysis presented thus far, the approach that we wish to discuss casts physical systems as graphs in space or space-time and then employs the above discussed multi-scale community detection to determine meaningful partitions.
In this case, nodes in the graph code basic physical units of interest (e.g., atoms, electrons, etc.). Multi-particle interactions or experimentally measured correlations in the physical system are then ascribed to edge weights between the nodes (for two-particle interactions or experimentally measured pair correlations grant3 ; grant7 ), or to three-node triangular weights (for three-particle interactions or correlations) , and so on. Given these static or time-dependent weights, the graph is then (similar to the discussion in earlier sections) partitioned into “communities” of nodes (e.g., clusters of atoms) that are more tightly linked to or correlated with each other than with nodes in other clusters ref:rzlocal . As in the earlier examples explored in this work, information theory based multi-scale community detection provides both local structural scales (e.g., primitive lattice cell, nearest neighbor distance, etc.) as well as global scales (such as correlation lengths) and any other additional intermediate scales if and when these are present.
The results of this approach for a two-dimensional Lennard-Jones system with vacancies are shown in Figure 10. When the edge weights between nodes are set equal to the Lennard-Jones strength associated with the distance between them, the multi-scale community detection algorithm recognizes both the typical triangular unit cells as well as larger scale domains (communities) in which the vacancy defects tend, on average, to lie on their boundaries. Partitions in which defects tend to aggregate at the domain boundaries is consistent with general expectations for stable domains and is intuitively appealing.
As the reader may envisage, the community detection method may be extended to general many-body systems with different types of species (e.g., disparate ion types in metallic glass formers grant3 ; grant7 ). One example is depicted in Figures 11, 12, and 13 corresponding to a ternary system of AlYFebased on a molecular dynamics simulation of atoms in which edge weights were set by pair potentials is provided in Figs. (12, 13). As seen in the partition of Figure 13, for which the inter-replica information theory were extremal and which lies in the solvable phase, below the liquidus temperature (the temperature at which the system is an equilibrium liquid), large clusters were detected. Along similar lines, clusters may be identified across many problems. In Figure (16) we show typical clusters found in a Kob-Andersen binary system. While for human analysis the complexity of potentially identifying pertinent clusters may grow dramatically with the number of atom types, for the mutli-resolution analysis there is no such increase.
In a similar manner, the edge weights can be set by experimentally measured pair correlations. In grant3 , atomic configurations consistent with the experimentally determined scattering data for quenched ZrPt nakamura ; saida ; sordelet ; wang were generated grant3 ; grant7 using Reverse Monte Carlo methods mcgreevy ; keen .
At low temperatures, typically the found structures in all of these cases are far larger than local patterns probed for and detected by current methods ps ; ref:mosayebiCLSGT ; ref:berthierCL ; ref:karmakarsastry ; kl . Four-point correlations have long been employed to ascertain spatio-temporal scales and the quantify “dynamical heterogeneities”, e.g., ref:karmakarsastry ; 4p . A long-standing challenge is the identification of structures of general character and scale in amorphous systems. There is, in fact, a proof that as supercooled liquid falls out of equilibrium to become an amorphous, there must be an accompanying divergent length scale ref:montanariCL . Methods of characterizing local structures Sheng ; ref:finney ; HA ; BO center on a given atom or link; as such, they are restricted from detecting general structures. Because of the lack of a simple crystalline reference, the structure of glasses is notoriously difficult to quantify beyond the very local scales. In Refs. grant3 ; grant7 ; grant6 , graph weights were determined empirically (potentials in a model system, experimentally measured partial pair density correlations in supercooled fluids, or pixels in a given image)—no theoretical input was invoked as to what the important scales should be or if an exotic order parameter may be concocted. Similarly, in a time dependent analysis for dynamically evolving systems, by employing replicas at different time slices as well as regarding the system as a higher dimensional “image” in space-time, using the inter-replica information theory correlations, spatio-temporal patterns were found and time dependent structures were quantified. In this approach, the data speak for themselves. We remark that notwithstanding the aforementioned difficulties, recently extremely large growth of static structure was observed by far simpler network analysis in certain binary metallic glasses that exhibit crisp icosahedral motifs grant22 . Similar to the description above, one may likely find other motifs in other systems. The problem is that guessing and hopefully finding pertinent patterns can be extremely challenging to do by conventional analysis.
In this work, we reviewed key features of a statistical-mechanics-based “community detection” approach to find pertinent features and structures (both spatial and temporal) in complex systems. In particular, we illustrated how this method may be applied to image segmentation and the analysis of amorphous materials. The demand for automated data mining approaches may become more acute with ever increasingly available data on numerous complex systems. The study of complex materials may be extremely challenging to carry out by current conventional means that rely on guessed patterns, simplified models, or brute force human examination.
We have benefited from interactions with numerous colleagues. In particular, we would like to thank S. Achilefu, S. Bloch, R. Darst, S. Fortunato, V. Gudkov, K. F. Kelton, T. Lookman, M. E. J. Newman, S. Nussinov, D. R. Reichman, and P. Sarder for numerous discussions and collaboration on some of the problems reviewed in this work and their outgrowths. We are further grateful to support by the NSF under Grants No. DMR-1106293 and DMR-1411229. ZN is indebted to the hospitality and support of the Feinberg foundation for visiting faculty program at the Weizmann Institute.
- (1) C. A. Angell, “Formation of Glasses from Liquids and Biopolymers”, Science 267 (5206), 1924-1935 (1995).
- (2) W. H. Zachariasen, “The Atomic Arrangement in Glass”, J. Am. Chem. Soc. 54, 3841 (1932).
- (3) T. Nakamura, E. Matsubara, M. Sakurai, M. Kasai, A. Inoue, and Y. Waseda, ”Structural study in amorphous Zr-noble metal (Pd, Pt and Au) alloys”, J. Non-Cryst. Solids 312-314, 517 (2002).
- (4) Junji Saida, Keiji Itoh, Shigeo Sato, Muneyuki Imafuku, Takashi Sanada, and Akihisa Inoue, “Evaluation of the local environment for nanoscale quasicrystal formation in ZrPt glassy alloy using Voronoi analysis”, J. Phys.: Condens. Matter 21, 375104 (2009).
- (5) D. J. Sordelet, R. T. Ott, M. Z. Li, S. Y. Wang, C. Z. Want, M. F. Besser, A. C. Y. Liu, and M. J. Kramer, “Structure of Zr Pt () Metallic Glasses”, Metallurgical and Materials Transactions A 39A, 1908-1916 (2008).
- (6) S. Y. Wang, C. Z. Wang, M. Z. Li, L. Huang, R. T. Ott, M. J. Kramer, D. J. Sordelet, and K. M. Ho, “Short- and medium-range order in a ZrPt glass: Experimental and simulation studies”, Phys. Rev. B 78, 184204 (2008).
- (7) R. L. McGreevy, “Understanding liquid structures”, J. Phys.: Condens. Matter 3 F9 (1991).
- (8) D. A. Keen and R. L. McGreevy, “Structural modelling of glasses using reverse Monte Carlo simulation”, Nature 344, 423-5 (1990).
- (9) H. W. Sheng, W. K. Luo, F. M. Alamgir, J. M. Bai, and E. Ma, “Atomic packing and short-to-medium-range order in metallic glasses”, Nature 439, 419-425 (2006).
- (10) J. L. Finney, “Random packings and the structure of simple liquids. I. The Geometry of Random Close packing”, Proc. R. Soc. London, Ser. A 319, 1539, 479-493 (1970).
- (11) J. Dana Honeycutt and Hans C. Andersen, “Molecular dynamics study of melting and freezing of small Lennard-Jones clusters”, J. Phys. Chem. 91, 4950-4963 (1987).
- (12) P. J. Steinhardt, D. R. Nelson and M. Ronchetti, “Bond-orientational order in liquids and glasses”, Phys. Rev. B 28, 784-805 (1983).
- (13) T.R. Kirkpatrick, D. Thirumalai, and P. G. Wolynes, “Scaling concepts for the dynamics of viscous liquids near an ideal glassy state”, Physical Review A 40, 1045-1054 (1989).
- (14) V. Lubchenko and P. G. Wolynes, “Theory of Structural Glasses and Supercooled Liquids”, Annu. Rev. Phys. Chem. 58, 235-266 (2007).
- (15) G. Tarjus, S. A. Kivelson, Z. Nussinov, and P. Viot, “The frustration-based approach of supercooled liquids and the glass transition: a review and critical assessment”, Journal of Physics: Condensed Matter 17, R1143-R1182 (2005).
- (16) Z. Nussinov, “Avoided phase transitions and glassy dynamics in geometrically frustrated systems and non-Abelian theories”, Physical Review B 69, 014208 (2004)
- (17) http://www.whitehouse.gov/mgi
- (18) Surowiecki, James (2005). The Wisdom of Crowds, Anchor Books. ISBN 0-385-72170-6.
- (19) Peter Ronhovde and Zohar Nussinov, “An Improved Potts Model Applied to Community Detection”, Physical Review E 81, 046114 (2010).
- (20) P. Ronhovde and Z. Nussinov, “Multiresolution community detection for megascale networks by information-based replica correlations”, Phys. Rev. E 80, 016109 (2009).
- (21) Bo Sun, Blake Leonard, Peter Ronhovde, and Zohar Nussinov, “An interacting replica approach applied to the traveling salesman problem”, e-print, arXiv:1406.7282.pdf (2014).
- (22) Marco Dorigo and Thomas St tzle (2004) Ant Colony OptimizationMIT Press, ISBN 0-262-04219-3.
Mitchell, Melanie (1996)
An Introduction to Genetic Algorithms, Cambridge, MA: MIT Press. ISBN 9780585030944.
- (24) S. Fortunato, “Community detection in graphs”, Physics Reports 486, 75-174 (2010).
- (25) M. E. J. Newman, “Modularity and community structure in networks”, Proc Natl Acad Sci U S A. 103 (23), 8577-8582 (2006).
- (26) M. E. J. Newman and M. Girvan, “Finding and evaluating community structure in networks”, Phys. Rev. E 69, 026113 (2004).
- (27) S. Fortunato and M. Barthelemy, “Resolution limit in community detection”, Proceedings of the National Academy of Sciences 104, 36-41 (2007).
- (28) A. Lancichinetti and S. Fortunato, “Community detection algorithms: A comparative analysis”, Phys. Rev. E 80, 056117 (2009).
- (29) V. D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre, “Fast unfolding of communities in large networks”, J. Stat. Mech. 10, 10008 (2008).
- (30) M. E. J. Newman, “Fast algorithm for detecting community structure in networks”, Phys. Rev. E 69, 066133 (2004).
- (31) V. Gudkov, V. Montelaegre, S. Nussinov, and Z. Nussinov, “Community detection in complex networks by dynamical simplex evolution”, Phys. Rev. E 78, 016113 (2008).
- (32) M. Rosvall and C. T. Bergstrom, “Maps of random walks on complex networks reveal community structure”, Proc. Natl. Aca. Sci. U.S.A. 105, 1118-1123 (2008).
- (33) Book chapter by U. Brandes, D. Dellng, M. Gaertler, R. Gorke, M. Hoefer, Z. Nikoloski, and D. Wagner, “On finding graph clusterings with maximum modularity” in the book “Graph-thoeretic concepts in computer science”, Lecture notes in Computer Science, Springer Berlin/Heidelberg, DOI: 10.1007/978-3-540-74839-7 (2007).
- (34) Richard K. Darst, David R. Reichman, Peter Ronhovde, and Zohar Nussinov, “An edge density definition of overlapping and weighted graph communities”, e-print, arXiv:1301.3120 (2013).
- (35) M. E. J. Newman, “Spectral methods for community detection and graph partitioning”, Physical Review E 88, 042822 (2013).
- (36) M. E. J. Newman, “Community detection and graph partitioning”, Europhysics Letters 103, 28003 (2013).
- (37) Richard K. Darst, Zohar Nussinov, and Santo Fortunato, “Improving the performance of algorithms to find communities in networks” Phys. Rev. E 89, 032809 (2014).
- (38) J. Reichardt and S. Bornholdt, “Statistical mechanics of community detection”, Phys. Rev. E 74, 016110 (2006).
P. Tiago Piexoto, “Efficient Monte Carlo and greedy heuristic for the inference of stochastic block models”, Physical Review E89, 012804 (2014).
- (40) J. M. Kumpula, J. Saramaki, K. Kaski, and J. Kertesz, “Limited resolution in complex network community detection with Potts model approach”, Eur. Phys. J. B 56, 41 (2007).
- (41) P. Ronhovde and Z. Nussinov, “Local multi resolution order in community detection”, J. of Stat. Mech. (2015) P01001.
- (42) Lucas G. S. Jeub, Prakash Balachandran, Mason A. Porter, Peter J. Mucha, and Michael W. Mahoney, “Think Locally, Act Locally: The Detection of Small, Medium-Sized, and Large Communities in Large Networks”, e-print, arXiv:1403.3795.pdf (2014).
Manlio De Domenico and Antonio Insolia, “Entropic Approach to Multiscale Clustering Analysis”, Entropy 2012, 14, 865 (2012).
- (44) P. Tiago Piexoto, “Hierarchical Block Structures and High-Resolution Model Selection in Large Networks”, Physical Review X 4, 011047 (2014).
- (45) Shai Wiseman, Marcelo Blatt, and Eytan Domany, “Superparamagnetic clustering of data”, Phys. Rev. E 57, 3767 (1998)
- (46) A. L. N. Fred and A. K. Jain, “Robust Data Clustering”, 2003 Proc. IEEE Comp. Soc. Conf. on Comp. Vis. Pattern Recog. 2, 128-133 (2003).
- (47) M. Meil, “Comparing clusterings - an information based distance”, J. Multivariate Anal. 98, 873-895 (2007).
- (48) L. Danon, A. Diaz-Guilera, J. Duch, and A. Arenas, “Comparing community structure identification”, J. Stat. Mech.: Theory Exp. 9, 09008 (2005).
- (49) G. Bianconi, “Statistical mechanics of multiplex networks: Entropy and overlap”, Physical Review E 87, 062806 (2013).
- (50) P. Ronhovde, S. Chakrabarty, M. Sahu, K. F. Kelton, N. A. Mauro, K . K. Sahu, and Z. Nussinov, “Detecting hidden spatial and spatio-temporal structures in glasses and complex physical systems by multiresolution network clustering”, The European Physics Journal E 34, 105 (2011).
- (51) P. Ronhovde, S. Chakrabarty, M. Sahu, K. K. Sahu, K. F. Kelton, N. Mauro, and Z. Nussinov “Detection of hidden structures on all scales in amorphous materials and complex physical systems: basic notions and applications to networks, lattice systems, and glasses”, Scientific Reports 2, 329 (2012).
- (52) D. Hu, P. Ronhovde, and Z. Nussinov, “A Replica Inference Approach to Unsupervised Multi-Scale Image Segmentation”, Phys. Rev. E 85, 016101 (2012).
- (53) Linda G. Shapiro and George C. Stockman (2001): Computer Vision, pp 279-325, New Jersey, Prentice-Hall, ISBN 0-13-030796-3.
- (54) Jiianbo Shi and Jitendra Malik (2000): ”Normalized Cuts and Image Segmentation”, IEEE Transactions on pattern analysis and machine intelligence, pp 888-905, Vol. 22, No. 8.
- (55) Ling Wang , Hong Cheng, Zicheng Liu, and Ce Zhu, “A robust elastic net approach for feature learning”, Journal of Visual Communication and Image Representation 25, 313 (2014).
- (56) A. A. Abin, F. Mahdisoltani, and H. Beigy, “WISECODE: wise image segmentation based on community detection”, The Imaging Science Journal 62, 327 (2014).
- (57) Dandan Hu, Pinaki Sarder, Peter Ronhovde, Sharon Bloch, Samuel Achilefu, and Zohar Nussinov, “Automatic segmentation of fluorescence lifetime microscopy images of cells using multiresolution community detection: a first study”, Journal of microscopy 253 (1), 54-64 (2014).
- (58) D. Hu, P. Sarder, P. Ronhovde, S. Bloch, S. Achilefu, and Z. Nussinov, “Community detection for fluorescent lifetime microscopy image segmentation”, Proc. SPIE 8949, Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XXI, 89491K (2014); http://dx.doi.org/10.1117/12.2036875
- (59) http://www.eecs.berkeley.edu/Research/Projects/CS/ vision/grouping/.
- (60) See [http://www.gifford.co.uk/?principia/Illusions/dalmatian.htm].
- (61) Dandan Hu, Peter Ronhovde, Zohar Nussinov, “Phase transitions in random Potts systems and the community detection problem: spin-glass type and dynamic perspectives”, Philosophical Magazine 92 (4), 406-445 (2012); e-print, arXiv:1008.2699 (2010).
- (62) Dandan Hu, Peter Ronhovde, Zohar Nussinov, “Stability-to-instability transition in the structure of large-scale networks”, Physical Review E 86, 066106 (2012).
- (63) Peter Ronhovde, Dandan Hu, and Zohar Nussinov, “Global disorder transition in the community structure of large-q Potts systems”, EPL (Europhysics Letters) 99 (3), 38006 (2012).
- (64) O. Melchert and A. K. Hartmann, “Information-theoretic approach to ground-state phase transitions for two- and three-dimensional frustrated spin systems”, Phys. Rev. E 87, 022107 (2013).
S. Cook, “The complexity of theorem-proving procedures”, in the Proceedings of the 3rd Annual ACM Symposium on Theory of Computing (Association for Computing Machinery, New York) p.151-158 (1971).
Peter Cheeseman , Bob Kanefsky, and William M. Taylor, “Where the REALLY Hard Problems Are” In J. Mylopoulos and R. Reiter (Eds.), Proceedings of 12th International Joint Conference on AI (IJCAI-91) Automated reasoning ,1 331 (1991).
- (67) Remi Monasson, Riccardo Zecchina, Scott Kirkpatrick, Bart Selman, and Lidror Troyansky, Nature 400, 133 (1999).
- (68) M. Mezard, G. Parisi, and R. Zecchina, “Analytic and Algorithmic Solution of Random Satisfiability Problems”, Science 297, 812 (2002).
- (69) A. Decelle, F. Krzakala, C. Moore, and L. Zdeborova, “Phase transition in the detection of modules in sparse networks”, Phys. Rev. Lett. 107, 065701 (2011), e-print, arXiv:1102.1182 (2011).
- (70) Elchanan Mossel, Joe Neeman, and Allan Sly, “Stochastic Block Models and Reconstruction”, e-print, arXiv:1202.1499 (2012).
- (71) Raj Rao Nadakuditi and M. E. J. Newman, “Graph Spectra and the Detectability of Community Structure in Networks”, Phys. Rev. Lett. 108, 188701 (2012).
- (72) R. K. Darst, D. R. Reichman, P. Ronhovde, and Z. Nussinov, “Algorithm independent bounds on community detection problems and associated transitions in stochastic block model graphs”, Journal of Complex Networks, doi: 10.1093/comnet/cnu042 (2014).
- (73) Greg Ver Steeg, Cristopher Moore, Aram Galstyan, and Armen Allahverdyan, “Phase transitions in community detection: A solvable toy model”, Europhysics Letters 106, 48004 (2014).
- (74) A. Montanari, “Finding One Community in a Sparse Graph”, e-print, arXiv:1502.05680 (2015).
- (75) Xiao Zhang, Raj Rao Nadakuditi, and M. E. J. Newman, “Spectra of random graphs with community structure and arbitrary degrees”, Physical Review E 89, 042816 (2014).
- (76) J. Reichardt and M. Leone, “(Un)detectable Cluster Structure in Sparse Networks”, Physical Review Letters 101, 78701 (2008).
- (77) S. Kirkpatrick, C. D. Jr. Gelatt, and M. P. Vecchi , ”Optimization by Simulated Annealing”. Science 220, 671 (1983).
- (78) J. Villain, R. Bidaux, J. P. Carton, and R. Conte, “Order as an effect of disorder”, J. Physique 41, 1263 (1980).
C. L. Henley, “Ordering due to disorder in a frustrated vector antiferromagnet”,, Phys. Rev. Lett.,62, 2056 (1989).
- (80) Z. Nussinov, M. Biskup, L. Chayes, and J. van den Brink, “Orbital order in classical models of transition-metal compounds” Europhys. Lett. 67, 990, (2004).
- (81) P. G. Wolynes, “Folding funnels and energy landscapes of larger proteins within the capillarity approximation”, Proc. Natl. Acad. Sci. U S A. 94 (12), 6170-6175 (1997).
- (82) D. S. Bassett, E. T. Owens, K. E. Daniels, and M. A. Porter, “Influence of network topology on sound propagation in granular materials”, Phys. Rev. E 86, 041306 (2012).
- (83) Federica Cerina, Vincenzo De Leo, Marc Barthelemy, and Alessandro Chessa, “Spatial Correlations in Attribute Communities”, PLoS ONE 7(5): e37507 (2012).
- (84) Petter Holme and Jari Saramaki, “Temporal networks”, Physics Reports 519, 97 (2012).
- (85) Alessio Cardillo, Jesus G mez-Gardenes, Massimiliano Zanin, Miguel Romance, David Papo, Francisco del Pozo, and Stefano Boccaletti, “Emergence of network features from multiplexity”, Scientific Reports 3, 1344 (2013).
- (86) Giovanni Petri and Paul Expert, “Temporal stability of network partitions”, Phys. Rev. E 90, 022813 (2014).
- (87) Robert L. Jack, Andrew J. Dunleavy, and C. Patrick Royall, “Information-Theoretic Measurements of Coupling between Structure and Dynamics in Glass Formers”, Phys. Rev. Lett. 113, 095703 (2014).
- (88) J.-P. Bouchaud and G. Biroli, “On the Adams-Gibbs-Kirkpatrick-Thirumalai-Wolynes scenario for the viscosity increase in glasses”, J. Chem. Phys. 121, 7347 (2004).
- (89) M. Mosayebi, E. D. Gado, P. Iig, and H. C. Ottinger, “Probing a Critical Length at the Glass Transition”, Phys. Rev. Lett. 104, 205704 (2010).
- (90) L. Berthier, G. Biroli, J.-P. Bouchaud, L. Cipelletti, D. El Masri, D. L’Hote, F. Ladieu, and M. Pierno, “Direct Experimental Evidence of a Growing Length Scale Accompanying the Glass Transition”, Science 310, 1797 (2005).
- (91) S. Karmakar, C. Dasgupta, and S. Sastry, “Growing length and time scales in glass-forming liquids”, Proc. Natl. Acad. Sci. U.S.A. 106, 3675 (2010).
- (92) J. Kurchan and D. Levine, “Correlation length for amorphous systems”, e-print, arXiv:0904.4850 (2009).
- (93) C. Dasgupta A. V. Indrani, S. Ramaswamy, and M. K. Phani, “Is there a growing correlation length near the glass transition?”, Europhys Lett.15, 307 (1991).
- (94) A. Montanari and G. Semerjian, “Rigorous Inequalities between Length and Time Scales in Glassy Systems”, Journal of Statistical Physics 125, 23-54 (2006).
- (95) W. Kob and H. C. Andersen, “Testing made-coupling theory for a supercooled binary Lennard-Jones mixture: The van Hove correlation function”, Phys. Rev. E 51, 4626 (1995).
- (96) Ryan Soklaski, Zohar Nussinov, Zachary Markow, K. F. Kelton, and Li Yang, “Connectivity of icosahedral network and a dramatically growing static length scale in Cu-Zr binary metallic glasses”, Phys. Rev. B 87, 184203 (2013).