Community detection is one of the critical issues when understanding social networks. In many real-world networks (e.g. Facebook, Twitter), in addition to topology of social network, content information is available as well. Even though different sources of information about social networks can be collected via social media, node attributes and the structure of networks are often interpreted separately in the research of community detection. Usually the primary attention of algorithms has only focused on the topology of the social networks while on the other hand, the decision of community assignments has been made solely based on node attributes. The partial use of data is tremendously inefficient. Sometimes, especially when the network is sparse, algorithms which are incapable of incorporating multiple data sources are often paralyzed and unsuccessful in recovering community assignment. It is of great interests to study how to incorporate topology features and node attributes into one algorithm.
Several papers address community detection with node attributes under the assumption that the observed node attributes are highly correlated with communities. The two main approaches are: heuristic measure-based models and probabilistic inference-based models. The heuristic measure-based models combine topology structure and node attributes in a heuristic function. L. Akoglu et al. proposed a parameter-free identification of cohesive subgroups (PICS) in attributed graphs by minimizing the total encoding costs.Y. Zhou et al.  proposed SA-Cluster based on structural and attribute similarities through a unified distance measure.The probabilistic inference-based approach usually assumes that the networks are generated by random processes and uses probabilistic generative models to combine both topology and attributes. J. Yang et al. 
developed Communities from Edge Structure and Node Attributes (CESNA) for detecting overlapping networks communities with node attributes. In CESNA model, the links are generated by process of BigCLAM and node attributes can be estimated by separate logistic models. B.F. Cai et al. proposed a popularity-productivity stochastic block model with a discriminative framework (PPSB-DC) as the probabilistic generative model. Y.Chen et al.  adopted Bayesian method and developed Bayesian nonparametric attribute (BNPA) model. A nonparametric method was introduced to determine the number of communities automatically. These probabilistic generative models can be further categorized based on two different ways of modeling the stochastic relationship between attributes X, communities F and graph G. CESNA and BNPA assume that communities “generate” both the network as well as attributes (Figure 1 (c) ) however PPSB-DC assumes that communities can be predicted based on attributes and then network are generated based on communities (Figure 1 (d) ).
Even though many studies have shown that social ties are not made random but constrained by social position  , it is possible that the observed node attributes may not contribute much to social position so that they are uncorrelated with communities. When communities and node attributes are not correlated, adding nodes attributes into the above models will not give more information about communities. In this paper we propose an approach that allows us to go beyond the similarity between communities and node attributes. One assumption we rely on is that node attributes will lead to heterogeneity in the degree of nodes (Figure 1 (e)). The idea of including heterogeneity in the degree in SBM was first introduced by Wang and Wong  and later revisited by Karrer . By including this heterogeneity, our approach is able to solve the more challenging problem where node attributes and communities are uncorrelated. Our intuition is that the node attributes label not only the nodes but the edges as well. Due to heterogeneity in the degree, different types of edges carry different information of communities, therefore our approach should be able to recover the communities more accurately.
Another important problem of interests is to understand to which extend the extra information of node attributes will improve the performance, especially when communities and node attributes are not correlated. Here we are focusing on the detectability threshold for our new model. E. Mossel et al. 
have proven that there exists a phase transition in the detectability of communities for two equal size communities in stochastic block model. S. Heimlicher et al. investigated the phase transition phenomena in more general context of labelled stochastic block model and generalized the detectability threshold. A. Ghasemian et al.  derived the detectability threshold in dynamic stochastic block model as a function of the rate of change and the strength of the communities. In this paper we derive the detectability thresholds for community structure in stochastic block model with node attributes and compare it with the original thresholds while no information of node attributes is available.
. Here, we develop a generative model by extending SBM to include heterogeneity due to node attributes in the degree of nodes. In our model, we first assign nodes with different nodes attributes to different communities and then generate the topology of network based on both the community assignment and the node attributes (Figure 1 (e)). The graphical model in Figure 1 (e) can be seen as an extension of the graphic model in Figure 1 (d). The main reason for generalizing the graphic model in Figure 1 (d) instead of the graphic model in Figure 1 (c) is that the graphic model in Figure 1 (d) is a combination of graphical models in Figure 1 (a) and Figure 1 (b), which are corresponding graphical models for clustering problem and community detection in stochastic block model. Therefore we find the graphic model in Figure 1 (d) is a better candidate to combine topology information and node attributes information. In our model, we also assume that all the node attributes are categorical variables. Finally we correct the degree of nodes based on node attributes, which leads to sub-communities structure (Figure 2). This assumption allows heterogeneity in communities and generalize the community in SBM.
We formally describe the generative process of a graph under stochastic block model with node attributes, where represents attributes, as follows. First, we construct an one-to-one map of node attributes from m-dimensional point to 1-dimensional point , where is the number of different types of observed attributes and is from to . Then we assign each of the n nodes into group according to node attributes and the number of nodes in each group is . Using a prior , we assign nodes in attributes category into K communities. We then generate the th element in adjacency matrix, where is the community assignment for node , is the attributes category for node and is the probability of forming an edge between a node from community with attributes and a node from community with attributes .The full likelihood of graph under SBM with node attribute is:
Since , sometimes it’s easier to work with the rescale matrix . When two nodes are from group with category of attributes , the rescale matrix . For subsequent analysis, we will focus on the choice of uniform prior since we are interested in the detectability threshold when attributes are not correlated with communities. We will also limit ourselves to an algorithmically difficult case of block model, where every group k has the same average degree conditional on the type of edge:
If this is not the case, reconstruction can be achieved by labeling nodes based on their degrees.
3 Detectability threshold in SBM with node attributes
The best-known rigorous detectability threshold in sparse SBM has been derived by E. Mossel et al. . In the sparse partition model, where , and , the clustering problem is solvable in polynomial time if . However for it is still an open question to find a rigorous detectability threshold in SBM. The Kesten-Stigum (KS) threshold in statistical physics can be treated as a non-rigorous threshold for  . Let be generated by SBM and define . If then the clustering problem is solvable and the Kesten-Stigum (KS) threshold can be achieved in polynomial time. In the sparse regime, , the graph generated by SBM is locally treelike in the sense that all most all nodes in the giant component have a neighborhood which is a tree up to distance . Therefore the threshold for reconstruction on tree can provide good insight into reconstruction on SBM.
As mentioned before, one intuition is that node attributes label the edges, therefore we consider a multi-type branching process of edges to generate the tree that approximates the graph generated by SBM with node attributes. By defining a Markov chain on the infinite tree, we can derive the construction thresholds on SBM with node attributes.
To construct the multi-type branching process, we first label the edge by the categories of node attributes at the two ends of the edge as , where is the attributes for the node that is closer to the root, is the attributes for the node at the far-end and is from to . So there are different types of edges. Map to and relabel the type edge as . Let dimensional matrix C be the matrix describing the expected number of children, where is the expected number of type edges induced by one type edge. Note that one type of edge will give birth to type of edges if and only if . Let and and ,
When moving outward a type edge, the stochastic transition matrix associate with the edge can be defined as:
The largest eigenvalue for thestochastic transition matrix is 1 and let the second largest eigenvalue be . Define in the matrix as . The robust reconstruction is possible when the absolute value of largest eigenvalue for matrix exceeds .
4 Belief propagation
To recover the community assignments in SBM with node attributes, we use Bayesian inference to learn the latent community:
where is the estimated group assignment and is the original group assignment. The distribution is too complex to compute directly since runs over exponential number of terms. In the regime , the graph is locally treelike therefore belief propagation, which is exact to calculate the marginal probability of community assignment on a tree, can be applied to calculate Bayesian inference efficiently. We’ll show that BP is an optimal algorithm in the sense that it can reach the detectability thresholds for SBM with node attributes.
To write the belief propagation equation, we define conditional marginal probability, denoted as , which is the probability that node belongs to group in the absence of node j. We can compute the messenger from to as:
where is the th element in the adjacency matrix for the graph generated by SBM with node attributes, denotes all the nodes connected to and is a normalization constant ensuring
to be a probability distribution. The marginal probabilitycan be calculated as:
where is a normalization constant ensuring to be a probability distribution. In SBM with node attributes, we have interactions between all pairs of nodes, therefore we have messengers to update for one iteration. To reduce the computational complexity to , we follow past work on BP for SBM citeDecelle. At the cost of making approximation to the messenger, when there is no edge between and, the messenger from to can be calculated as:
Now only messengers sent on edges are needed to be calculated. By introducing an external field, the messenger from to when there is an edge between and can be approximated as:
where the external field can be defined as:
It’s worth noting that is a fixed point in (9).
5 Phase transition in BP and simulation
In this section, we will study the stability of the fixed point under random perturbations. As discussed above, in the sparse regime, the graph generated by SBM with node attributes is locally treelike. Here consider such a tree with levels. On the leave the fixed point is perturbed as , where is random variable. Then the influence of perturbation on leave to the root can be calculated as:
where is the number of type edges on the path from leave to the root and is the transfer matrix for type edges, which, by following the calculation in , can be defined as:
As , as well,therefore ,where is the largest eigenvalue for
. Now let us consider the variance at rootinduced by the random perturbation on all leaves at level . Since the influence of each leaf is independent, the variance of the root can be written as:
When the variances on leaves are amplified exponentially, the fixed point is unstable and BP algorithm is able to recover the community assignment with high probability, otherwise the perturbation on leaves will vanish and the fixed point in stable under BP algorithm. From eq.(13), when is , to determine the phase transition in BP, it’s sufficient to calculate . This calculation can be done by viewing this summation as a weight associated multi-type branching process. Consider thus a multi-type branching process with Possion distribution with mean if the parent-child edge in the corresponding multi-type branching process belongs to type . The variance amplified along the tree generated by the above multi-type branching process and the expected values of the variance at level can be calculated as:
where the th element of is , is an
-dimensional unit vector with theth element equal to and is the node attribute type of the root node . When the largest eigenvalue of exceeds , the fixed point of BP is unstable and the community is detectable. Noting that , therefore BP is an optimal algorithm in the sense that it can reach the detectability threshold in SBM with node attributes even when node attributes and communities are not correlated.
Next, we compare the detectability thresholds for SBM with node attributes with the detectability threshold for the original SBM without information of node attributes. In the following discussion, we will limit ourselves to the case where , and satisfy equation (15),
where . For SBM with node attributes, the community is detectable if
For SBM without information of node attributes, the community is detactable if
By simple calculation, it can be shown that , therefore even in the situation where the observed node attributes are uncorrelated with communities, including node attributes into model will give us more infomation about communities.
We conduct the following simulation to verify the claim of phase transition in BP. Considering for simplicity only two communities and two node attributes, we generate a series of graphs by SBM with node attributes for nodes and various choice of when controlling average degree to be . We use to represnt different choices of and to represent the strength of communities. When the clusterings are maximally strong while at the clusterings are weak. The accuracy of reconstruction is measure by matric introduced by .
In figure 3, we plot metric against for different values of and for each curve, we use a vertical dash line in the same color as the corresponding curve to indicate the detectability threshold. Figure 3 shows that BP can recover communities that are positively correlated with true communities all the way down to the detectability thresholds for various choice of . The algorithm has larger metric with smaller .
In this paper, we consider a model that uses information of nodes attributes in a different way such that this approach will provide more information of latent communities beyond the information carried by SBM even when node attributes are not correlated with communities. We have derived a theoretical detectability threshold for SBM with node attributes, which coincides with phase transition in BP. We also conduct a numerical analysis of the phase transition in BP. While constricted to the two symmetric communities with two node attributes, this condition is sufficient to illustrate how the information of node attributes affects detectability even the node attributes are not correlated with communities.
A nature extension will include edge contents and dynamic settings into the model. Our approach can be applied to this case by including different type of edges into the multi-branching process. On the theoretical front, it has been conjectured  that, for , there’s a regime that the clustering problem is solvable but not in polynomial time. Emmanuel Abbe and Colin Sandon  have developed a non-efficient algorithm that is shown to break down KS threshold at in SBM. As a future work, we’ll try to develop an algorithm that can break down the detectability threshold in our model for large numbers of groups.
I am grateful to Professor Wenxin Jiang and Professor Noshir Contractor for helpful discussion.
-  Akoglu, Leman, et al. ”PICS: Parameter-free Identification of Cohesive Subgroups in Large Attributed Graphs.” SDM. 2012.
-  Zhou, Yang, Hong Cheng, and Jeffrey Xu Yu. ”Graph clustering based on structural/attribute similarities.” Proceedings of the VLDB Endowment 2.1 (2009): 718-729.
-  Yang, Jaewon, Julian McAuley, and Jure Leskovec. ”Community detection in networks with node attributes.” Data mining (ICDM), 2013 ieee 13th international conference on. IEEE, 2013.
-  Chai, Bian-fang, et al. ”Combining a popularity-productivity stochastic block model with a discriminative-content model for general structure detection.”Physical review E 88.1 (2013): 012807.
-  Chen, Yi, et al. ”Network structure exploration in networks with node attributes.” Physica A: Statistical Mechanics and its Applications (2016).
-  McPherson, Miller, Lynn Smith-Lovin, and James M. Cook. ”Birds of a feather: Homophily in social networks.” Annual review of sociology (2001): 415-444.
-  Centola, Damon, et al. ”Homophily, cultural drift, and the co-evolution of cultural groups.” Journal of Conflict Resolution 51.6 (2007): 905-929.
-  Wang, Yuchung J., and George Y. Wong. ”Stochastic blockmodels for directed graphs.” Journal of the American Statistical Association 82.397 (1987): 8-19.
-  Karrer, Brian, and Mark EJ Newman. ”Stochastic blockmodels and community structure in networks.” Physical Review E 83.1 (2011): 016107.
-  Mossel E, Neeman J, Sly A. Stochastic block models and reconstruction. arXiv preprint arXiv:1202.1499. 2012 Feb 7.
-  Janson, Svante, and Elchanan Mossel. ”Robust reconstruction on trees is determined by the second eigenvalue.” Annals of probability (2004): 2630-2649.
-  Heimlicher, Simon, Marc Lelarge, and Laurent Massoulié. ”Community detection in the labelled stochastic block model.” arXiv preprint arXiv:1209.2910 (2012).
-  Ghasemian, Amir, et al. ”Detectability thresholds and optimal algorithms for community structure in dynamic networks.” arXiv preprint arXiv:1506.06179(2015).
-  Holland, Paul W., Kathryn Blackmond Laskey, and Samuel Leinhardt. ”Stochastic blockmodels: First steps.” Social networks 5.2 (1983): 109-137.
-  Faust, Katherine, and Stanley Wasserman. ”Blockmodels: Interpretation and evaluation.” Social networks 14.1 (1992): 5-61.
-  Snijders, Tom AB, and Krzysztof Nowicki. ”Estimation and prediction for stochastic blockmodels for graphs with latent block structure.” Journal of classification 14.1 (1997): 75-100.
-  Kesten, Harry, and Bernt P. Stigum. ”Additional limit theorems for indecomposable multidimensional Galton-Watson processes.” The Annals of Mathematical Statistics 37.6 (1966): 1463-1481.
-  Kesten, Harry, and Bernt P. Stigum. ”Limit theorems for decomposable multi-dimensional Galton-Watson processes.” Journal of Mathematical Analysis and Applications 17.2 (1967): 309-338.
Decelle, Aurelien, et al. ”Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications.” Physical Review E84.6 (2011): 066106.
-  Abbe, Emmanuel, and Colin Sandon. ”Detection in the stochastic block model with multiple clusters: proof of the achievability conjectures, acyclic BP, and the information-computation gap.” arXiv preprint arXiv:1512.09080(2015).