BL-MNE: Emerging Heterogeneous Social Network Embedding through Broad Learning with Aligned Autoencoder

Network embedding aims at projecting the network data into a low-dimensional feature space, where the nodes are represented as a unique feature vector and network structure can be effectively preserved. In recent years, more and more online application service sites can be represented as massive and complex networks, which are extremely challenging for traditional machine learning algorithms to deal with. Effective embedding of the complex network data into low-dimension feature representation can both save data storage space and enable traditional machine learning algorithms applicable to handle the network data. Network embedding performance will degrade greatly if the networks are of a sparse structure, like the emerging networks with few connections. In this paper, we propose to learn the embedding representation for a target emerging network based on the broad learning setting, where the emerging network is aligned with other external mature networks at the same time. To solve the problem, a new embedding framework, namely "Deep alIgned autoencoder based eMbEdding" (DIME), is introduced in this paper. DIME handles the diverse link and attribute in a unified analytic based on broad learning, and introduces the multiple aligned attributed heterogeneous social network concept to model the network structure. A set of meta paths are introduced in the paper, which define various kinds of connections among users via the heterogeneous link and attribute information. The closeness among users in the networks are defined as the meta proximity scores, which will be fed into DIME to learn the embedding vectors of users in the emerging network. Extensive experiments have been done on real-world aligned social networks, which have demonstrated the effectiveness of DIME in learning the emerging network embedding vectors.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

11/30/2017

LATTE: Application Oriented Social Network Embedding

In recent years, many research works propose to embed the network struct...
11/30/2017

LATTE: Application Oriented Network Embedding

In recent years, many research works propose to embed the networked data...
06/19/2020

Measuring transnational social fields through binational link-tracing sampling

We advance binational link-tracing sampling design, an innovative data c...
02/10/2020

AnomalyDAE: Dual autoencoder for anomaly detection on attributed networks

Anomaly detection on attributed networks aims at finding nodes whose pat...
09/01/2021

Deep Dual Support Vector Data Description for Anomaly Detection on Attributed Networks

Networks are ubiquitous in the real world such as social networks and co...
11/29/2017

Representation Learning for Scale-free Networks

Network embedding aims to learn the low-dimensional representations of v...
05/15/2019

Missing Movie Synergistic Completion across Multiple Isomeric Online Movie Knowledge Libraries

Online knowledge libraries refer to the online data warehouses that syst...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

In the era of big data, a rapidly increasing number of online application websites appear recently, which can be represented as massive and complex networks. The representative examples include online social networks, like Facebook and Twitter, e-commerce sites, like Amazon and eBay, academic sites, like DBLP and Google Scholar, as well as POIs recommendation sites, like Foursquare and Yelp. These network data can be very difficult to deal with due to their extremely large scale (involving millions even billions of nodes), complex structures (containing heterogeneous links) as well as diverse attributes (attached to the nodes or links). Great challenges exist in handling these complex network representation data with traditional machine learning algorithms, which usually take feature vectors as the input and cannot handle graph data directly. A general representation of heterogeneous networks as feature vectors is desired for knowledge discovery from such complex-structured data. In this paper, we will use online social networks as the example to illustrate the studied problem as well as the learning framework.

In recent years, many research works propose to embed the online social network data into a low-dimensional feature space [14, 18, 6], in which each node is represented as a unique feature vector. From these feature vectors, the original network structure can be effectively reconstructed. With these embedded feature vectors, classic machine learning algorithms can be applied to deal with the social network data directly, and the storage space can also be saved greatly. However, most existing social network embedding methods are proposed for homogeneous networks, which learn the feature vectors for user nodes merely based on the social connections among them. When applied to handle real-world social network data, these embedding models can hardly work well. The main reason is that the internal social links are usually very sparse in online social networks [18], which can hardly preserve the complete network structure. For a pair of users who are not directed connected, these models will not be able determine the closeness of these users’ feature vectors in the embedding space. Such a problem will be more severe when it comes to the emerging social networks [24], which denote the newly created online social networks with very few social connections.

Aligned Heterogeneous Translation based Graph Homogeneous Network Heterogeneous Network
Property Network Embedding Embedding [3, 19, 11] Embedding [14, 18, 6] Embedding [4, 5]
target network emerging regular regular regular
network attributed heterogeneous multi-relational homogeneous heterogeneous
#network multiple single single single
proximity meta proximity first order first/second order/random walk first order [4], meta path [5]
multi-source fusion anchor link based fusion N/A N/A N/A
TABLE I: Summary of related problems.

Meanwhile, as discovered in [30], to enjoy more social network services, people nowadays are usually involved in multiple online social networks at the same time. For instance, people tend to join in Facebook for casual socialization with their classmates; they will use Foursquare to search for nearby restaurants for dinner; and they will turn to use Instagram to share photos with their friends online. Users who are involved in these emerging social networks may have been using other well-developed social networks (e.g., Facebook, Twitter) for a long time. Information available for the users in other aligned mature networks is usually very abundant and of a sufficient amount. Effective information exchanges from these mature networks to the emerging networks for the shared users can help overcome the information sparsity problem promisingly, which is a important topic covered in the broad learning task [22, 31] to be introduced as follows. To denote the accounts owned by the same people in different online social networks, an anchor link will be added to connect their account pair between the networks [30]. Formally, the online social networks connected by the anchor links (between the shared user accounts) are called multiple aligned social networks [30].

Problem Studied: In this paper, we propose to study the emerging network embedding problem across multiple aligned heterogeneous social networks simultaneously based on the broad learning setting, which is formally named as the “Broad Learning based eMerging Network Embedding” (BL-MNE) problem. In the concurrent embedding process based on the broad learning setting, BL-MNE aims at distilling relevant information from both the emerging and other aligned mature networks to derive compliment knowledge and learn a good vector representation for user nodes in the emerging network.

Here, “Broad Learning[22, 31] is a new type of learning task, which focuses on fusing multiple large-scale information sources of diverse varieties together and carrying out synergistic data mining tasks across these fused sources in one unified analytic [25, 21, 30, 24, 26, 23, 29, 27, 28]. In the real world, on the same information entities, e.g., social media users [25, 21, 30, 24], movie knowledge library entries [31] and employees in companies [26, 23, 29, 27, 28], a large amount of information can actually be collected from various sources. These sources are usually of different varieties, like Foursquare vs Twitter [25, 21, 30, 24], IMDB vs Douban Movie sites [31], Yammer vs company organizational chart [26, 23, 29, 27, 28]. Each information source provides a specific signature of the same entity from a unique underlying aspect. Effective fusion of these different information sources provides an opportunity for researchers and practitioners to understand the entities more comprehensively, which renders “Broad Learning” an extremely important learning task. Fusing and mining multiple information sources of large volumes and diverse varieties are the fundamental problems in big data studies. “Broad Learning” investigates the principles, methodologies and algorithms for synergistic knowledge discovery across multiple information sources, and evaluates the corresponding benefits [22, 31]. Great challenges exist in “Broad Learning” for the effective fusion of relevant knowledge across different aligned information sources depends upon not only the relatedness of these information sources, but also the target application problems. “Broad Learning” aims at developing general methodologies, which will be shown to work for a diverse set of applications, while the specific parameter settings can be learned for each application from the training data [22, 31].

BL-MNE is significantly different from existing network embedding problems [3, 19, 14, 11, 18, 4, 6, 5] in several perspectives. First of all, the target network studied in BL-MNE is an emerging network suffering from the information sparsity problem, which is different from the embedding problems for regular networks [3, 19, 11, 4, 5]. Secondly, the networks studied in BL-MNE are all heterogeneous networks containing complex links and diverse attributes, which renders BL-MNE different from existing homogeneous network embedding problems [14, 18, 6]. Furthermore, BL-MNE is based on the multiple aligned networks setting, where information from aligned networks will be exchanged to refine the embedding results mutually, and it is different from the existing single-network based embedding problems [3, 19, 14, 11, 4, 18, 4, 6, 5]. We also provide a summary about the difference between BL-MNE and existing works in Table I (which summarizes and compares several related works in different aspects), and more information about other related works will be introduced Section V at the end of the paper.

The BL-MNE problem is not an easy problem, and it has several great challenges to deal with, which are provided as follows:

  • Problem Formulation: To overcome the information sparsity problem, BL-MNE studies the concurrent embedding of multiple aligned social networks, which is still an open problem to this context so far. Formal definition and formulation of the BL-MNE problem is required before we introduce the solutions.

  • Heterogeneity of Networks: The networks studied in this paper are of very complex structures. Besides the regular social connections among users, there also exist many other types of links as well as diverse attributes attached to the user nodes. Effective incorporating these heterogeneous information into a unified embedding analytic is a great challenge.

  • Multiple Aligned Network Embedding Framework: Due to the significant differences between BL-MNE with the existing works, few existing network embedding models can be applied to address the BL-MNE directly. A new embedding learning framework is needed to learn the emerging network embedding vectors across multiple aligned networks synergistically.

To address all these challenges aforementioned, in this paper, we introduce a novel multiple aligned heterogeneous social network embedding framework, named “Deep alIgned autoencoder based eMbEdding” (DIME). To handle the heterogeneous link and attribute information in a unified analytic, we introduce the aligned attribute augmented heterogeneous network concept in this paper. From these heterogeneous networks a set of meta paths are introduced to represent the diverse connections among users in online social networks (via social links, other diverse connections, and various attributes). A set of meta proximity

measures are defined for each of the meta paths denoting the closeness among users. The meta proximity information will be fed into a deep learning framework, which takes the input information from multiple aligned heterogeneous social networks simultaneously, to achieve the embedding feature vectors for all the users in these aligned networks. Based on the connection among users, framework

DIME aims at embedding close user nodes to a close area in the low-dimensional feature space for each of the social networks respectively. Meanwhile, framework DIME also poses constraints on the feature vectors corresponding to the shared users across networks to map them to a relatively close region. In this way, information can be transferred from the mature networks to the emerging network and solve the information sparsity problem.

The remaining parts of this paper are organized as follows. We will provide the terminology definition and problem formulation in Section II. Information about the framework is available in Section III, which will be evaluated in Section IV. Finally, we will introduce the related works in Section V and conclude this paper in Section VI.

Ii Terminology Definition and Problem Formulation

In this section, we will first introduce the definitions of several important terminologies, based on which we will then provide the formulation of the BL-MNE problem.

Ii-a Terminology Definition

The social networks studied in this paper contain different categories of nodes and links, as well as very diverse attributes attached to the nodes. Formally, we can represent these network structured data as the attributed heterogeneous social networks.

Definition 1 (Attributed Heterogeneous Social Networks): The attributed heterogeneous social network can be represented as a graph , where denotes the set of nodes belonging to various categories and represents the set of diverse links among the nodes. What’s more, = denotes the set of attributes attached to the nodes in . For user in the network, we can represent the type of attribute associated to as , and all the attributes has can be represented as .

The social network datasets used in this paper include Foursquare and Twitter. Formally, the Foursquare and Twitter can both be represented as the attributed heterogeneous social networks , where involves the user and post nodes, and contains the links among users and those between users and posts. In addition, the nodes in are also attached with a set of attributes, i.e., . For instance, for the posts written by users, we can obtain the contained textual contents, timestamps and checkins, which can all be represented as the attributes of the post nodes.

Between Foursquare and Twitter, there may exist a large number of shared common users, who can align the networks together. In this paper, we will follow the concept definitions proposed in [30], and call the user account correspondence relationships as the anchor links. Meanwhile, the networks connected by the anchor links are called the multiple aligned attributed heterogeneous social networks (or aligned social networks for short).

Definition 2 (Multiple Aligned Social Networks): Formally, given attributed heterogeneous social networks with shared users, they can be defined as multiple aligned social networks . Set represents the anchor links between and . User pair iff and are the accounts of the same user in networks and respectively.

For the Foursquare and Twitter social networks used in this paper, we can represent them as two aligned social networks , which will be used as an example to illustrate the models. A simple extension of the proposed framework can be applied to k aligned networks very easily.

Ii-B Problem Formulation

Problem Definition (BL-MNE Problem): Given two aligned networks , where is an emerging network and is a mature network, BL-MNE aims at learning a mapping function to project the user node in to a feature space of dimension (). The objective of mapping functions is to ensure the embedding results can preserve the network structural information, where similar user nodes will be projected to close regions. Furthermore, in the embedding process, BL-MNE also wants to transfer information between and to overcome the information sparsity problem in .

Iii Proposed Method

In this section, we will introduce the framework DIME in detail. At the beginning, we provide the notations used in the paper. After that, in Section III-B, we will talk about how to calculate the meta proximity scores among users based on information in the attributed heterogeneous social networks. With the meta proximity measures, the DIME framework will be introduced in Section III-C to obtain the embedding vectors of user nodes across aligned networks, where information from other aligned mature networks will be used to refine the embedding vectors in the emerging sparse network.

Iii-a Notations

In the sequel, we will use the lower case letters (e.g., ) to represent scalars, lower case bold letters (e.g., ) to denote column vectors, bold-face upper case letters (e.g., ) to denote matrices, and upper case calligraphic letters (e.g., ) to denote sets. Given a matrix , we denote and as the row and column of matrix respectively. The (, ) entry of matrix can be denoted as either or , which will be used interchangeably in this paper. We use and to represent the transpose of matrix and vector . For vector , we represent its -norm as . The -norm of matrix can be represented as . The element-wise product of vectors and of the same dimension is represented as , while the element-wise product of matrices and of the same dimensions is represented as .

ID Notation Heterogeneous Network Meta Path Semantics
U U User User Follow
U U U User User User Follower of Follower
U U U User User User Common Out Neighbor
U U U User User User Common In Neighbor
U U U User User User Followee of Followee
U P W P U User Post Word Post User Posts Containing Common Words
U P T P U User Post Time Post User Posts Containing Common Timestamps
U P L P U User Post Location Post User Posts Attaching Common Location Check-ins
TABLE II: Summary of Social Meta Paths (for both Foursquare and Twitter).

Iii-B Heterogeneous Network Meta Proximity

For each attributed heterogeneous social network, the closeness among users can be denoted by the friendship links among them, where friends tend to be closer compared with user pairs without connections. Meanwhile, for the users who are not directly connected by the friendship links, few existing embedding methods can figure out their closeness, as these methods are mostly built based on the direct friendship link only. In this section, we propose to infer the potential closeness scores among the users with the heterogeneous information in the networks based on meta path concept [16], which are formally called the meta proximity in the paper.

Iii-B1 Friendship based Meta Proximity

In online social networks, the friendship links are the most obvious indicator of the social closeness among users. Online friends tend to be closer with each other compared with the user pairs who are not friends. Users’ friendship links also carry important information about the local network structure information, which should be preserved in the embedding results. Based on such an intuition, we propose the friendship based meta proximity concept as follows.

Definition 3 (Friendship based Meta Proximity): For any two user nodes in an online social network (e.g., ), if and are friends in , the friendship based meta proximity between and in the network is , otherwise the friendship based meta proximity score between them will be instead. To be more specific, we can represent the friendship based meta proximity score between users as , where iff .

Based on the above definition, the friendship based meta proximity scores among all the users in network can be represented as matrix , where entry equals to . Here denotes the simplest meta path of length in the form , and its formal definition will be introduced in the following subsection.

When network is an emerging online social network which has just started to provide services for a very short time, the friendship links among users in tend to be very limited (majority of the users are isolated in the network with few social connections). In other words, the friendship based meta proximity matrix will be extremely sparse, where few entries will have value and most of the entries are s. With such a sparse matrix, most existing embedding models will fail to work. The reason is that the sparse friendship information available in the network can hardly categorize the relative closeness relationships among the users (especially for those who are even not connected by friendship links), which renders these existing embedding models may project all the nodes to random regions.

To overcome such a problem, besides the social links, we propose to calculate the proximity scores for the users with the diverse link and attribute information in the heterogeneous networks in this paper. Based on a new concept named attribute augmented meta path, a set of meta proximity measures will be defined with each of the meta paths, which will be introduced in the following sections.

Iii-B2 Attribute Augmented Meta Path

To handle the diverse links and attributes simultaneously in a unified analytic, we propose to treat the attributes as nodes as well and introduce the attribute augmented network. If a node has certain attributes, a new type of link “have” will be added to connected the node and the newly added attribute node. The structure of the attribute augmented network can be described with the attribute augmented network schema as follows.

Definition 4 (Attribute Augmented Network Schema): Formally, the network schema of a given online social network can be represented as , where and denote the set of node and attribute categories in the network, while represents the set of link types in the network, and represents the relationship between node and attribute node types.

For instance, about the attributed heterogeneous social network introduced after Definition 1 in Section II, we can represent its network schema as . The node type set involves node types (or for simplicity), while the attribute type set includes {Word, Time, Location} (or for short). As to the link types involved in the network, the link type set contains , which represents the friendship link type and the write link type respectively.

Based on the attribute augmented network schema, we can represent the general correlation among users (especially those who are directly connected by friendship links) with the attributed augmented meta path starting and ending with the user node type.

Definition 5 (Attribute Augmented Meta Path): Given a network schema , the attribute augmented meta path denotes a sequence of node/attribute types connected by the link types or the “have” relation type (between node and attribute type). Formally, the attribute augmented meta path (of length , ) can be represented as , where and (superscript denotes the reverse of relation type direction). In the case that , i.e., meta paths starts and ends with the user node type, the meta paths will be called the social meta paths specifically.

Based on the above definition, a set of different social meta path can be extracted from the network, whose notations, concrete representations and the physical meanings are illustrated in Table II. Here, meta paths are all based on the user node type and follow link type; meta paths involve the user, post node type, attribute node type, as well as the write and have link type. Based on each of the meta paths, there will exist a set of concrete meta path instances connecting users in the networks. For instance, given a user pair and , they may have been checked-in at 5 different common locations, which will introduce concrete meta path instance of meta path connecting and indicating their strong closeness (in location check-ins). In the next subsection, we will introduce how to calculate the proximity score for the users based on these extracted meta paths.

Iii-B3 Heterogeneous Network Meta Proximity

The set of attribute augmented social meta paths extracted in the previous subsection create different kinds of correlations among users (especially for those who are not directed connected by friendship links). With these social meta paths, different types of proximity scores among the users can be captured. For instance, for the users who are not friends but share lots of common friends, they may also know each other and can be close to each other; for the users who frequently checked-in at the same places, they tend to be more close to each other compared with those isolated ones with nothing in common. Therefore, these meta paths can help capture much broader network structure information compared with the local structure captured by the friendship based meta proximity covered in Section III-B1. In this part, we will introduce the method to calculate the proximity scores among users based on these social meta paths.

As shown in Table II, all the social meta paths extracted from the networks can be represented as set . Given a pair of users, e.g., and , based on meta path , we can represent the set of meta path instances connecting and as . Users and can have multiple meta path instances going into/out from them. Formally, we can represent all the meta path instances going out from user (or going into ), based on meta path , as set (or ). The proximity score between and based on meta path can be represented as the following meta proximity concept formally.

Definition 6 (Meta Proximity): Based on meta path , the meta proximity between users and in can be represented as

Meta proximity considers not only the meta path instances between users but also penalizes the number of meta path instances going out from/into and at the same time. It is also reasonable. For instance, sharing some common location check-ins with some extremely active users (who have thousands of checkins) may not necessarily indicate closeness with them, since they may have common check-ins with almost all other users simply due to his very large check-in record volume instead of their closeness.

With the above meta proximity definition, we can represent the meta proximity scores among all users in the network based on meta path as matrix , where entry . All the meta proximity matrices defined for network can be represented as . Based on the meta paths extracted for network , similar matrices can be defined as well, which can be denoted as .

Fig. 1: The DIME Framework.

Iii-C Deep Network Synergistic Embedding

With these calculated meta proximity introduced in the previous section, we will introduce the embedding framework DIME in this part. DIME is based on the aligned auto-encoder model, which extends the traditional deep auto-encoder model to the multiple aligned heterogeneous networks scenario. To make this paper self-contained, we will first briefly introduce some background knowledge about the auto-encoder model first in Section III-C1. After that, we will talk about the embedding model component for one single heterogeneous network in Section III-C2, which takes the various meta proximity matrices as input. DIME effectively couples the embedding process of the emerging network with other aligned mature networks, where cross-network information exchange and refinement is achieved via the loss term defined based on the anchor links.

Iii-C1 Deep Auto-Encoder Model Review

Auto-encoder is an unsupervised neural network model, which projects the instances (in original feature representations) into a lower-dimensional feature space via a series of non-linear mappings. Auto-encoder model involves two steps: encoder and decoder. The encoder part projects the original feature vectors to the objective feature space, while the decoder step recovers the latent feature representation to a reconstruction space. In auto-encoder model, we generally need to ensure that the original feature representation of instances should be as similar to the reconstructed feature representation as possible.

Formally, let represent the original feature representation of instance , and be the latent feature representation of the instance at hidden layers in the encoder step, the encoding result in the objective feature space can be represented as with dimension . Formally, the relationship between these variables can be represented with the following equations:

Meanwhile, in the decoder step, the input will be the latent feature vector (i.e., the output of the encoder step), and the final output will be the reconstructed vector . The latent feature vectors at each hidden layers can be represented as . The relationship between these vector variables can be denoted as

The objective of the auto-encoder model is to minimize the loss between the original feature vector and the reconstructed feature vector of all the instances in the network. Formally, the loss term can be represented as

Iii-C2 Deep Dime-Sh Model

When applying the auto-encoder model for one single homogeneous network, e.g., for , we can fit the model with the node meta proximity feature vectors, i.e., rows corresponding to users in matrix (introduced in Section III-B1). In the case that is heterogeneous, multiple node meta proximity matrices have been defined before (i.e., ), how to fit these matrices simultaneously to the auto-encoder models is an open problem. In this part, we will introduce the single-heterogeneous-network version of framework DIME, namely DIME-SH, which will be used as an important component of framework DIME as well. For each user node in the network, DIME-SH computes the embedding vector based on each of the proximity matrix independently first, which will be further fused to compute the final latent feature vector in the output hidden layer.

As shown in the architecture in Figure 1 (either the left component for network 1 or the right component for network 2), about the same instance, DIME-SH takes different feature vectors extracted from the meta paths as the input. For each meta path, a series of separated encoder and decoder steps are carried out simultaneously, whose latent vectors are fused together to calculate the final embedding vector for user . In the DIME-SH model, the input feature vectors (based on meta path ) of user can be represented as , which denotes the row corresponding to users in matrix defined before. Meanwhile, the latent representation of the instance based on the feature vector extracted via meta path at different hidden layers can be represented as .

One of the significant difference of model DIME-SH from traditional auto-encoder model lies in the (1) combination of multiple hidden vectors to obtain the embedding vector in the encoder step, and (2) the dispatch of embedding vector back to the hidden vectors in the decoder step. As shown in the architecture, formally, these extra steps can be represented as

In the fusion and dispatch steps, full connection layers are used in order to incorporate all the information captured by all the meta paths.

What’s more, since the input feature vectors are extremely sparse (lots of the entries are s), simply feeding them to the model may lead to some trivial solutions, like vectors for both and the decoded vectors . To overcome such a problem, another significant difference of model DIME-SH

from traditional auto-encoder model lies in the loss function definition, where the loss introduced by the non-zero features will be assigned with a larger weight. In addition, by adding the loss function for each of the meta paths, the final loss function in

DIME-SH can be formally represented as

where vector is the weight vector corresponding to feature vector . Entries in vector are filled with value s except the entries corresponding to non-zero element in , which will be assigned with value ( denoting a larger weight to fit these features). Here, we need to add a remark that “simply discarding the entries corresponding zero values in the input vectors from the loss function” will not work here, since it will allow the model to decode there entries to any random values, which will not be what we want. In a similar way, we can define the loss function for the embedding result in network , which can be formally represented as .

Iii-C3 Deep Dime Framework

DIME-SH has incorporate all these heterogeneous information in the model building, the meta proximity calculated based on which can help differentiate the closeness among different users. However, for the emerging networks which just start to provide services, the information sparsity problem may affect the performance of DIME-SH significantly. In this part, we will introduce DIME, which couples the embedding process of the emerging network with another mature aligned network. By accommodating the embedding between the aligned networks, information can be transferred from the aligned mature network to refine the embedding results in the emerging network effectively. The complete architecture of DIME is shown in Figure 1, which involve the DIME-SH components for each of the aligned networks, where the information transfer component aligns these separated DIME-SH models together.

To be more specific, given a pair of aligned heterogeneous networks ( is an emerging network and is a mature network), we can represent the embedding results as matrices and for all the user nodes in and respectively. The row of matrix (or the row of matrix ) denotes the encoded feature vector of user in (or in ). If and are the same user, i.e., , by placing vectors and in a close region in the embedding space, we can use the information from to refine the embedding result in .

Information transfer is achieved based on the anchor links, and we only care about the anchor users. To adjust the rows of matrices and to remove non-anchor users and make the same rows correspond to the same user, we introduce the binary inter-network transitional matrix . Entry iff the corresponding users are connected by anchor links, i.e., . Furthermore, the encoded feature vectors for users in these two networks can be of different dimensions, i.e., , which can be accommodated via the projection .

Formally, the introduced information fusion loss between networks and can be represented as

By minimizing the information fusion loss function , we can use the anchor users’ embedding vectors from the mature network to adjust his embedding vectors in the emerging network . Even through in such a process the embedding vector in can be undermined by , it will not be a problem since is our target network and we only care about the embedding result of the emerging network in the paper.

network
property Twitter Foursquare
# node user 5,223 5,392
tweet/tip 9,490,707 48,756
location 297,182 38,921
# link friend/follow 164,920 76,972
write 9,490,707 48,756
locate 615,515 48,756
TABLE III: Properties of the Heterogeneous Networks

The complete objective function of framework include the loss terms introduced by the component DIME-SH for networks , , and the information fusion loss, which can be denoted as

Parameters and denote the weights of the information fusion loss term and the regularization term. In the objective function, term is added to the above objective function to avoid overfitting, which can be formally represented as

To optimize the above objective function, we utilize Stochastic Gradient Descent (SGD). To be more specific, the training process involves multiple epochs. In each epoch, the training data is shuffled and a minibatch of the instances are sampled to update the parameters with SGD. Such a process continues until either convergence or the training epochs have been finished.

Sampling Ratio
metric method
AUC DIME 0.7920.007 0.8220.006 0.8380.005 0.8430.003 0.8470.003 0.8500.003 0.8520.002 0.8500.004 0.8520.003 0.8520.004
DIME-SH 0.7740.006 0.7950.005 0.8020.006 0.8090.004 0.8150.005 0.8220.005 0.8270.006 0.8260.005 0.8300.004 0.8330.003
Auto-encoder 0.6970.006 0.7310.006 0.7520.005 0.7610.005 0.7610.005 0.7630.004 0.7630.004 0.7700.003 0.7730.005 0.7770.005
LINE 0.6940.008 0.7160.003 0.7310.007 0.7380.005 0.7410.006 0.7440.005 0.7480.005 0.7480.008 0.750.003 0.7500.006
DeepWalk 0.6710.010 0.6610.011 0.6700.009 0.6750.009 0.6820.005 0.6870.004 0.7010.007 0.7180.007 0.7330.008 0.7470.006
Accuracy DIME 0.7190.006 0.7480.005 0.7630.004 0.7670.003 0.7730.003 0.7750.004 0.7770.003 0.7750.003 0.7770.004 0.7770.004
DIME-SH 0.7040.007 0.7230.004 0.7280.006 0.7370.003 0.7390.006 0.7470.005 0.7530.006 0.7540.006 0.7570.005 0.7610.003
Auto-encoder 0.6420.005 0.6680.005 0.6840.005 0.6920.005 0.6910.005 0.6910.004 0.6910.004 0.6990.004 0.7000.005 0.7030.005
LINE 0.6370.005 0.6660.004 0.6760.008 0.6760.005 0.6770.004 0.6790.006 0.6790.005 0.6790.008 0.6810.003 0.6820.007
DeepWalk 0.6320.008 0.6260.009 0.6330.008 0.6340.008 0.6370.006 0.6410.004 0.6550.007 0.6690.006 0.6800.006 0.6870.004
Recall DIME 0.6410.008 0.7020.011 0.7320.007 0.7460.008 0.7550.008 0.7610.006 0.7670.006 0.7680.003 0.7710.005 0.7720.008
DIME-SH 0.6410.011 0.6890.006 0.6920.010 0.7030.009 0.7070.009 0.7130.013 0.7200.010 0.7190.012 0.7250.009 0.7310.008
Auto-encoder 0.5640.016 0.6490.016 0.7140.007 0.7430.013 0.7260.012 0.6800.009 0.6710.007 0.6810.008 0.6800.008 0.6870.007
LINE 0.8190.008 0.7700.007 0.8000.008 0.7490.011 0.7400.008 0.7310.010 0.7240.007 0.7210.007 0.7150.005 0.7160.009
DeepWalk 0.6450.020 0.6580.020 0.6780.016 0.6810.016 0.6800.016 0.6820.010 0.6920.009 0.7020.008 0.7060.005 0.7070.007
F1 DIME 0.7000.007 0.7350.008 0.7560.006 0.7620.006 0.7690.005 0.7720.005 0.7750.004 0.7740.004 0.7760.005 0.7760.006
DIME-SH 0.6840.009 0.7110.005 0.7180.008 0.7280.006 0.7310.007 0.7380.008 0.7440.007 0.7450.009 0.7490.006 0.7530.005
Auto-encoder 0.6120.009 0.6620.009 0.6930.007 0.7070.007 0.7010.007 0.6880.006 0.6850.006 0.6940.006 0.6940.007 0.6980.008
LINE 0.6930.006 0.6980.005 0.7120.008 0.6980.007 0.6960.006 0.6950.009 0.6930.006 0.6920.009 0.6920.005 0.6930.009
DeepWalk 0.6360.011 0.6370.014 0.6480.010 0.6500.010 0.6520.010 0.6550.006 0.6670.008 0.6790.006 0.6880.006 0.6930.005
TABLE IV: Link prediction result of the comparison methods (parameter changes in , ).

Iv Experiments

To demonstrate the effectiveness of the learnt embedding feature vectors, extensive experiments have been done on real-world aligned heterogeneous social networks, Foursquare and Twitter. Two different tasks are done in this section for embedding result evaluation purposes, which include link prediction and community detection

. In this section, we will provide some basic descriptions about the aligned heterogeneous social network dataset first. After that, we will introduce the experimental settings, covering the comparison embedding methods, as well as the experimental settings, and evaluation metrics for

link prediction and community detection tasks. Finally, we will show the experimental results about link prediction and community detection, followed by the parameter analysis.

Iv-a Dataset Description

The data used in the experiments include two aligned heterogeneous social networks Foursquare and Twitter simultaneously. The basic statistical information about the Foursquare and Twitter datasets is available in Table III. The data crawling strategy and method is introduced in great deital in [8, 30].

  • Twitter: Twitter is a famous micro-blogging site that allows users to write, read and share posts with their friends online. We have crawled Twitter users, and follow links among them. These crawled Twitter users have posted tweets, among which have location checkins.

  • Foursquare: Foursquare is a famous location based social network (LBSN), which provides users with various kinds of location-related services. From Foursquare, we have crawled users together with friendship links among them. These Foursquare users have written posts which all attach location checkins. Among these crawled Foursquare users, of them are aligned by anchor links with Twitter.

In the experiments, we will use Foursquare as the emerging network and Twitter as the aligned mature network, since Twitter has more dense information than Foursquare. The results for the reverse case (Foursquare: mature; Twitter: emerging) are not shown here due to the limited space.

Source Code: The source code of DIME is available at site: http://www.ifmlab.org/files/code/Aligned-Autoencoder.zip.

Iv-B Experimental Settings

In this paper, we are mainly focused on studying the embedding models, and different network embedding comparison methods will be introduced first. After that, we will introduce the experimental settings for both link prediction and community detection tasks, which will be used as the evaluation tasks to determine whether the embedding results are good or not. A set of frequently used evaluation metrics for link prediciton and community detection will be introduced afterwards.

Iv-B1 Embedding Comparison Methods

Sampling Ratio
metric method
Density DIME 0.0020.000 0.0030.000 0.0030.000 0.0030.000 0.0040.000 0.0030.000 0.0040.000 0.0030.000 0.0030.000 0.0040.000
DIME-SH 0.0020.000 0.0020.000 0.0030.000 0.0030.000 0.0030.000 0.0030.000 0.0030.000 0.0030.000 0.0030.000 0.0030.000
Auto-encoder 0.0020.000 0.0020.000 0.0020.000 0.0020.000 0.0030.000 0.0030.000 0.0030.000 0.0030.000 0.0030.000 0.0030.000
LINE 0.0010.000 0.0020.000 0.0020.000 0.0020.000 0.0020.000 0.0020.000 0.0020.000 0.0020.000 0.0020.000 0.0020.000
DeepWalk 0.0010.000 0.0010.000 0.0010.000 0.0010.000 0.0010.000 0.0010.000 0.0010.000 0.0010.000 0.0010.000 0.0010.000
Separability DIME 0.2300.007 0.2960.027 0.3600.021 0.3690.028 0.4400.023 0.3810.028 0.4660.022 0.3470.004 0.3980.025 0.4140.018
DIME-SH 0.2510.033 0.2350.007 0.2880.013 0.2940.010 0.3110.020 0.2820.018 0.2980.007 0.3700.011 0.3620.010 0.3720.017
Auto-encoder 0.2470.031 0.1810.012 0.2010.016 0.2510.020 0.2590.013 0.2720.014 0.2720.019 0.2960.012 0.2810.019 0.2710.011
LINE 0.1320.003 0.1530.010 0.1670.004 0.1570.007 0.1790.008 0.1860.007 0.2030.007 0.2000.011 0.1900.010 0.2210.013
DeepWalk 0.1130.001 0.1170.002 0.1200.002 0.1210.003 0.1230.002 0.1210.001 0.1230.002 0.1240.003 0.1240.002 0.1230.002
Coverage DIME 0.1870.005 0.2280.016 0.2640.011 0.2690.015 0.3060.011 0.2760.014 0.3180.011 0.2580.002 0.2850.013 0.2920.009
DIME-SH 0.2000.021 0.1900.005 0.2240.008 0.2270.006 0.2370.011 0.2200.011 0.2290.004 0.2700.006 0.2660.005 0.2710.009
Auto-encoder 0.1980.020 0.1530.009 0.1670.011 0.2010.013 0.2060.008 0.2140.009 0.2130.012 0.2280.007 0.2190.012 0.2130.007
LINE 0.1170.003 0.1330.008 0.1430.003 0.1360.005 0.1520.006 0.1570.005 0.1680.005 0.1670.008 0.1600.007 0.1810.009
DeepWalk 0.1020.001 0.1050.001 0.1070.002 0.1080.002 0.1100.001 0.1080.001 0.1100.001 0.1100.002 0.1110.002 0.1100.001
Expansion DIME 0.8130.005 0.7720.016 0.7360.011 0.7310.015 0.6940.011 0.7240.014 0.6820.011 0.7420.002 0.7150.013 0.7080.009
DIME-SH 0.8000.021 0.8100.005 0.7760.008 0.7730.006 0.7630.011 0.7800.011 0.7710.004 0.7300.006 0.7340.005 0.7290.009
Auto-encoder 0.8020.020 0.8470.009 0.8330.011 0.7990.013 0.7940.008 0.7860.009 0.7870.012 0.7720.007 0.7810.012 0.7870.007
LINE 0.8830.003 0.8670.008 0.8570.003 0.8640.005 0.8480.006 0.8430.005 0.8320.005 0.8330.008 0.8400.007 0.8190.009
DeepWalk 0.8980.001 0.8950.001 0.8930.002 0.8920.002 0.8900.001 0.8920.001 0.8900.001 0.8900.002 0.8890.002 0.8900.001
TABLE V: Community detection result of the comparison methods (parameter changes in , ).

The network embedding models compared in the experiments are listed as follows

  • DIME: DIME is the synergistic embedding model for multiple aligned heterogeneous networks introduced in this paper. DIME preserves both the local and global network structure with a set of meta proximity calculated from each of the heterogeneous network. DIME transfers the information from the aligned mature networks to the emerging network with the anchor links, which accommodate the learnt embedding feature vectors for the anchor users in the aligned networks.

  • DIME-SH: DIME-SH is a variant model of DIME proposed in this paper, which preserves both the local and global network structure with a set of meta proximity based on the heterogeneous networks. DIME-SH effectively fuses the heterogeneous information inside the network, where the fusion weight of information in different categories can be learnt automatically.

  • Auto-encoder Model: The Auto-Encoder model proposed in [2] can project the instances into a low-dimensional feature space. In the experiments, we build the Auto-Encoder model merely based on the friendship link among users, where the feature vector for each user is his/her social adjacency vector. Here, we also adjust the loss term for Auto-Encoder by weighting the non-zero features more with parameter as introduced in Section III-C2.

  • LINE Model: The LINE model is a scalable network embedding model proposed in [18], which optimizes an objective function that preserves both the local and global network structures. LINE also uses a edge-sampling algorithm to addresses the limitation of the classical stochastic gradient descent, which improves the inference effectiveness and the efficiency greatly.

  • DeepWalk Model: The DeepWalk model [14] extends the word2vec model [12] to the network embedding scenario. DeepWalk uses local information obtained from truncated random walks to learn latent representations.

Iv-B2 Link Prediction Experimental Setting

Given the emerging network, from which we can obtain all the existing links inside the network as the positive set. Meanwhile, from the network, a subset of the non-existing links are randomly sampled as the negative set according to the negative positive sampling ratio . Here denotes that the negative set is of the same size as the positive set. Meanwhile, represents the negative set is 9 times larger than the positive set. The positive and negative sets are divided into two subsets with 10-fold cross validation, where folds are used as the training set and fold is used as the testing set.

To denote different degrees of information sparsity, the emerging network is further sampled to remove information randomly from the network, which is controlled by the sampling ratio . Here, the sampling denotes removing the positive links in the training set, as well as the posts from the emerging network to make the network sparse. denotes of the information is preserved ( of the information is randomly removed); while denotes that all the information is preserved in the emerging network. The network embedding is learnt based on the training set and network after sampling.

With the learnt embedding feature vectors, the remaining links in the training set is used to build a supervised link prediction model. For each link in the training set, the embedding feature vector of the nodes and are concatenated as the link feature vector. Depending on whether link appears in the positive set or negative set, will be assigned with the or

label. SVM is used as the base classifier for all the embedding models. We train SVM with the training set, and then apply the trained SVM to the testing set to infer the labels and the formation probabilities of these links. By comparing the prediction labels (and inference probabilities) with the ground truth labels, we can evaluate the performance of the embedding models with different kinds of evaluation metrics to be introduced in the next subsection.

In the experiments, hidden layers are involved in framework DIME

(3 hidden layers in encoder step, 3 in decoder step, and 1 fusion hidden layer). The number neuron in these hidden layers are

, , , , , and