I Introduction
With the prosperity of online social networks, people nowadays are usually involved in different social networks simultaneously to enjoy more applications. This provides an opportunity and a challenge to combine heterogeneous social networks and to collectively mine them. More often than not, finding node correspondence across different networks is the very first step for many data mining task and a fundamental building block in numerous applications, such as, the social network link prediction [1, 2] and the crossdomain recommendation [3, 4]. The problem formulated thus far is typically referred to as social networks alignment. The social networks can virtually be model as graphs [5, 6, 7], so the focus of this paper can be stated as aligning two social graphs (also known as social networks). A naive approach for graph alignment is to make use of usernames to identify the same users across different social graphs. However, due to the fact that there exist many users who deliberately use different usernames, such an assumption could be easily violated in most realworld applications.
Most prior efforts, which focus on leveraging the graph structure (i.e. topology) for graph alignment, can fall into two main categories. The first category is the unsupervised approach, where no information about the node correspondence across graphs is available. These approaches deal with the graph alignment problem by finding the structural similarities between nodes across different graphs. For example, IsoRank [8] propagates pairwise topological similarities in the product graph. NetAlign [9] leverages the maxproduct belief propagation based on the graph topology. However, these approaches are limited to graphs with similar structure and fail to scale up to highly irregular and large graphs. The justification is that such structural similarity is hard to satisfy in some applications and may lead to suboptimal results. For instance, a user might be very active on one social graph site (e.g., Facebook), but behaves more quietly on another site (e.g., LinkedIn) [10, 11].
To alleviate the aforementioned limitations, the second category is the supervised approach, where the node correspondences between different graphs are partially available. Most of them [12, 13, 14, 2, 15] directly utilize the structural features of social graphs (e.g.,
users’ pointtopoint constraints, potential manifold alignment, followerfollowee relations) to train a classifier under the supervision of known node correspondences. Nevertheless, these approaches may struggle to capture the intrinsic structural information of social graphs and be sensitive to the slight changes or noises of graph structure. Hence, some recent studies
[11, 5] started to consider either leveraging vertex and edge attributes (e.g., demographic information of the users and the communication types between different users) to guide the global structurebased graph alignment or using an embedding approach to capture the structural regularities of social graphs [16], which can further compensate the lack of supervision information and achieve a more accurate alignment.Despite their efficacy, existing graph alignment methods may be confined by three key bottlenecks. First, the supervised approaches require massive amounts of crossgraph node correspondences and/or attribute information on nodes and edges, however, manual labeling of sufficient training data for diverse social applications onthefly is often prohibitive. Second, although the unsupervised approaches sound appealing, their performance is significantly worsen than that of supervised approaches. Third, although there are some methods consider first deriving embeddings for each graph and then learning a transformation between the embeddings, their results are often unsatisfying because their embeddings are predefined.
In this paper, we introduce a scalable Unsupervised Adversarial Graph Alignment (UAGA) framework that significantly outperforms stateoftheart unsupervised approaches and reaches or outperforms supervised baselines, without required any crossgraphs annotated data. We only use structure information from two different graphs, which are denoted as the source and target graphs respectively. The proposed framework consists of three steps, namely, embedding, matching and refining. First, it utilizes unsupervised graph embedding approach to learn the individual feature representations of source and target graphs respectively. Next, it leverages adversarial training to learn a linear mapping from the source embedding space to the target embedding space, followed by a refinement procedure which derives the pseudo node correspondences from the resulting shared embedding space and finetune the mapping with the closedform Procrustes solution from [17]. Finally, we extend the UAGA method to incremental UAGA (iUAGA) for improving both the embedding quality and the alignment performance iteratively.
Overall our paper makes the following contributions:

To the best of our knowledge, we are the first to approach graph alignment in a fully unsupervised manner, where neither anchor links nor node attribute information (e.g. user’s personal profiles) are demanded. Our UAGA, instead of aligning the nodes between two graphs directly, first learns the node representations of the source and target graphs and then aligns the corresponding embedding spaces using adversarial training in conjunction with an adhoc refinement procedure.

We highlight the hubness problem that a certain number of nodes become hubs and nearest neighbors of many other nodes in a highdimensional embedding space, and which, if not be addressed, will make the alignment by nearest searching more difficult and inaccurate. We hence propose a novel crossgraph similarity adaptation method that mitigates such issue and permits a remarkable performance improvement.

To improve the embedding quality, we further propose iUAGA which utilizes the pseudo anchor links obtained in the refinement procedure to extend the training graphs by iteratively revealing the unobserved user links and retraining our UAGA model with the extended training graphs.

Comprehensive experiments on realworld data clearly demonstrate the effectiveness of our proposed approaches for unsupervised graph alignment, compared to the stateoftheart unsupervised counterparts.
The remainder of the paper is organized as follows. We discuss related work in Section II and introduce preliminary concepts in Section III. Section IV introduces the proposed UAGA, including domainadversarial training, refinement procedure and orthogonality constraint. In Section V we extend the UAGA to incremental UAGA. Section VI provides the empirical studies on several standard graph alignment datasets and the analytical experiments will also be implemented. We conclude the paper in Section VII.
Ii Related Work
The graph alignment, which aims to find node correspondence across different graphs, has attracted lots of research interests with extensive literatures in the past decade. Graph alignment was first studied by exploring the graph structure to align the topology of the underlying graphs. For example, IsoRank [8] aligns different graphs greedily based on the pairwise topology similarities (i.e., node similarity) calculated with spectral graph theory and finds the alignment by solving the maximum bipartite matching. NetAlign [9] proposes a distributed and efficient algorithm for the sparse bipartite graph matching problem by using the maxproduct belief propagation based on the graph topology. UniAlign [10] proposes to distinguish the nodes of graphs by types (e.g. users and groups) and find the correspondences at different granularities at once for the fast bipartite or unipartite graph matching problem. Klau [18] formulates the pairwise global graph alignment problem as the graphbased maximum structural matching problem, which is solved by a Lagrangian relaxation approach. These approaches, however, ignore the rich attribute information on nodes and/or edges in many real graphs and are prone to lead to suboptimal results.
Recently, many attributedgraphbased approaches [19] have been proposed to tackle the graph alignment problem by leveraging the nodes’ attributes (e.g. user’s social, spatial, temporal and text information). [12]
aims to infer the anchor links between two attributed social graphs with a supervised learning model and explicitly ensure the pointtopoint constraint by the stable matching.
[15]propose to utilize the graph representation learning approach to learn the embeddings of multiple graphs, and the followerfollowee relations of each user are represented as the input and output context vectors in the embedded space to enhance the alignment performance.
[20]models users’ interests as latent topics and integrate relationships adaptively by a hierarchical Bayesian model and utilizes transfer learning to transfer both the topological and topical knowledge across subgraphs in a composite graph.
[13] is an users’ behaviordriven approach, which leverages the users’ unique behavioral patterns to reduce redundant information so as to construct the userspecific features for the classification of corresponding nodes. FINAL [11] holds the view that the topology consistency among different graphs be easily violated in some cases (e.g.the active level of the same user in different social graphs may be distinct). Therefore, they leverage the node/edge attribute information to guide the topologybased alignment process based on the alignment consistency principle. gsaNA
[5] proposes to align large and highly irregular graphs based on the observation that most prior efforts are limited since they can only handle small graphs (e.g. protein protein interaction graphs). Specifically, they reduce the search space of the problem by partitioning the vertices into buckets and present an iterative algorithm to incrementally map vertices of input graphs based on the similarity of the vertices in the same bucket of different graphs. In addition, [21] proposes to tackle the unsupervised multiple anonymized social graphs alignment problem by penalizing the violation to the the“transitivity law” and the “pointtopoint property”.Iii Preliminary
Iiia Task Formulation
We consider the problem of aligning two different social graphs. Denote a social graph as , where represents the user nodes of the graph, and the edge set denotes the connections. In this paper, we focus on the following settings: many users are simultaneously involved in two different social graphs, forming anchor links across the two graphs. Without loss of generality, we refer to one graph as the source graph, and the other as the target graph, denoted with and respectively. For each node in the source graph, we aim to identify, if any, its counterpart in the target graph. Formally, we can formulate the aforementioned problem as the following graph alignment problem,
Definition 1
(Graph Alignment Problem) Given two social graphs and and without any observed anchor links (node correspondences), it aims to identify hidden anchor links across and by graph alignment.
IiiB Unsupervised Graph Embedding
Recently, graph embedding models [22, 23, 24], for instance, DeepWalk [25], LINE [26] and Node2Vec [27] were developed to be capable of learning continuous feature representations for nodes in graphs. Specifically, DeepWalk is a unsupervised deep embedding method, which uses local information obtained from truncated random walks to learn features that capture the graph structure independent of the labels’ distribution. In this work, we utilizes DeepWalk as an example to learn the source and target graph embeddings, denoted with and respectively, where is a dimensional feature vector. These embedded feature representations are distributed, which means that each social node is represented by a subset of the latent dimensions. Using these structural feature representations, we are capable of finding the crossgraph node correspondences by learning a mapping function from the source to target embedded feature spaces.
DeepWalk, inspired by Word2Vec [28], borrows the methodology of Skipgrams [29] for training. The objective of Skipgrams is for each word
in a text corpus, the model is trained to maximize the probability of words
within a window of size given . In order to learn the feature representation that captures the shared similarities in local graph structure between nodes (i.e., encoding cocitation similarity), we have the following optimization problem,(1) 
where is the window size and is a mapping function: . Readers can refer to [25] for more implementation details.
Iv Unsupervised Adversarial Graph Alignment
The overview of proposed UAGA is shown in Figure 1. Firstly, we learn the graph feature representations with unsupervised graph embedding approach (i.e. DeepWalk). Secondly, we utilize the domainadversarial training to learn an initial proxy of . Next, we leverage the learned to select pseudo anchor links for mapping refinement. Finally, we use the refined mapping to map all source nodes into target latent feature space. We improve the performance over graph alignment by changing the metric of the space, which leads to spread more of those nodes in datadense areas.
Suppose we have two graph embedding spaces (denoted by the source and the target domains) that trained on two different social graphs. In this paper, we propose to learn a mapping, without requiring any form of crossdomain supervision, between them such that the two different embedding spaces can be well aligned.
Let and be two sets of and graph embeddings coming from the source and target embedding spaces, respectively. Ideally, if we have known crossgraph node correspondences that specifies which corresponds to which , we are able to learn a linear mapping of size between the source and the target space such that
(2) 
where is the dimension of the embeddings, and and are two aligned matrices of size formed by node embeddings selected from and , respectively. During the testing phase, the transformation result of any embeddings in the source domain can be defined as . Note that the Equation (2) can directly be used as portion of loss for supervised learning if we have groundtruth node correspondences.
We show how to learn this mapping without using any crossdomain supervision. The proposed UAGA framework consists of two steps: domainadversarial training for learning an initial proxy of , followed by a refinement procedure which utilizes the best matching nodes to create pseudo anchor links for applying Equation (2). We will describe the details of each of these steps as follows.
Iva DomainAdversarial Training
In this step, our ultimate goal is to make the mapped and indistinguishable. Firstly, we define a discriminator, which aims to discriminate between elements randomly sampled from and . The mapping , which can be seen as the generator, is trained to prevent the discriminator from making accurate predictions by making the embedding features and become indistinguishable. Consequently, the domainadversarial training procedure is a twoplayer game, where the first player is the discriminator trained to distinguish the source domain from the target domain, and the second player is the generator trained simultaneously to confuse the discriminator.
Given the mapping , the discriminator, parameterized by , is optimized by minimizing the following objective function:
(3) 
where is the probability that embedding comes from the source embedding space, vice versa. Given the discriminator, the mapping aims to fool the discriminator’s ability to precisely predict the original domain of the embeddings by minimizing the following objective function:
(4) 
To train our model, we follow the standard generative adversarial nets (GANs) [30] training procedure, which alternately and iteratively optimizes the discriminator and the mapping to respectively minimize and .
IvB The Hubness Problem and Refinement Procedure
The domain adversarial training step learns a mapping function that matches the global source and target embedding spaces, without considering the complex multimode structures underlying the data distributions. In other words, the finegrained pointtopoint constraint (i.e. no two edges share a common endpoint) between two sets of nodes was not explicitly enforced in our unsupervised scenario.
To tackle the aforementioned challenge, we propose a refinement procedure to enable finegrained pointtopoint graph alignment with the generated pseudo anchor links. The merits of the refinement procedure can be concluded in two aspects. First, we circumvent the potential mode collapse problem (which is a notorious problem in traditional GANs) in the domainadversarial training step by introducing additional supervision (pseudo anchor links) to enable pointtopoint alignment. Second, when selecting the pseudo anchor links, we mitigate the socalled hubness problem (i.e. points tending to be nearest neighbors of many points in highdimensional spaces) [31] in deciding mutual nearest neighbors by introducing a CrossGraph Similarity Scaling (CGSS) scheme.
Specifically, we use the learned in previous domainadversarial training step as an initial proxy and then build many pseudo anchor links that specifies which corresponds to which . To obtain highquality pseudo anchor links, we only retain nodes from and that are mutual nearest neighbors. However, the nearest neighbors are usually asymmetric: being a NN of does not imply that is a NN of . In the embedding feature spaces, this will lead to a phenomenon that is harmful to aligning node embeddings based on a nearest neighbor rule: some nodes in the embedding space, which we call hubs, are more likely to be the nearest neighbors of many other nodes, but the others (called antihubs) are not nearest neighbors of any node.
In our work, we propose the CGSS to mitigate the socalled hubness problem. We consider a bipartite neighborhood graph, where each node of given anchor links is connected to its nearest neighbors in the other graph. On this bipartite graph, the neighborhood associated with a mapped source node embedding is denoted by . All elements of are nodes from the target graph. Likewise the neighborhood associated with an embedding of the target graph is represented by . The mean similarity of a source embedding to its target neighborhood is denoted by,
where
denotes the cosine similarity. Similarly, the mean similarity of a target embedding
to its source neighborhood is represented by . We compute all source and target node embeddings with the efficient nearest neighbors algorithm [32]. Formally, we utilize these similarities to define a crossdomain similarity measure between the mapped source embedding and target embedding as,(5) 
The motivation of the CGSS scheme is intuitive: it increases the similarity associated with isolated node embeddings, while it decreases the ones of embeddings lying in dense areas. In other words, it encourages nodes which lie in the lowdensity regions to be selected as the pseudo anchor links. Formally, the mapping is further refined via Procrustes: we apply Equation (2) on these pseudo anchor links to refine .
IvC Orthogonality Constraint
In our work, we additional propose a simple update step to make the mapping matrix
stays close to an orthogonal matrix as training proceeds
[33, 34]. The orthogonal constraint has the following advantages: 1) preserves the individual characteristic and quality of source and target feature embeddings respectively; 2) stabilizes the learning process as training proceeds; and 3) preserves the dot product of vectors and their distances. Specifically, our model will be updated iteratively and alternatively by using the following update strategy on ,With the orthogonality constraint, the Equation (2
) can be boiled down to the Procrustes problem, which provides a closedform solution obtained from the singular value decomposition (SVD) of
,(6) 
V Incremental UAGA
To further effectively use the pseudo anchor links for learning more accurate mapping, we further propose an incremental UAGA (iUAGA) procedure, which progressively selects a set of pseudo anchor links and then leverage them to refine the mapping by using Equation (2). We select a pseudo anchor link if the following two conditions are satisfied. First, we require the crossdomain similarity (i.e. Equation (5)) should exceed the threshold parameter, which we set as 0.7 or 0.75 in the experiment. The second requirement is that the pairs of embedding features are mutually nearest neighbors of each other according to CGSS. We suppose that unless they are mutual nearest neighbors, the pairwise relationship is not reliable. These two requirements decrease the number of the pseudo anchor links, but improve its accuracy and the final alignment performance. The set of pseudo anchor links is denoted by .
Another challenging problem in graph alignment is that some existing edges are still unobserved, as they have neither been explicitly built nor been crawled [15]. To solve this problem, previous supervised approaches [16, 15] extend both the source and target graphs with the help of the observed anchor links and the structure of the other graph. In our unsupervised scenario, however, it is very difficult or even impossible to reveal these unobserved edges, which may easily result in unreliable representations when embedding graphs into the latent feature spaces. Motivated by this, our approach proposes to progressively compensate the lack of crossgraph supervision with the refinement procedure by pseudolabeling many reliable anchor links, which can help to reveal the unobserved edges (i.e. intragraph pseudo user links). The justification is that if two nodes are unconnected in one graph, but their counterparts (according to the pseudo anchor links) are linked in the other graph, it is feasible to add an edge between them in the present graph, as shown in Figure 2. By leveraging the pseudolabeled user links, we can further improve both the graph embedding quality and the final alignment performance. The reason behind it is that the extended graphs can provide richer structural information for graph embedding and mapping.
Formally, given two graphs and with pseudo anchor links , the extended graph of the source graph can be denoted as,
(7) 
Likewise we can extend the target graph into .
Time Complexity Analysis The entire procedure of training the iUAGA is shown in Algorithm 1. The graph embedding by leveraging the DeepWalk can be computed in [25]. Algorithm 1 iteratively and progressively generates pseudo anchor links and pseudo user links and linearly maps the source embedding features to the target space, and it can be computed in . Therefore, the overall time complexity of iUAGA is .
Vi Experiments
In this section, we extensively evaluate our approach and compare it with both supervised and unsupervised graph alignment baselines on realworld data. We also provide a detailed analysis of the proposed framework, empirically demonstrating the effect of our contributions. All the results in the following are reported as averages over five runs.
Via Experimental Configuration
ViA1 Datasets
To evaluate the performance thoroughly, we consider different types of realworld graphs with different scales to generate datasets. Table I shows the statistics of these realworld social graphs ^{1}^{1}1http://aminer.org/cosnet.
ViA2 Aligned Graph Generation
Given a graph , we attempt to generate two subgraphs, which denoted by and , for the following evaluations. For example, we are first given a graph (Last.fm) with 9997 nodes together with their 511490 following edges. Then, we randomly discard 5% edges of and treat it as the source subgraph , where nodes are all inherited from . Finally, we further discard another (different from ) 5% edges of and treat it as the target subgraph . With such a strategy, both and share the common edge ratio from , which reflects the overlap level. For the rest 10%, randomly discarded 5% belongs to and the other 5% belongs to , which reflects the difference level. Note that only edges can be different between and , every pair of node correspondence between and are linked as the groundtruth anchor links. In training phase, we have no access to any anchor links. Specifically, we select the most informative nodes, whose degree is higher than 3, as our from the original graph.
Graphs  Nodes  Edges 

Last.fm  9997  511490 
Flickr  10000  1684207 
MySpace  10000  104056 
ViA3 Comparison Methods
We compare the performance of the proposed UAGA and iUAGA algorithms against four unsupervised centralitybased
(centrality: computes node centrality measures for a graph) graph alignment algorithms: Degree, Closeness, Betweenness and Eigenvector
^{2}^{2}2http://snap.stanford.edu/snap/doc.html. For the proposed UAGA model, we consider several variants of it in the following experiments:
AdvNN uses domainadversarial training to learn the mapping and then leverages the roughly learned mapping to find anchor links according to Nearest Neighbors similarity measure (NN);

AdvCGSS replaces NN by the proposed CGSS scheme;

AdvRefineNN adds the refinement procedure to AdvNN;

AdvRefineCGSS further replaces the NN of AdvRefineNN by the CGSS scheme. By adding the incremental training procedure to AdvRefineCGSS, we can obtain the iUAGA model.
Method  Last.fm Last.fm  Flickr Flickr  MySpace MySpace  Avg  

P@1  P@5  P@10  P@1  P@5  P@10  P@1  P@5  P@10  P@1  P@5  P@10  
Supervised methods with crossgraph supervision  
ProcrustesNN  0.4479  0.6467  0.8534  0.4801  0.6554  0.8887  0.2144  0.4289  0.5479  0.3809  0.5770  0.7633 
ProcrustesCGSS  0.4836  0.6754  0.9211  0.5346  0.7056  0.9679  0.2880  0.5136  0.6067  0.4354  0.6316  0.8319 
Unsupervised methods without crossgraph supervision  
Degree  0.0167  0.0938  0.2043  0.0124  0.1603  0.2237  0.0266  0.1286  0.1471  0.0557  0.1276  0.1917 
Closeness  0.0094  0.0352  0.1178  0.0096  0.0844  0.1657  0.0137  0.1035  0.2035  0.0109  0.0744  0.1623 
Betweenness  0.0076  0.0245  0.1056  0.0084  0.0291  0.1045  0.0081  0.0410  0.1106  0.0080  0.0317  0.1069 
Eigenvector  0.0105  0.1291  0.1943  0.0101  0.0945  0.1847  0.0181  0.1125  0.2045  0.0129  0.1120  0.1945 
AdvNN  0.3368  0.5372  0.6234  0.3750  0.6092  0.6970  0.1414  0.2744  0.3466  0.2844  0.4736  0.5557 
AdvCGSS  0.4361  0.6429  0.7171  0.4625  0.6923  0.7773  0.1858  0.3301  0.3956  0.3615  0.5551  0.6300 
AdvRefineNN  0.4404  0.6400  0.7128  0.5205  0.7240  0.7909  0.1642  0.2906  0.3984  0.3750  0.5515  0.6340 
AdvRefineCGSS  0.5445  0.7286  0.7802  0.5990  0.7835  0.8448  0.2062  0.3407  0.4389  0.4499  0.6176  0.6880 
iUAGA  0.6473  0.8179  0.8712  0.6343  0.8106  0.8743  0.2073  0.3769  0.4815  0.4963  0.6685  0.7423 
We also compare the performance against two supervised variants of the proposed UAGA, including ProcrustesNN and ProcrustesCGSS, which utilize 30% groundtruth anchors links for Procrustes and the rest of anchor links are used for testing. For the sake of fair comparison, we will not utilize the groundtruth anchors links to extend the source and the target user links, which will introduce additional groundtruth user links to the source and target graphs. Note that the focus of this paper is unsupervised setting, so we do not compare with the stateoftheart supervised methods.
ViA4 Evaluation Metric and Scenarios
To quantitatively evaluate the proposed model, we consider as the performance metric. For , we use the learned mapping and the similarity measure (NN or CGSS) to output a list of candidate nodes (i.e. the top nodes in descending order according to the similarity values) for each node in the source graph, and then compute the alignment accuracy with respect to the groundtruth.
Based on the aforementioned realworld graphs, we follow the predefined graph generation approach (Section VIA2) to construct three alignment scenarios for evaluations.
S1. Last.fm Last.fm
Given a graph with 9997 nodes together with 511490 edges, we generate and respectively.
S2. Flickr Flickr
Given a graph with 10000 nodes, together with 1684207 edges, we generate and respectively.
S3. MySpace MySpace
Given a graph with 10000 nodes together with 511490 edges, we generate and respectively.
S4. Flickr Last.fm
We have a source graph (Last.fm) with 996 nodes, together with 1544 edges. We have a target graph (Flickr) with 1001 nodes, together with 1513 edges. The source and target graphs own 510 groundtruth anchor links (provided by the original datasets).
S5. Flickr Myspace
We have a source graph (Flickr) with 1000 nodes, together with 1623 edges. We have a target graph (MySpace) with 997 nodes, together with 1402 edges. The source and target graphs own 509 groundtruth anchor links (provided by the original datasets).
ViA5 Model Architectures and Training Details
The graph embeddings were trained using DeepWalk with Skipgrams by setting the window size to 5. The embedding dimension is set to 32, thus the mapping has size of
. For the adversarial training, the discriminator was a twolayer neural network of size
with LeakyReLU as the activation function. The input to the discriminator is corrupted with dropout noise with a rate of 0.1. We include a smoothing coefficient
in the discriminator predictions. Both the discriminator andwere trained by stochastic gradient descent (SGD) with a fixed learning rate of
and a decay of . The batch size was set to 1000 in each experiment.For the CGSS, we have a hyperparameter which defines the size of the neighborhood. We empirically found that the performance is insensitive to , and thus does not need crossvalidation. The results are barely change when = 5, 10 and 50. Consequently, was set to 10 in all experiments.
In our experiments, we used C++ implementations of Degree, Closeness, Betweenness, Eigenvector algorithms from their source codes ^{3}^{3}3https://github.com/snapstanford/snap. Experiments were carried out on machine that has Tesla P40 GPU, 24GB of memory, Intel(R) Xeon(R) CPU E52680 v4
2.40GHz 14core, 2TB disk space, running Ubuntu GNU/Linux. UAGA and iUAGA are implemented by PyTorch.
ViB Results and Discussion
ViB1 Graph Alignment Performance
We first compare the proposed model with topologybased baselines, without leveraging the node/edge attribute information to guide the alignment process. The results are presented in Table II, and we report for . The UAGA and iUAGA models are found to significantly and consistently outperform unsupervised comparison methods on all graph alignment scenarios given different experimental settings (). Moreover, iUAGA is the topperforming unsupervised model and significantly outperforms the unsupervised baselines. Specifically, iUAGA achieves 30% to 70% improvements in terms of the alignment accuracy over the existing unsupervised methods.
Meanwhile, the experimental results reveal several insightful observations.

Both the UAGA and iUAGA models substantially outperform previous unsupervised alignment approaches (e.g. Degree and Eigenvector), which clearly demonstrates the contributions of the proposed method in learning finegrained onetoone crossgraph alignment.

Both AdvCGSS and AdvRefineCGSS outperform AdvNN and AdvRefineNN. This validates that CGSS provides a strong and robust similarity measure for finding anchor links when generating the pseudo anchor links or executing the testing procedure. Moreover, it also empirically verifies that the CGSS is capable of alleviating the socalled hubness problem and provides reliable pseudo anchor links.

Looking at the differences between AdvRefineCGSS and AdvCGSS or AdvRefineNN and AdvNN, we can observe that the proposed refinement procedure provides a significant gain in performance over different scenarios. This indicates that it is beneficial to select pseudo anchor links with the guidance of crossgraph similarity measure.

iUAGA significantly outperforms AdvRefineCGSS (: 0.4499 0.4963, 0.6176 0.6685, 0.6880 0.7423), which demonstrates that it is beneficial to iteratively reveal the unobserved user links by the pseudo anchor links, which contributes to refine the roughly learned mapping by using the enlarged training dataset.

The proposed iUAGA either is on par, or outperforms supervised baselines, e.g., Avg: 0.4963 0.4354, 0.6685 0.6316 and 0.7423 0.8319, which further demonstrates that the proposed unsupervised method can even exceed supervised methods without requiring any crossdomain correspondence.
Method  Flickr Last.fm  Flickr MySpace  

P@10  P@20  P@30  P@10  P@20  P@30  
Supervised methods with crossgraph supervision  
ProcrustesNN  0.2764  0.3809  0.4930  0.2905  0.3671  0.5125 
ProcrustesCGSS  0.3227  0.4246  0.5581  0.3505  0.4271  0.5870 
Unsupervised methods without crossgraph supervision  
Degree  0.0364  0.0821  0.1149  0.0524  0.1073  0.1426 
Closeness  0.0265  0.0539  0.0944  0.0291  0.0753  0.1008 
Betweenness  0.0130  0.0452  0.0871  0.0159  0.0565  0.0863 
Eigenvector  0.0402  0.0977  0.1235  0.0642  0.1137  0.1472 
AdvNN  0.1673  0.2490  0.3222  0.1794  0.2602  0.3587 
AdvCGSS  0.2483  0.3508  0.4192  0.2595  0.3243  0.4109 
AdvRefineNN  0.2515  0.3673  0.4310  0.2746  0.3502  0.4371 
AdvRefineCGSS  0.3344  0.4684  0.5127  0.3209  0.4010  0.5059 
ViB2 Influence of
we evaluate the impact of the number of the crossgraph shared edges (the overlapped edge ratio is ranging from 0.8 to 0.92 and the number of vertices remains unchanged) on the alignment accuracy. The experimental results on and are demonstrated in Figure 3(a)3(b). It is noteworthy that the is not monotonically increased with the increasing of . On the one hand, the justification is that the source and target graphs are vulnerable to contain isomorphic nodes (i.e., nodes in different graphs share the same/analogous topology structure) when the overlapped edge ratio is large. On the other hand, the will decease when decreases to a certain small value since the source and target graphs would be much dissimilar in that case.
Algorithm  P2  P100  P500  P1000 

DeepWalkemb  1  0.597  0.305  0.197 
Common Neighbor  1  0.960  0.541  0.356 
iUAGAS  1  1  0.923  0.857 
ViC Case Studies
In this part, several case studies are implemented to evaluate the effectiveness of the proposed methods on realworld applications.
ViC1 Anchor Link Inference between Different Social Graphs
The results of anchor link inference between different social graphs are reported on Table III. In terms of the (for =10, 20, 30), our proposed approaches significantly outperforms the unsupervised baselines. For example, AdvRefineCGSS outperforms Degree by 0.298 in terms of on . In particular, we observe that AdvRefineCGSS shows comparable or even better performances on most cases compared with ProcrustesCGSS, without requiring any crossgraph annotated data.
ViC2 Link Prediction
We utilize () of
as the testing graph for the link prediction and want to evaluate the effectiveness of iUAGA in terms of improving the link prediction accuracy. We follow the same evaluation metric (
i.e. ) with [22]. We consider the following comparisons.
DeepWalkemb, which utilizes DeepWalk to learn the embedding of and then use the obtained representations to predict the unobserved user links;

Common Neighbor [35], which only uses the number of common neighbors to measure the similarity between nodes for user links prediction;

iUAGAS, which utilizes the iUAGA method to add pseudo user links to by leveraging knowledge from (its counterpart in ) and then use the extended to learn the graph embedding.
The results are shown in Table IV, we can see that iUAGAS significantly outperforms the other baselines, e.g. 0.541 0.923 () and 0.356 0.857 (). This confirms that iUAGA, which incrementally pseudolabels the unobserved user links with high accuracy, is capable of improving the embedding quality, thus resulting in a more accurate link prediction.
ViC3 Embedding Visualization
In this part, we utilize tSNE [36] to visualize the embedding features of both the source graph (randomly extract 100 nodes together with 1268 edges from of ) and the target graph (their counterparts in of ) before and after graph alignment by UAGA, respectively. The results are presented in Figure 4(a)4(b), where we can observe that the embedding features become much more domaininvariant after graph alignment. The justification is that the feature discrepancy is significantly reduced in the embedding space of our proposed UAGA.
Vii Conclusions
In this paper, we study the unsupervised graph alignment problem. We propose a UAGA framework, which learns the source and target embedding spaces in an unsupervised embedding manner, and then attempts to align the two spaces via adversarial training, followed by a refinement procedure. The proposed methods do not require any crossgraph annotated data. Furthermore, we extend our UAGA to iUAGA, which we iteratively reveal the unobserved user links based on the pseudo anchor links. Thus it can be used to further improve the embedding quality and the final alignment accuracy. Our proposed approaches have achieved better performance than several unsupervised methods, and received comparable or even better performances than supervised methods on realworld data, which clearly demonstrates the effectiveness of our newly proposed approaches for unsupervised graph alignment.
References
 [1] Y. Dong, J. Tang, S. Wu, J. Tian, N. V. Chawla, J. Rao, and H. Cao, “Link prediction and recommendation across heterogeneous social networks,” in 2012 IEEE 12th International Conference on Data Mining, pp. 181–190, IEEE, 2012.
 [2] J. Zhang, P. S. Yu, and Z.H. Zhou, “Metapath based multinetwork collective link prediction,” in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1286–1295, ACM, 2014.
 [3] L. Hu, J. Cao, G. Xu, L. Cao, Z. Gu, and C. Zhu, “Personalized recommendation via crossdomain triadic factorization,” in Proceedings of the 22nd international conference on World Wide Web, pp. 595–606, ACM, 2013.
 [4] C.Y. Li and S.D. Lin, “Matching users and items across domains to improve the recommendation quality,” in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 801–810, ACM, 2014.
 [5] A. Yaşar and Ü. V. Çatalyürek, “An iterative global structureassisted labeled network aligner,” in Proceedings of the 24nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2614–2623, ACM, 2018.
 [6] W. Huang, T. Zhang, Y. Rong, and J. Huang, “Adaptive sampling towards fast graph representation learning,” in Advances in Neural Information Processing Systems, pp. 4558–4567, 2018.
 [7] J. Li, Y. Rong, H. Cheng, H. Meng, W. Huang, and J. Huang, “Semisupervised graph classification: A hierarchical graph perspective,” 2019.
 [8] R. Singh, J. Xu, and B. Berger, “Global alignment of multiple protein interaction networks with application to functional orthology detection,” Proceedings of the National Academy of Sciences, 2008.
 [9] M. Bayati, M. Gerritsen, D. F. Gleich, A. Saberi, and Y. Wang, “Algorithms for large, sparse network alignment problems,” in Data Mining, 2009. ICDM’09. Ninth IEEE International Conference on, pp. 705–710, IEEE, 2009.
 [10] D. Koutra, H. Tong, and D. Lubensky, “Bigalign: Fast bipartite graph alignment,” in 2013 IEEE 13th International Conference on Data Mining, pp. 389–398, IEEE, 2013.
 [11] S. Zhang and H. Tong, “Final: Fast attributed network alignment,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1345–1354, ACM, 2016.
 [12] X. Kong, J. Zhang, and P. S. Yu, “Inferring anchor links across multiple heterogeneous social networks,” in Proceedings of the 22nd ACM international conference on Information & Knowledge Management, pp. 179–188, ACM, 2013.
 [13] R. Zafarani and H. Liu, “Connecting users across social media sites: a behavioralmodeling approach,” in Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 41–49, ACM, 2013.
 [14] S. Tan, Z. Guan, D. Cai, X. Qin, J. Bu, and C. Chen, “Mapping users across networks by manifold alignment on hypergraph.,” in AAAI, vol. 14, pp. 159–165, 2014.
 [15] L. Liu, W. K. Cheung, X. Li, and L. Liao, “Aligning users across social networks using network embedding.,” in IJCAI, pp. 1774–1780, 2016.
 [16] T. Man, H. Shen, S. Liu, X. Jin, and X. Cheng, “Predict anchor links across social networks via an embedding approach.,” in IJCAI, vol. 16, pp. 1823–1829, 2016.
 [17] P. H. Schönemann, “A generalized solution of the orthogonal procrustes problem,” Psychometrika, vol. 31, no. 1, pp. 1–10, 1966.
 [18] G. W. Klau, “A new graphbased method for pairwise global network alignment,” BMC bioinformatics, vol. 10, no. 1, p. S59, 2009.
 [19] S. Liu, S. Wang, F. Zhu, J. Zhang, and R. Krishnan, “Hydra: Largescale social identity linkage via heterogeneous behavior modeling,” in Proceedings of the 2014 ACM SIGMOD international conference on Management of data, pp. 51–62, ACM, 2014.
 [20] E. Zhong, W. Fan, J. Wang, L. Xiao, and Y. Li, “Comsoc: adaptive transfer of user behaviors over composite social network,” in Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 696–704, ACM, 2012.
 [21] J. Zhang and S. Y. Philip, “Multiple anonymized social networks alignment,” in Data Mining (ICDM), 2015 IEEE International Conference on, pp. 599–608, IEEE, 2015.
 [22] D. Wang, P. Cui, and W. Zhu, “Structural deep network embedding,” in Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1225–1234, ACM, 2016.
 [23] J. Qiu, Y. Dong, H. Ma, J. Li, K. Wang, and J. Tang, “Network embedding as matrix factorization: Unifying deepwalk, line, pte, and node2vec,” in Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp. 459–467, ACM, 2018.
 [24] S. M. Kazemi and D. Poole, “Simple embedding for link prediction in knowledge graphs,” arXiv preprint arXiv:1802.04868, 2018.
 [25] B. Perozzi, R. AlRfou, and S. Skiena, “Deepwalk: Online learning of social representations,” in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 701–710, ACM, 2014.
 [26] J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei, “Line: Largescale information network embedding,” in Proceedings of the 24th International Conference on World Wide Web, pp. 1067–1077, International World Wide Web Conferences Steering Committee, 2015.
 [27] A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 855–864, ACM, 2016.

[28]
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in
Advances in neural information processing systems, pp. 3111–3119, 2013.  [29] T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” arXiv preprint arXiv:1301.3781, 2013.
 [30] I. Goodfellow, J. PougetAbadie, M. Mirza, B. Xu, D. WardeFarley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, pp. 2672–2680, 2014.
 [31] G. Dinu, A. Lazaridou, and M. Baroni, “Improving zeroshot learning by mitigating the hubness problem,” arXiv preprint arXiv:1412.6568, 2014.
 [32] J. Johnson, M. Douze, and H. Jégou, “Billionscale similarity search with gpus,” arXiv preprint arXiv:1702.08734, 2017.
 [33] S. L. Smith, D. H. Turban, S. Hamblin, and N. Y. Hammerla, “Offline bilingual word vectors, orthogonal transformations and the inverted softmax,” arXiv preprint arXiv:1702.03859, 2017.

[34]
M. Cisse, P. Bojanowski, E. Grave, Y. Dauphin, and N. Usunier, “Parseval
networks: Improving robustness to adversarial examples,” in
Proceedings of the 34th International Conference on Machine LearningVolume 70
, pp. 854–863, JMLR. org, 2017.  [35] D. LibenNowell and J. Kleinberg, “The linkprediction problem for social networks,” Journal of the American society for information science and technology, vol. 58, no. 7, pp. 1019–1031, 2007.
 [36] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell, “Decaf: A deep convolutional activation feature for generic visual recognition,” in International conference on machine learning, pp. 647–655, 2014.
Comments
There are no comments yet.