Heterogeneous Graph Convolutional Networks for Temporal Community Detection

09/23/2019 ∙ by Yaping Zheng, et al. ∙ Harbin Institute of Technology Tencent QQ Nanyang Technological University 0

The Graph Convolutional Networks (GCN) has demonstrated superior performance in representing graph data, especially homogeneous graphs. However, the real-world graph data is usually heterogeneous and evolves with time, e.g., Facebook and DBLP, which has seldom been studied. To cope with this issue, we propose a novel approach named temporal heterogeneous graph convolutional network (THGCN). THGCN first embeds both spatial information and node attribute information together. Then, it captures short-term evolutionary patterns from the aggregations of embedded graph signals through compression network. Meanwhile, the long-term evolutionary patterns of heterogeneous graph data are also modeled via a TCN temporal convolutional network. To the best of our knowledge, this is the first attempt to model temporal heterogeneous graph data with a focus on community discovery task.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

Introduction

The Graph Convolutional Network (GCN) has demonstrated superior performance in representing graph data, especially homogeneous graphs. However, there exist few work in modeling heterogeneous graph via GCN. The possible reason is that it is not a straightforward task to directly adapt the original GCN to represent heterogeneous graph data, widely seen in the real-world applications, containing various types of nodes and edges. Furthermore, such heterogeneous graph data continuously evolves which makes aforementioned research issue even more challenging.

In the literature, rigorous research effort has been made with flourished results. In [2]

, the spectral CNN was proposed which defined graph Fourier transform and proposed the corresponding convolution theorem customized for graph data. Consequently, some research work has been proposed which could locally convolute over the nearest neighbors within

(a small value) steps, and thus could dramatically reduce the overall computational cost [3]. In [6], the authors proposed wavelet neutral network (GWNN) which could further reduce the computational cost and preserve localized features very well. There also exist some pioneer attempts, i.e., GCN based approaches, to model temporal graph [7] as well as heterogeneous graph data [5]. To the best of our knowledge, this is the first attempt to model temporal and heterogeneous graph data (THG) simultaneously. Particularly, we choose temporal community detection as a specific instance of THG problem and propose a temporal heterogeneous graph convolutional network (THGCN) approach to address this research issue.

The Proposed Approach

Problem Formulation

Let denote a graph, and denote node set and edge set, respectively. and . According to [5], graph type is defined as , where is the set of node types and is the set of edge types. If , is a heterogeneous graph. Let denote heterogeneous communities to be detected at time . With respect to , we further define the adjacency matrix as well as the degree matrix as follows.

Definition 1.

Let denote adjacency matrix of , and with as its element entry, where denote numbers of node types. Note that is a matrix and if , and 0 otherwise.

Definition 2.

Given adjacency matrix of , the degree matrix of can be defined as where and Laplacian matrix .

Figure 1: The framework of the proposed THGCN. It consists of three components: (a) heterogeneous graph convolutional network component to embed neighbors’ spatial information; (b) compression network component to exchange short-term community evolutionary information; (c) temporal convolutional network component to keep long-term community evolutionary information.

The Proposed THGCN

As aforementioned, the proposed THGCN is to model temporal heterogeneous graph data with a focus on temporal community detection task. As is known, GCN could represent graph data well by convoluting node attributes of each node with its nearest neighbors at its spectral space. It is yet challenging to directly adapt GCN to model temporal heterogeneous graph. Thus in this paper, we first propose a heterogeneous version of GCN, i.e., HGCN, to model heterogeneous graph and then extend this approach to predict community from a family of detected communities denoted as . The framework of the proposed THGCN is depicted in Figure 1.

Heterogeneous GCN component

To model heterogeneous graph data, we first propose this HGCN component which convolutes attributes of each node node with its nearest neighbors. Base on [3], our convolutional operation can be defined as

where and is the weight matrix of the layer, is a weight matrix representing the importance of different types of nodes in .

Compression network component

After acquiring HGCN embedding representations, this core component is to capture the short-term evolutionary patterns of heterogeneous communities. However, by concatenating graph signals

at different steps, the feature map is a sparse and high-dimensional tensor matrix. To compress this matrix, we follow the Compressed International Network (CIN)

[4]

to compress tensor matrix into a vector, and the operation is defined as

where with its -th column denote the -th node of and is Hadamard product.

We will detail the operation of this component as follows. First, we perform Hadamard product on layers of two feature maps (at different time) and then the feature vector is mapped from 2D space to 3D space. Then, the generated high dimensional tensor will be compressed into vector by multiplying each layer with a weight matrix . is learnt during the model training process. At last, the compressed feature vectors are concatenated together with their weight.

Temporal convolutional network component

For the temporal community detection, a revised dilated convolution component, originally proposed in TCN [1], is proposed in our approach to preserve the long-term evolutionary patterns of communities, defined as

where denotes the dilated coefficient, denotes the embedding vector of the current node , and denotes the current node type. If the node of type

is connected to the target node, its neighbors embedding vectors should be considered at a certain probability

. This probability is empirically defined.

By considering neighbors embedding information, the

could be detected via certain community partition techniques. During the training stage, the detected communities could be supervised by the ground-truth labels and thus the loss function of this component can be written as

where is the predicted community, and is the ground-truth community.

Conclusion and Future Work

This paper proposed a novel approach, temporal heterogeneous graph convolutional network (THGCN), with the target of modeling temporal and heterogeneous graph data. This task is challenging and yet important. We are performing extensive experiments to show its efficacy and will report the results in the near future.

References

  • [1] S. Bai, J. Z. Kolter, and V. Koltun (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271. Cited by: Temporal convolutional network component.
  • [2] J. Bruna, W. Zaremba, A. Szlam, and Y. Lecun (2014) Spectral networks and locally connected networks on graphs. In ICLR, (English (US)). Cited by: Introduction.
  • [3] T. Kipf and M. Welling (2017) Semi-supervised classification with graph convolutional networks. In ICLR, Cited by: Introduction, Heterogeneous GCN component.
  • [4] J. Lian, X. Zhou, F. Zhang, Z. Chen, X. Xie, and G. Sun (2018) Xdeepfm: combining explicit and implicit feature interactions for recommender systems. In SIGKDD, pp. 1754–1763. Cited by: Compression network component.
  • [5] X. Wang, H. Ji, C. Shi, B. Wang, Y. Ye, P. Cui, and P. S. Yu (2019) Heterogeneous graph attention network. In WWW, pp. 2022–2032. External Links: ISBN 978-1-4503-6674-8, Link, Document Cited by: Introduction, Problem Formulation.
  • [6] B. Xu, H. Shen, Q. Cao, Y. Qiu, and X. Cheng (2019)

    Graph wavelet neural network

    .
    In ICLR, Cited by: Introduction.
  • [7] B. Yu, H. Yin, and Z. Zhu (2018)

    Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting

    .
    In IJCAI, pp. 3634–3640. Cited by: Introduction.