DynamicGEM: A Library for Dynamic Graph Embedding Methods

DynamicGEM is an open-source Python library for learning node representations of dynamic graphs. It consists of state-of-the-art algorithms for defining embeddings of nodes whose connections evolve over time. The library also contains the evaluation framework for four downstream tasks on the network: graph reconstruction, static and temporal link prediction, node classification, and temporal visualization. We have implemented various metrics to evaluate the state-of-the-art methods, and examples of evolving networks from various domains. We have easy-to-use functions to call and evaluate the methods and have extensive usage documentation. Furthermore, DynamicGEM provides a template to add new algorithms with ease to facilitate further research on the topic.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

02/19/2020

Inductive Representation Learning on Temporal Graphs

Inductive representation learning on temporal graphs is an important ste...
11/23/2021

Learning Dynamic Preference Structure Embedding From Temporal Networks

The dynamics of temporal networks lie in the continuous interactions bet...
03/21/2019

Node Embedding over Temporal Graphs

In this work, we present a method for node embedding in temporal graphs....
06/30/2020

Online Dynamic Network Embedding

Network embedding is a very important method for network data. However, ...
06/08/2020

Little Ball of Fur: A Python Library for Graph Sampling

Sampling graphs is an important task in data mining. In this paper, we d...
10/13/2021

libdlr: Efficient imaginary time calculations using the discrete Lehmann representation

We introduce libdlr, a library implementing the recently introduced disc...
09/16/2020

Captum: A unified and generic model interpretability library for PyTorch

In this paper we introduce a novel, unified, open-source model interpret...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Graph embedding methods aim to represent each node of a graph in a low-dimensional vector space while preserving certain graph’s properties

(goyal2018graph). Such methods have been used to tackle many real-world tasks, e.g., friend recommendation in social networks, genome classification in biology networks, and visualizing topics in research using collaboration networks.

More recently, much attention has been devoted to extending static embedding techniques to capture graph evolution. Applications include temporal link prediction, and understanding the evolution dynamics of network communities. Most methods aim to efficiently update the embedding of the graph at each time step using information from previous embedding and from changes in the graph. Some methods also capture the temporal patterns of the evolution in the learned embedding, leading to improved link prediction performance.

In this paper, we present DynamicGEM, an easy-to-use toolkit of state-of-the-art dynamic graph embedding methods. In contrast to existing popular graph embedding libraries like GEM (goyal3gem)

, which implement static graph embedding methods, DynamicGEM implements methods which can handle the evolution of networks over time. Further, we provide a comprehensive framework to evaluate the methods by providing support for four tasks on dynamic networks: graph reconstruction, static and temporal link prediction, node classification, and temporal visualization. For each task, our framework includes multiple evaluation metrics to quantify the performance of the methods. We further share synthetic and real networks for evaluation. Thus, our library is an end-to-end framework to experiment with dynamic graph embedding. The software is available at

https://github.com/palash1992/DynamicGEM. We provide extensive documentation for installation and testing of various methods available in the library. The documentation also contains an easy-to-use guide for quickly adding new methods to the framework.

2 Dynamic Graph Embedding Algorithms

Dynamic graph embedding algorithms aim to capture the dynamics of the network and its evolution. These methods are useful to predict the future behavior of the network, such as future connections within a network. The problem can be defined formally as follows.

Consider a weighted graph , with and as the set of vertices and edges respectively. Given an evolution of graph , where represents the state of graph at time , a dynamic graph embedding method aims to represent each node in a series of low-dimensional vector space by learning mappings and . The methods differ in the definition of and the properties of the network preserved by .

There are various existing state of the art methods trying to solve this problem that we have incorporated and included them in this python package including:

  1. [noitemsep,topsep=0pt,parsep=0pt,partopsep=0pt,leftmargin=*]

  2. Optimal SVD

    : This method decomposes adjacency matrix of the graph at each time step using Singular Value Decomposition (SVD) to represent each node using the

    largest singular values (ou2016asymmetric).

  3. Incremental SVD: This method utilizes a perturbation matrix capturing the dynamics of the graphs along with performing additive modification on the SVD (BRAND200620).

  4. Rerun SVD: This method uses incremental SVD to create the dynamic graph embedding. In addition to that, it uses a tolerance threshold to restart the optimal SVD calculations and avoid deviation in incremental graph embedding (zhang2017timers).

  5. Dynamic TRIAD: This method utilizes the triadic closure process to generate a graph embedding that preserves structural and evolution patterns of the graph (zhou2018dynamic).

  6. AEalign: This method uses deep auto-encoder to embed each node in the graph and aligns the embeddings at different time steps using a rotation matrix (goyal2018dyngem).

  7. dynGEM: This method utilizes deep auto-encoders to incrementally generate embedding of a dynamic graph at snapshot  (goyal2018dyngem).

  8. dyngraph2vecAE: This method models the interconnection of nodes within and across time using multiple fully connected layers (goyal2018dyngraph2vec)

  9. dyngraph2vecRNN

    : This method uses sparsely connected Long Short Term Memory (LSTM) networks to learn the embedding

    (goyal2018dyngraph2vec)

  10. dyngraph2vecAERNN

    : This method uses a fully connected encoder to initially acquire low dimensional hidden representation and feeds this representation into LSTMs to capture network dynamics 

    (goyal2018dyngraph2vec).

3 Software Package

DynamicGEM implements the state-of-the-art dynamic graph embedding methods in Python and MATLAB and provides a Python interface for all the methods. The library takes as input a series of dynamic graphs as list of Networkx Digraph each corresponding to a single time step. In this section, we discuss details of library usage.

3.1 Usage

Next, we demonstrate the visualization of the graph embedding performed by dyngraph2vecRNN on stochastic block model dataset.

from time import time
#import helper libraries
from dynamicgem.visualization import plot_dynamic_sbm_embedding as pltdyn
from dynamicgem.graph_generation import dynamic_SBM_graph as sbm
#import the methods
from dynamicgem.embedding.dynRNN       import DynRNN
# Parameters for Stochastic block model graph
node_num      = 1000 #1000 total nodes
community_num = 2 # two communities
node_change_num = 10 #migrate 10 nodes in each time steps
length          = 4 # total time steps
#Generate the dynamic graph
dynamic_sbm_series = list(sbm.get_community_diminish_series_v2(node_num,
community_num, length,  1, node_change_num))
graphs     = [g[0] for g in dynamic_sbm_series]
#Initialize embedding algorithm
embedding= DynRNN(d=128, beta=5, n_prev_graphs=2, nu1=1e-6, nu2=1e-6,
n_enc_units=[500,300], n_dec_units=[500,300], rho=0.3,n_iter=250, xeta=1e-3,
n_batch=100, modelfile= [’./intm/enc_model.json’, ’./intm/dec_model.json’],
weightfile=[’./intm/enc_wghts.hdf5’, ’./intm/dec_weghts.hdf5’],
savefilesuffix = ”testing”  )
embs = []
for temp_var in range(lookback+1, length+1):
                emb, _ = embedding.learn_embeddings(graphs[:temp_var])
                embs.append(emb)
pltdyn.plot_dynamic_sbm_embedding_v2(embs[-5:-1], dynamic_sbm_series[-5:])
Figure 1: Visualization of the embedding generated by dyngraph2vecRNN after performing dimensionality reduction using TSNE on stochastic block model diminishing community dataset. The red colored nodes migrate from yellow to blue community in next iteration, however based on the temporal learning, dyngraph2vecRNN is able embed them closer to the blue community.

3.2 Documentation and Design

DynamicGEM package contains README files in DynamicGEM and its sub directories including dynamicgem/dynamictriad, and dynamicgem/graph-generation directories containing explanation about the repository, its structure, setup, implemented methods, usage, dependencies, and other useful information for user guidance. The repository is organized in an easy to navigate manner. The subdirectories are organized based on the functionality which they serve as follows:

  • [noitemsep,topsep=0pt,parsep=0pt,partopsep=0pt,leftmargin=*]

  • DynamicGEM/embedding: It contains implementation of the algorithms listed in section 3.2. In addition to dynamic graph embedding algorithms, this sub directory contains implementation for some static graph embedding methods on which the dynamic methods are built.

  • DynamicGEM/evaluation: It contains implementations of graph reconstruction, static and temporal link prediction and visualization for evaluation purposes.

  • DynamicGEM/utils: It contains implementation of utility functions for data preparation, plotting, embedding formatting, evaluation, and a variety of other functions that are building blocks of other functions.

  • DynamicGEM/graph-generation: It consists of functions to generate dynamic Stochastic Block Models (SBM) (wang1987stochastic) with diminishing community.

  • DynamicGEM/visualization: It contains functions for visualizing dynamic and static graph embeddings.

  • DynamicGEM/experiments: It contains useful hyper-parameter tuning function implementations.

  • DynamicGEM/TIMERS: It contains the TIMERS source code.

  • DynamicGEM/dynamicTriad: It contains the dynamicTriad source code.

4 Conclusion

DynamicGEM is a Python library with an extensive documentation that includes implementations for a variety of state-of-the-art dynamic graph embedding methods. This library can help researchers and developers to perform a wide range of experiments related to dynamic graphs that evolve over time, their different characteristics, and prediction of these characteristics that can help improvements in this field of study. The library allows easy integration of novel approaches in the domain and functions to evaluate their efficacy. The aim of the library is to evaluate such methods with greater ease.

The authors are grateful to the Defense Advanced Research Projects Agency (DARPA), contract W911NF-17-C-0094, for their support. We would also like to acknowledge support from the National Science Foundation (NSF grant IIS-9988642) and the Multidisciplinary Research Program of the Department of Defense (MURI N00014-00-1-0637).


References