DeepAI
Log In Sign Up

Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec

10/09/2017
by   Jiezhong Qiu, et al.
0

Since the invention of word2vec, the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factorization framework with closed forms. Our analysis and proofs reveal that: (1) DeepWalk empirically produces a low-rank transformation of a network's normalized Laplacian matrix; (2) LINE, in theory, is a special case of DeepWalk when the size of vertices' context is set to one; (3) As an extension of LINE, PTE can be viewed as the joint factorization of multiple networks' Laplacians; (4) node2vec is factorizing a matrix related to the stationary distribution and transition probability tensor of a 2nd-order random walk. We further provide the theoretical connections between skip-gram based network embedding algorithms and the theory of graph Laplacian. Finally, we present the NetMF method as well as its approximation algorithm for computing network embedding. Our method offers significant improvements over DeepWalk and LINE for conventional network mining tasks. This work lays the theoretical foundation for skip-gram based network embedding methods, leading to a better understanding of latent network representation learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/11/2017

Generalized Neural Graph Embedding with Matrix Factorization

Recent advances in language modeling such as word2vec motivate a number ...
06/25/2020

Time-varying Graph Representation Learning via Higher-Order Skip-Gram with Negative Sampling

Representation learning models for graphs are a successful family of tec...
05/19/2021

Free Energy Node Embedding via Generalized Skip-gram with Negative Sampling

A widely established set of unsupervised node embedding methods can be i...
04/26/2017

Riemannian Optimization for Skip-Gram Negative Sampling

Skip-Gram Negative Sampling (SGNS) word embedding model, well known by i...
06/26/2019

NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization

We study the problem of large-scale network embedding, which aims to lea...
02/28/2019

Global Vectors for Node Representations

Most network embedding algorithms consist in measuring co-occurrences of...
01/04/2023

Solving The Ordinary Least Squares in Closed Form, Without Inversion or Normalization

By connecting the LU factorization and the Gram-Schmidt orthogonalizatio...