Self-Supervised Learning of Contextual Embeddings for Link Prediction in Heterogeneous Networks

07/22/2020
by   Ping Wang, et al.
0

Representation learning methods for heterogeneous networks produce a low-dimensional vector embedding for each node that is typically fixed for all tasks involving the node. Many of the existing methods focus on obtaining a static vector representation for a node in a way that is agnostic to the downstream application where it is being used. In practice, however, downstream tasks require specific contextual information that can be extracted from the subgraphs related to the nodes provided as input to the task. To tackle this challenge, we develop SLiCE, a framework bridging static representation learning methods using global information from the entire graph with localized attention driven mechanisms to learn contextual node representations. We first pre-train our model in a self-supervised manner by introducing higher-order semantic associations and masking nodes, and then fine-tune our model for a specific link prediction task. Instead of training node representations by aggregating information from all semantic neighbors connected via metapaths, we automatically learn the composition of different metapaths that characterize the context for a specific task without the need for any pre-defined metapaths. SLiCE significantly outperforms both static and contextual embedding learning methods on several publicly available benchmark network datasets. We also interpret the semantic association matrix and provide its utility and relevance in making successful link predictions between heterogeneous nodes in the network.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/03/2020

Self-Supervised Graph Representation Learning via Global Context Prediction

To take full advantage of fast-growing unlabeled networked data, this pa...
research
03/15/2023

NESS: Learning Node Embeddings from Static SubGraphs

We present a framework for learning Node Embeddings from Static Subgraph...
research
09/07/2021

HMSG: Heterogeneous Graph Neural Network based on Metapath Subgraph Learning

Many real-world data can be represented as heterogeneous graphs with dif...
research
02/07/2022

Graph Self-supervised Learning with Accurate Discrepancy Learning

Self-supervised learning of graph neural networks (GNNs) aims to learn a...
research
10/30/2022

DyG2Vec: Representation Learning for Dynamic Graphs with Self-Supervision

The challenge in learning from dynamic graphs for predictive tasks lies ...
research
08/30/2022

Associative Learning for Network Embedding

The network embedding task is to represent the node in the network as a ...
research
10/29/2020

GripNet: Graph Information Propagation on Supergraph for Heterogeneous Graphs

Heterogeneous graph representation learning aims to learn low-dimensiona...

Please sign up or login with your details

Forgot password? Click here to reset