GPT-GNN: Generative Pre-Training of Graph Neural Networks

06/27/2020
by   Ziniu Hu, et al.
43

Graph neural networks (GNNs) have been demonstrated to be powerful in modeling graph-structured data. However, training GNNs usually requires abundant task-specific labeled data, which is often arduously expensive to obtain. One effective way to reduce the labeling effort is to pre-train an expressive GNN model on unlabeled data with self-supervision and then transfer the learned model to downstream tasks with only a few labels. In this paper, we present the GPT-GNN framework to initialize GNNs by generative pre-training. GPT-GNN introduces a self-supervised attributed graph generation task to pre-train a GNN so that it can capture the structural and semantic properties of the graph. We factorize the likelihood of the graph generation into two components: 1) Attribute Generation and 2) Edge Generation. By modeling both components, GPT-GNN captures the inherent dependency between node attributes and graph structure during the generative process. Comprehensive experiments on the billion-scale Open Academic Graph and Amazon recommendation data demonstrate that GPT-GNN significantly outperforms state-of-the-art GNN models without pre-training by up to 9.1

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/03/2022

Neural Graph Matching for Pre-training Graph Neural Networks

Recently, graph neural networks (GNNs) have been shown powerful capacity...
research
05/31/2019

Pre-Training Graph Neural Networks for Generic Structural Feature Extraction

Graph neural networks (GNNs) are shown to be successful in modeling appl...
research
09/15/2022

DiP-GNN: Discriminative Pre-Training of Graph Neural Networks

Graph neural network (GNN) pre-training methods have been proposed to en...
research
02/09/2021

SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks

Graph neural networks (GNNs) work well when the graph structure is provi...
research
02/16/2022

Task-Agnostic Graph Explanations

Graph Neural Networks (GNNs) have emerged as powerful tools to encode gr...
research
11/16/2018

Pre-training Graph Neural Networks with Kernels

Many machine learning techniques have been proposed in the last few year...
research
02/24/2021

Pre-Training on Dynamic Graph Neural Networks

The pre-training on the graph neural network model can learn the general...

Please sign up or login with your details

Forgot password? Click here to reset