Segmented Graph-Bert for Graph Instance Modeling

by   Jiawei Zhang, et al.

In graph instance representation learning, both the diverse graph instance sizes and the graph node orderless property have been the major obstacles that render existing representation learning models fail to work. In this paper, we will examine the effectiveness of GRAPH-BERT on graph instance representation learning, which was designed for node representation learning tasks originally. To adapt GRAPH-BERT to the new problem settings, we re-design it with a segmented architecture instead, which is also named as SEG-BERT (Segmented GRAPH-BERT) for reference simplicity in this paper. SEG-BERT involves no node-order-variant inputs or functional components anymore, and it can handle the graph node orderless property naturally. What's more, SEG-BERT has a segmented architecture and introduces three different strategies to unify the graph instance sizes, i.e., full-input, padding/pruning and segment shifting, respectively. SEG-BERT is pre-trainable in an unsupervised manner, which can be further transferred to new tasks directly or with necessary fine-tuning. We have tested the effectiveness of SEG-BERT with experiments on seven graph instance benchmark datasets, and SEG-BERT can out-perform the comparison methods on six out of them with significant performance advantages.


G5: A Universal GRAPH-BERT for Graph-to-Graph Transfer and Apocalypse Learning

The recent GRAPH-BERT model introduces a new approach to learning graph ...

Graph-Bert: Only Attention is Needed for Learning Graph Representations

The dominant graph neural networks (GNNs) over-rely on the graph links, ...

MxPool: Multiplex Pooling for Hierarchical Graph Representation Learning

How to utilize deep learning methods for graph classification tasks has ...

Sensitivity Analysis on Transferred Neural Architectures of BERT and GPT-2 for Financial Sentiment Analysis

The explosion in novel NLP word embedding and deep learning techniques h...

SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning

Textual graphs (TGs) are graphs whose nodes correspond to text (sentence...

Are Graph Representation Learning Methods Robust to Graph Sparsity and Asymmetric Node Information?

The growing popularity of Graph Representation Learning (GRL) methods ha...

Code Repositories


A list of latest research papers on graph-bert

view repo


Source code of "Segmented GRAPH-BERT Model"

view repo

Please sign up or login with your details

Forgot password? Click here to reset