DeepAI AI Chat
Log In Sign Up

Heterogeneous Graph Transformer

by   Ziniu Hu, et al.

Recent years have witnessed the emerging success of graph neural networks (GNNs) for modeling structured data. However, most GNNs are designed for homogeneous graphs, in which all nodes and edges belong to the same types, making them infeasible to represent heterogeneous structures. In this paper, we present the Heterogeneous Graph Transformer (HGT) architecture for modeling Web-scale heterogeneous graphs. To model heterogeneity, we design node- and edge-type dependent parameters to characterize the heterogeneous attention over each edge, empowering HGT to maintain dedicated representations for different types of nodes and edges. To handle dynamic heterogeneous graphs, we introduce the relative temporal encoding technique into HGT, which is able to capture the dynamic structural dependency with arbitrary durations. To handle Web-scale graph data, we design the heterogeneous mini-batch graph sampling algorithm—HGSampling—for efficient and scalable training. Extensive experiments on the Open Academic Graph of 179 million nodes and 2 billion edges show that the proposed HGT model consistently outperforms all the state-of-the-art GNN baselines by 9


page 1

page 2

page 3

page 4


Relation Embedding based Graph Neural Networks for Handling Heterogeneous Graph

Heterogeneous graph learning has drawn significant attentions in recent ...

Representing Social Networks as Dynamic Heterogeneous Graphs

Graph representations for real-world social networks in the past have mi...

PIGEON: Optimizing CUDA Code Generator for End-to-End Training and Inference of Relational Graph Neural Networks

Relational graph neural networks (RGNNs) are graph neural networks (GNNs...

Heterformer: A Transformer Architecture for Node Representation Learning on Heterogeneous Text-Rich Networks

We study node representation learning on heterogeneous text-rich network...

Heterogeneous Graph Learning for Acoustic Event Classification

Heterogeneous graphs provide a compact, efficient, and scalable way to m...

EXPERT: Public Benchmarks for Dynamic Heterogeneous Academic Graphs

Machine learning models that learn from dynamic graphs face nontrivial c...

Graph Transformer Networks: Learning Meta-path Graphs to Improve GNNs

Graph Neural Networks (GNNs) have been widely applied to various fields ...