Design Space for Graph Neural Networks

by   Jiaxuan You, et al.

The rapid evolution of Graph Neural Networks (GNNs) has led to a growing number of new architectures as well as novel applications. However, current research focuses on proposing and evaluating specific architectural designs of GNNs, as opposed to studying the more general design space of GNNs that consists of a Cartesian product of different design dimensions, such as the number of layers or the type of the aggregation function. Additionally, GNN designs are often specialized to a single task, yet few efforts have been made to understand how to quickly find the best GNN design for a novel task or a novel dataset. Here we define and systematically study the architectural design space for GNNs which consists of 315,000 different designs over 32 different predictive tasks. Our approach features three key innovations: (1) A general GNN design space; (2) a GNN task space with a similarity metric, so that for a given novel task/dataset, we can quickly identify/transfer the best performing architecture; (3) an efficient and effective design space evaluation method which allows insights to be distilled from a huge number of model-task combinations. Our key results include: (1) A comprehensive set of guidelines for designing well-performing GNNs; (2) while best GNN designs for different tasks vary significantly, the GNN task space allows for transferring the best designs across different tasks; (3) models discovered using our design space achieve state-of-the-art performance. Overall, our work offers a principled and scalable approach to transition from studying individual GNN designs for specific tasks, to systematically studying the GNN design space and the task space. Finally, we release GraphGym, a powerful platform for exploring different GNN designs and tasks. GraphGym features modularized GNN implementation, standardized GNN evaluation, and reproducible and scalable experiment management.


page 17

page 18

page 19


A Unified View on Graph Neural Networks as Graph Signal Denoising

Graph Neural Networks (GNNs) have risen to prominence in learning repres...

AutoTransfer: AutoML with Knowledge Transfer – An Application to Graph Neural Networks

AutoML has demonstrated remarkable success in finding an effective neura...

Improving Robustness of Graph Neural Networks with Heterophily-Inspired Designs

Recent studies have exposed that many graph neural networks (GNNs) are s...

BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networks

Mapping the connectome of the human brain using structural or functional...

gSuite: A Flexible and Framework Independent Benchmark Suite for Graph Neural Network Inference on GPUs

As the interest to Graph Neural Networks (GNNs) is growing, the importan...

Efficient Automatic Machine Learning via Design Graphs

Despite the success of automated machine learning (AutoML), which aims t...

Are Graph Neural Networks Really Helpful for Knowledge Graph Completion?

Knowledge graphs (KGs) facilitate a wide variety of applications due to ...

Please sign up or login with your details

Forgot password? Click here to reset