Deep Graph Neural Networks with Shallow Subgraph Samplers

12/02/2020
by   Hanqing Zeng, et al.
0

While Graph Neural Networks (GNNs) are powerful models for learning representations on graphs, most state-of-the-art models do not have significant accuracy gain beyond two to three layers. Deep GNNs fundamentally need to address: 1). expressivity challenge due to oversmoothing, and 2). computation challenge due to neighborhood explosion. We propose a simple "deep GNN, shallow sampler" design principle to improve both the GNN accuracy and efficiency – to generate representation of a target node, we use a deep GNN to pass messages only within a shallow, localized subgraph. A properly sampled subgraph may exclude irrelevant or even noisy nodes, and still preserve the critical neighbor features and graph structures. The deep GNN then smooths the informative local signals to enhance feature learning, rather than oversmoothing the global graph signals into just "white noise". We theoretically justify why the combination of deep GNNs with shallow samplers yields the best learning performance. We then propose various sampling algorithms and neural architecture extensions to achieve good empirical results. Experiments on five large graphs show that our models achieve significantly higher accuracy and efficiency, compared with state-of-the-art.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/19/2022

Decoupling the Depth and Scope of Graph Neural Networks

State-of-the-art Graph Neural Networks (GNNs) have limited scalability w...
research
10/06/2022

Geodesic Graph Neural Network for Efficient Graph Representation Learning

Recently, Graph Neural Networks (GNNs) have been applied to graph learni...
research
06/08/2020

Eigen-GNN: A Graph Structure Preserving Plug-in for GNNs

Graph Neural Networks (GNNs) are emerging machine learning models on gra...
research
09/22/2020

GraphCrop: Subgraph Cropping for Graph Classification

We present a new method to regularize graph neural networks (GNNs) for b...
research
06/14/2022

RoSGAS: Adaptive Social Bot Detection with Reinforced Self-Supervised GNN Architecture Search

Social bots are referred to as the automated accounts on social networks...
research
12/21/2022

A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks

A central challenge of building more powerful Graph Neural Networks (GNN...
research
05/17/2023

SIMGA: A Simple and Effective Heterophilous Graph Neural Network with Efficient Global Aggregation

Graph neural networks (GNNs) realize great success in graph learning but...

Please sign up or login with your details

Forgot password? Click here to reset