Decoupling the Depth and Scope of Graph Neural Networks

01/19/2022
by   Hanqing Zeng, et al.
0

State-of-the-art Graph Neural Networks (GNNs) have limited scalability with respect to the graph and model sizes. On large graphs, increasing the model depth often means exponential expansion of the scope (i.e., receptive field). Beyond just a few layers, two fundamental challenges emerge: 1. degraded expressivity due to oversmoothing, and 2. expensive computation due to neighborhood explosion. We propose a design principle to decouple the depth and scope of GNNs – to generate representation of a target entity (i.e., a node or an edge), we first extract a localized subgraph as the bounded-size scope, and then apply a GNN of arbitrary depth on top of the subgraph. A properly extracted subgraph consists of a small number of critical neighbors, while excluding irrelevant ones. The GNN, no matter how deep it is, smooths the local neighborhood into informative representation rather than oversmoothing the global graph into "white noise". Theoretically, decoupling improves the GNN expressive power from the perspectives of graph signal processing (GCN), function approximation (GraphSAGE) and topological learning (GIN). Empirically, on seven graphs (with up to 110M nodes) and six backbone GNN architectures, our design achieves significant accuracy improvement with orders of magnitude reduction in computation and hardware cost.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

12/02/2020

Deep Graph Neural Networks with Shallow Subgraph Samplers

While Graph Neural Networks (GNNs) are powerful models for learning repr...
10/27/2021

Node-wise Localization of Graph Neural Networks

Graph neural networks (GNNs) emerge as a powerful family of representati...
10/25/2021

Nested Graph Neural Networks

Graph neural network (GNN)'s success in graph classification is closely ...
06/09/2019

Redundancy-Free Computation Graphs for Graph Neural Networks

Graph Neural Networks (GNNs) are based on repeated aggregations of infor...
01/29/2022

Rewiring with Positional Encodings for Graph Neural Networks

Several recent works use positional encodings to extend the receptive fi...
08/23/2021

Graph Attention Multi-Layer Perceptron

Graph neural networks (GNNs) have recently achieved state-of-the-art per...
07/06/2019

What graph neural networks cannot learn: depth vs width

This paper studies the capacity limits of graph neural networks (GNN). R...

Code Repositories

shaDow_GNN

NeurIPS 2021: Improve the GNN expressivity and scalability by decoupling the depth and receptive field of state-of-the-art GNN architectures


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.