On Size Generalization in Graph Neural Networks

10/17/2020
by   Gilad Yehudai, et al.
0

Graph neural networks (GNNs) can process graphs of different sizes but their capacity to generalize across sizes is still not well understood. Size generalization is key to numerous GNN applications, from solving combinatorial optimization problems to learning in molecular biology. In such problems, obtaining labels and training on large graphs can be prohibitively expensive, but training on smaller graphs is possible. This paper puts forward the size-generalization question and characterizes important aspects of that problem theoretically and empirically. We show that even for very simple tasks, GNNs do not naturally generalize to graphs of larger size. Instead, their generalization performance is closely related to the distribution of patterns of connectivity and features and how that distribution changes from small to large graphs. Specifically, we show that in many cases, there are GNNs that can perfectly solve a task on small graphs but generalize poorly to large graphs and that these GNNs are encountered in practice. We then formalize size generalization as a domain-adaption problem and describe two learning setups where size generalization can be improved. First, as a self-supervised learning problem (SSL) over the target domain of large graphs. Second, as a semi-supervised learning problem when few samples are available in the target domain. We demonstrate the efficacy of these solutions on a diverse set of benchmark graph datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/14/2022

Towards Better Generalization with Flexible Representation of Multi-Module Graph Neural Networks

Graph neural networks (GNNs) have become compelling models designed to p...
research
05/24/2023

Size Generalizability of Graph Neural Networks on Biological Data: Insights and Practices from the Spectral Perspective

We investigate the question of whether the knowledge learned by graph ne...
research
01/25/2023

Graph Neural Tangent Kernel: Convergence on Large Graphs

Graph neural networks (GNNs) achieve remarkable performance in graph mac...
research
07/16/2022

SizeShiftReg: a Regularization Method for Improving Size-Generalization in Graph Neural Networks

In the past few years, graph neural networks (GNNs) have become the de f...
research
10/20/2021

Distributionally Robust Semi-Supervised Learning Over Graphs

Semi-supervised learning (SSL) over graph-structured data emerges in man...
research
03/09/2023

On the Expressiveness and Generalization of Hypergraph Neural Networks

This extended abstract describes a framework for analyzing the expressiv...
research
06/17/2020

Self-supervised Learning on Graphs: Deep Insights and New Direction

The success of deep learning notoriously requires larger amounts of cost...

Please sign up or login with your details

Forgot password? Click here to reset