How does over-squashing affect the power of GNNs?

by   Francesco Di Giovanni, et al.

Graph Neural Networks (GNNs) are the state-of-the-art model for machine learning on graph-structured data. The most popular class of GNNs operate by exchanging information between adjacent nodes, and are known as Message Passing Neural Networks (MPNNs). Given their widespread use, understanding the expressive power of MPNNs is a key question. However, existing results typically consider settings with uninformative node features. In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity. We do so by measuring the level of pairwise interactions between nodes that MPNNs allow for. This measure provides a novel quantitative characterization of the so-called over-squashing effect, which is observed to occur when a large volume of messages is aggregated into fixed-size vectors. Using our measure, we prove that, to guarantee sufficient communication between pairs of nodes, the capacity of the MPNN must be large enough, depending on properties of the input graph structure, such as commute times. For many relevant scenarios, our analysis results in impossibility statements in practice, showing that over-squashing hinders the expressive power of MPNNs. We validate our theoretical findings through extensive controlled experiments and ablation studies.


page 1

page 2

page 3

page 4


Graph Neural Networks can Recover the Hidden Features Solely from the Graph Structure

Graph Neural Networks (GNNs) are popular models for graph learning probl...

A unifying point of view on expressive power of GNNs

Graph Neural Networks (GNNs) are a wide class of connectionist models fo...

Randomized Message-Interception Smoothing: Gray-box Certificates for Graph Neural Networks

Randomized smoothing is one of the most promising frameworks for certify...

What functions can Graph Neural Networks compute on random graphs? The role of Positional Encoding

We aim to deepen the theoretical understanding of Graph Neural Networks ...

Graph Joint Attention Networks

Graph attention networks (GATs) have been recognized as powerful tools f...

Distilling Self-Knowledge From Contrastive Links to Classify Graph Nodes Without Passing Messages

Nowadays, Graph Neural Networks (GNNs) following the Message Passing par...

Characterizing Graph Datasets for Node Classification: Beyond Homophily-Heterophily Dichotomy

Homophily is a graph property describing the tendency of edges to connec...

Please sign up or login with your details

Forgot password? Click here to reset