DeepAI AI Chat
Log In Sign Up

Statistical Mechanics of Generalization In Graph Convolution Networks

by   Cheng Shi, et al.

Graph neural networks (GNN) have become the default machine learning model for relational datasets, including protein interaction networks, biological neural networks, and scientific collaboration graphs. We use tools from statistical physics and random matrix theory to precisely characterize generalization in simple graph convolution networks on the contextual stochastic block model. The derived curves are phenomenologically rich: they explain the distinction between learning on homophilic and heterophilic graphs and they predict double descent whose existence in GNNs has been questioned by recent work. Our results are the first to accurately explain the behavior not only of a stylized graph learning model but also of complex GNNs on messy real-world datasets. To wit, we use our analytic insights about homophily and heterophily to improve performance of state-of-the-art graph neural networks on several heterophilic benchmarks by a simple addition of negative self-loop filters.


page 8

page 10

page 27


Theory of Graph Neural Networks: Representation and Learning

Graph Neural Networks (GNNs), neural network architectures targeted to l...

KerGNNs: Interpretable Graph Neural Networks with Graph Kernels

Graph kernels are historically the most widely-used technique for graph ...

Simplified Graph Convolution with Heterophily

Graph convolutional networks (GCNs) (Kipf Welling, 2017) attempt to ...

Learning Mechanically Driven Emergent Behavior with Message Passing Neural Networks

From designing architected materials to connecting mechanical behavior a...

Graph Neural Networks as Gradient Flows

Dynamical systems minimizing an energy are ubiquitous in geometry and ph...

GPNet: Simplifying Graph Neural Networks via Multi-channel Geometric Polynomials

Graph Neural Networks (GNNs) are a promising deep learning approach for ...

The Role of Isomorphism Classes in Multi-Relational Datasets

Multi-interaction systems abound in nature, from colloidal suspensions t...