DeepAI AI Chat
Log In Sign Up

Statistical Mechanics of Generalization In Graph Convolution Networks

12/26/2022
by   Cheng Shi, et al.
0

Graph neural networks (GNN) have become the default machine learning model for relational datasets, including protein interaction networks, biological neural networks, and scientific collaboration graphs. We use tools from statistical physics and random matrix theory to precisely characterize generalization in simple graph convolution networks on the contextual stochastic block model. The derived curves are phenomenologically rich: they explain the distinction between learning on homophilic and heterophilic graphs and they predict double descent whose existence in GNNs has been questioned by recent work. Our results are the first to accurately explain the behavior not only of a stylized graph learning model but also of complex GNNs on messy real-world datasets. To wit, we use our analytic insights about homophily and heterophily to improve performance of state-of-the-art graph neural networks on several heterophilic benchmarks by a simple addition of negative self-loop filters.

READ FULL TEXT

page 8

page 10

page 27

04/16/2022

Theory of Graph Neural Networks: Representation and Learning

Graph Neural Networks (GNNs), neural network architectures targeted to l...
01/03/2022

KerGNNs: Interpretable Graph Neural Networks with Graph Kernels

Graph kernels are historically the most widely-used technique for graph ...
02/08/2022

Simplified Graph Convolution with Heterophily

Graph convolutional networks (GCNs) (Kipf Welling, 2017) attempt to ...
02/03/2022

Learning Mechanically Driven Emergent Behavior with Message Passing Neural Networks

From designing architected materials to connecting mechanical behavior a...
06/22/2022

Graph Neural Networks as Gradient Flows

Dynamical systems minimizing an energy are ubiquitous in geometry and ph...
09/30/2022

GPNet: Simplifying Graph Neural Networks via Multi-channel Geometric Polynomials

Graph Neural Networks (GNNs) are a promising deep learning approach for ...
09/30/2020

The Role of Isomorphism Classes in Multi-Relational Datasets

Multi-interaction systems abound in nature, from colloidal suspensions t...