DeepAI AI Chat
Log In Sign Up

Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion

by   Haotian Ju, et al.

Graph neural networks are widely used tools for graph prediction tasks. Motivated by their empirical performance, prior works have developed generalization bounds for graph neural networks, which scale with graph structures in terms of the maximum degree. In this paper, we present generalization bounds that instead scale with the largest singular value of the graph neural network's feature diffusion matrix. These bounds are numerically much smaller than prior bounds for real-world graphs. We also construct a lower bound of the generalization gap that matches our upper bound asymptotically. To achieve these results, we analyze a unified model that includes prior works' settings (i.e., convolutional and message-passing networks) and new settings (i.e., graph isomorphism networks). Our key idea is to measure the stability of graph neural networks against noise perturbations using Hessians. Empirically, we find that Hessian-based measurements correlate with the observed generalization gaps of graph neural networks accurately; Optimizing noise stability properties for fine-tuning pretrained graph neural networks also improves test performance on several graph-level classification tasks.


A PAC-Bayesian Approach to Generalization Bounds for Graph Neural Networks

In this paper, we derive generalization bounds for the two primary class...

On the Stability of Graph Convolutional Neural Networks under Edge Rewiring

Graph neural networks are experiencing a surge of popularity within the ...

Stability and Generalization Capabilities of Message Passing Graph Neural Networks

Message passing neural networks (MPNN) have seen a steep rise in popular...

Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees

We consider transfer learning approaches that fine-tune a pretrained dee...

A Graph Neural Network with Negative Message Passing for Graph Coloring

Graph neural networks have received increased attention over the past ye...

Adversarial Weight Perturbation Improves Generalization in Graph Neural Network

A lot of theoretical and empirical evidence shows that the flatter local...

Interpretable Stability Bounds for Spectral Graph Filters

Graph-structured data arise in a variety of real-world context ranging f...