Graph Neural Networks as Gradient Flows

06/22/2022
by   Francesco Di Giovanni, et al.
101

Dynamical systems minimizing an energy are ubiquitous in geometry and physics. We propose a gradient flow framework for GNNs where the equations follow the direction of steepest descent of a learnable energy. This approach allows to explain the GNN evolution from a multi-particle perspective as learning attractive and repulsive forces in feature space via the positive and negative eigenvalues of a symmetric "channel-mixing" matrix. We perform spectral analysis of the solutions and conclude that gradient flow graph convolutional models can induce a dynamics dominated by the graph high frequencies which is desirable for heterophilic datasets. We also describe structural constraints on common GNN architectures allowing to interpret them as gradient flows. We perform thorough ablation studies corroborating our theoretical analysis and show competitive performance of simple and lightweight models on real-world homophilic and heterophilic datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/09/2022

Towards Understanding Graph Neural Networks: An Algorithm Unrolling Perspective

The graph neural network (GNN) has demonstrated its superior performance...
research
01/10/2020

An eikonal equation approach to thermodynamics and the gradient flows in information geometry

We can consider the equations of states in thermodynamics as the general...
research
08/06/2022

Triple Sparsification of Graph Convolutional Networks without Sacrificing the Accuracy

Graph Neural Networks (GNNs) are widely used to perform different machin...
research
12/26/2022

Statistical Mechanics of Generalization In Graph Convolution Networks

Graph neural networks (GNN) have become the default machine learning mod...
research
05/24/2023

Reversible and irreversible bracket-based dynamics for deep graph neural networks

Recent works have shown that physics-inspired architectures allow the tr...
research
02/14/2022

Semi-Equivariant GNN Architectures for Jet Tagging

Composing Graph Neural Networks (GNNs) of operations that respect physic...

Please sign up or login with your details

Forgot password? Click here to reset