DeepAI AI Chat
Log In Sign Up

Graph Neural Networks as Gradient Flows

by   Francesco Di Giovanni, et al.

Dynamical systems minimizing an energy are ubiquitous in geometry and physics. We propose a gradient flow framework for GNNs where the equations follow the direction of steepest descent of a learnable energy. This approach allows to explain the GNN evolution from a multi-particle perspective as learning attractive and repulsive forces in feature space via the positive and negative eigenvalues of a symmetric "channel-mixing" matrix. We perform spectral analysis of the solutions and conclude that gradient flow graph convolutional models can induce a dynamics dominated by the graph high frequencies which is desirable for heterophilic datasets. We also describe structural constraints on common GNN architectures allowing to interpret them as gradient flows. We perform thorough ablation studies corroborating our theoretical analysis and show competitive performance of simple and lightweight models on real-world homophilic and heterophilic datasets.


page 1

page 2

page 3

page 4


Towards Understanding Graph Neural Networks: An Algorithm Unrolling Perspective

The graph neural network (GNN) has demonstrated its superior performance...

An eikonal equation approach to thermodynamics and the gradient flows in information geometry

We can consider the equations of states in thermodynamics as the general...

Triple Sparsification of Graph Convolutional Networks without Sacrificing the Accuracy

Graph Neural Networks (GNNs) are widely used to perform different machin...

Statistical Mechanics of Generalization In Graph Convolution Networks

Graph neural networks (GNN) have become the default machine learning mod...

Reversible and irreversible bracket-based dynamics for deep graph neural networks

Recent works have shown that physics-inspired architectures allow the tr...

Semi-Equivariant GNN Architectures for Jet Tagging

Composing Graph Neural Networks (GNNs) of operations that respect physic...