DeepAI AI Chat
Log In Sign Up

Graph Neural Networks as Gradient Flows

06/22/2022
by   Francesco Di Giovanni, et al.
Twitter
101

Dynamical systems minimizing an energy are ubiquitous in geometry and physics. We propose a gradient flow framework for GNNs where the equations follow the direction of steepest descent of a learnable energy. This approach allows to explain the GNN evolution from a multi-particle perspective as learning attractive and repulsive forces in feature space via the positive and negative eigenvalues of a symmetric "channel-mixing" matrix. We perform spectral analysis of the solutions and conclude that gradient flow graph convolutional models can induce a dynamics dominated by the graph high frequencies which is desirable for heterophilic datasets. We also describe structural constraints on common GNN architectures allowing to interpret them as gradient flows. We perform thorough ablation studies corroborating our theoretical analysis and show competitive performance of simple and lightweight models on real-world homophilic and heterophilic datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/09/2022

Towards Understanding Graph Neural Networks: An Algorithm Unrolling Perspective

The graph neural network (GNN) has demonstrated its superior performance...
01/10/2020

An eikonal equation approach to thermodynamics and the gradient flows in information geometry

We can consider the equations of states in thermodynamics as the general...
08/06/2022

Triple Sparsification of Graph Convolutional Networks without Sacrificing the Accuracy

Graph Neural Networks (GNNs) are widely used to perform different machin...
12/26/2022

Statistical Mechanics of Generalization In Graph Convolution Networks

Graph neural networks (GNN) have become the default machine learning mod...
05/24/2023

Reversible and irreversible bracket-based dynamics for deep graph neural networks

Recent works have shown that physics-inspired architectures allow the tr...
02/14/2022

Semi-Equivariant GNN Architectures for Jet Tagging

Composing Graph Neural Networks (GNNs) of operations that respect physic...