Graph Neural Networks Inspired by Classical Iterative Algorithms

by   Yongyi Yang, et al.

Despite the recent success of graph neural networks (GNN), common architectures often exhibit significant limitations, including sensitivity to oversmoothing, long-range dependencies, and spurious edges, e.g., as can occur as a result of graph heterophily or adversarial attacks. To at least partially address these issues within a simple transparent framework, we consider a new family of GNN layers designed to mimic and integrate the update rules of two classical iterative algorithms, namely, proximal gradient descent and iterative reweighted least squares (IRLS). The former defines an extensible base GNN architecture that is immune to oversmoothing while nonetheless capturing long-range dependencies by allowing arbitrary propagation steps. In contrast, the latter produces a novel attention mechanism that is explicitly anchored to an underlying end-toend energy function, contributing stability with respect to edge uncertainty. When combined we obtain an extremely simple yet robust model that we evaluate across disparate scenarios including standardized benchmarks, adversarially-perturbated graphs, graphs with heterophily, and graphs involving long-range dependencies. In doing so, we compare against SOTA GNN approaches that have been explicitly designed for the respective task, achieving competitive or superior node classification accuracy.


page 1

page 2

page 3

page 4


Improving the Long-Range Performance of Gated Graph Neural Networks

Many popular variants of graph neural networks (GNNs) that are capable o...

EIGNN: Efficient Infinite-Depth Graph Neural Networks

Graph neural networks (GNNs) are widely used for modelling graph-structu...

Implicit Graph Neural Networks

Graph Neural Networks (GNNs) are widely used deep learning models that l...

Representing Long-Range Context for Graph Neural Networks with Global Attention

Graph neural networks are powerful architectures for structured datasets...

Implicit vs Unfolded Graph Neural Networks

It has been observed that graph neural networks (GNN) sometimes struggle...

Stochastic Graph Neural Networks

Graph neural networks (GNNs) model nonlinear representations in graph da...

Capturing Graphs with Hypo-Elliptic Diffusions

Convolutional layers within graph neural networks operate by aggregating...