Graph Neural Networks Inspired by Classical Iterative Algorithms

03/10/2021
by   Yongyi Yang, et al.
0

Despite the recent success of graph neural networks (GNN), common architectures often exhibit significant limitations, including sensitivity to oversmoothing, long-range dependencies, and spurious edges, e.g., as can occur as a result of graph heterophily or adversarial attacks. To at least partially address these issues within a simple transparent framework, we consider a new family of GNN layers designed to mimic and integrate the update rules of two classical iterative algorithms, namely, proximal gradient descent and iterative reweighted least squares (IRLS). The former defines an extensible base GNN architecture that is immune to oversmoothing while nonetheless capturing long-range dependencies by allowing arbitrary propagation steps. In contrast, the latter produces a novel attention mechanism that is explicitly anchored to an underlying end-toend energy function, contributing stability with respect to edge uncertainty. When combined we obtain an extremely simple yet robust model that we evaluate across disparate scenarios including standardized benchmarks, adversarially-perturbated graphs, graphs with heterophily, and graphs involving long-range dependencies. In doing so, we compare against SOTA GNN approaches that have been explicitly designed for the respective task, achieving competitive or superior node classification accuracy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/19/2020

Improving the Long-Range Performance of Gated Graph Neural Networks

Many popular variants of graph neural networks (GNNs) that are capable o...
research
02/22/2022

EIGNN: Efficient Infinite-Depth Graph Neural Networks

Graph neural networks (GNNs) are widely used for modelling graph-structu...
research
09/14/2020

Implicit Graph Neural Networks

Graph Neural Networks (GNNs) are widely used deep learning models that l...
research
06/22/2022

Descent Steps of a Relation-Aware Energy Produce Heterogeneous Graph Neural Networks

Heterogeneous graph neural networks (GNNs) achieve strong performance on...
research
01/21/2022

Representing Long-Range Context for Graph Neural Networks with Global Attention

Graph neural networks are powerful architectures for structured datasets...
research
11/12/2021

Implicit vs Unfolded Graph Neural Networks

It has been observed that graph neural networks (GNN) sometimes struggle...
research
06/29/2022

Optimization-Induced Graph Implicit Nonlinear Diffusion

Due to the over-smoothing issue, most existing graph neural networks can...

Please sign up or login with your details

Forgot password? Click here to reset