Implicit Graph Neural Diffusion Based on Constrained Dirichlet Energy Minimization

by   Guoji Fu, et al.

Implicit graph neural networks (GNNs) have emerged as a potential approach to enable GNNs to capture long-range dependencies effectively. However, poorly designed implicit GNN layers can experience over-smoothing or may have limited adaptability to learn data geometry, potentially hindering their performance in graph learning problems. To address these issues, we introduce a geometric framework to design implicit graph diffusion layers based on a parameterized graph Laplacian operator. Our framework allows learning the geometry of vertex and edge spaces, as well as the graph gradient operator from data. We further show how implicit GNN layers can be viewed as the fixed-point solution of a Dirichlet energy minimization problem and give conditions under which it may suffer from over-smoothing. To overcome the over-smoothing problem, we design our implicit graph diffusion layer as the solution of a Dirichlet energy minimization problem with constraints on vertex features, enabling it to trade off smoothing with the preservation of node feature information. With an appropriate hyperparameter set to be larger than the largest eigenvalue of the parameterized graph Laplacian, our framework guarantees a unique equilibrium and quick convergence. Our models demonstrate better performance than leading implicit and explicit GNNs on benchmark datasets for node and graph classification tasks, with substantial accuracy improvements observed for some datasets.


page 1

page 2

page 3

page 4


Dirichlet Energy Constrained Learning for Deep Graph Neural Networks

Graph neural networks (GNNs) integrate deep architectures and topologica...

Optimization-Induced Graph Implicit Nonlinear Diffusion

Due to the over-smoothing issue, most existing graph neural networks can...

Re-Think and Re-Design Graph Neural Networks in Spaces of Continuous Graph Diffusion Functionals

Graph neural networks (GNNs) are widely used in domains like social netw...

A Note on Over-Smoothing for Graph Neural Networks

Graph Neural Networks (GNNs) have achieved a lot of success on graph-str...

Implicit vs Unfolded Graph Neural Networks

It has been observed that graph neural networks (GNN) sometimes struggle...

Analysis of Graph Neural Networks with Theory of Markov Chains

In this paper, we provide a theoretical tool for the interpretation and ...

The Geometric Occam's Razor Implicit in Deep Learning

In over-parameterized deep neural networks there can be many possible pa...

Please sign up or login with your details

Forgot password? Click here to reset