DeepAI
Log In Sign Up

A Deep Graph Neural Networks Architecture Design: From Global Pyramid-like Shrinkage Skeleton to Local Topology Link Rewiring

12/16/2020
by   Gege Zhang, et al.
0

Expressivity plays a fundamental role in evaluating deep neural networks, and it is closely related to understanding the limit of performance improvement. In this paper, we propose a three-pipeline training framework based on critical expressivity, including global model contraction, weight evolution, and link's weight rewiring. Specifically, we propose a pyramidal-like skeleton to overcome the saddle points that affect information transfer. Then we analyze the reason for the modularity (clustering) phenomenon in network topology and use it to rewire potential erroneous weighted links. We conduct numerical experiments on node classification and the results confirm that the proposed training framework leads to a significantly improved performance in terms of fast convergence and robustness to potential erroneous weighted links. The architecture design on GNNs, in turn, verifies the expressivity of GNNs from dynamics and topological space aspects and provides useful guidelines in designing more efficient neural networks.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/02/2021

Evaluating Deep Graph Neural Networks

Graph Neural Networks (GNNs) have already been widely applied in various...
09/21/2021

wsGAT: Weighted and Signed Graph Attention Networks for Link Prediction

Graph Neural Networks (GNNs) have been widely used to learn representati...
12/19/2020

A pipeline for fair comparison of graph neural networks in node classification tasks

Graph neural networks (GNNs) have been investigated for potential applic...
07/06/2021

Dirichlet Energy Constrained Learning for Deep Graph Neural Networks

Graph neural networks (GNNs) integrate deep architectures and topologica...
06/30/2021

Curvature Graph Neural Network

Graph neural networks (GNNs) have achieved great success in many graph-b...
02/01/2022

Investigating Transfer Learning in Graph Neural Networks

Graph neural networks (GNNs) build on the success of deep learning model...