Learning Graph Normalization for Graph Neural Networks

09/24/2020
by   Yihao Chen, et al.
0

Graph Neural Networks (GNNs) have attracted considerable attention and have emerged as a new promising paradigm to process graph-structured data. GNNs are usually stacked to multiple layers and the node representations in each layer are computed through propagating and aggregating the neighboring node features with respect to the graph. By stacking to multiple layers, GNNs are able to capture the long-range dependencies among the data on the graph and thus bring performance improvements. To train a GNN with multiple layers effectively, some normalization techniques (e.g., node-wise normalization, batch-wise normalization) are necessary. However, the normalization techniques for GNNs are highly task-relevant and different application tasks prefer to different normalization techniques, which is hard to know in advance. To tackle this deficiency, in this paper, we propose to learn graph normalization by optimizing a weighted combination of normalization techniques at four different levels, including node-wise normalization, adjacency-wise normalization, graph-wise normalization, and batch-wise normalization, in which the adjacency-wise normalization and the graph-wise normalization are newly proposed in this paper to take into account the local structure and the global structure on the graph, respectively. By learning the optimal weights, we are able to automatically select a single best or a best combination of multiple normalizations for a specific task. We conduct extensive experiments on benchmark datasets for different tasks, including node classification, link prediction, graph classification and graph regression, and confirm that the learned graph normalization leads to competitive results and that the learned weights suggest the appropriate normalization techniques for the specific task. Source code is released here https://github.com/cyh1112/GraphNormalization.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

10/27/2021

Node-wise Localization of Graph Neural Networks

Graph neural networks (GNNs) emerge as a powerful family of representati...
02/14/2022

Improved Aggregating and Accelerating Training Methods for Spatial Graph Neural Networks on Fraud Detection

Graph neural networks (GNNs) have been widely applied to numerous fields...
02/05/2022

MarkovGNN: Graph Neural Networks on Markov Diffusion

Most real-world networks contain well-defined community structures where...
09/07/2020

GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training

Normalization plays an important role in the optimization of deep neural...
06/12/2020

Towards Deeper Graph Neural Networks with Differentiable Group Normalization

Graph neural networks (GNNs), which learn the representation of a node b...
12/08/2021

Adaptive Kernel Graph Neural Network

Graph neural networks (GNNs) have demonstrated great success in represen...
04/18/2021

Ranking Structured Objects with Graph Neural Networks

Graph neural networks (GNNs) have been successfully applied in many stru...

Code Repositories

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.