Increase and Conquer: Training Graph Neural Networks on Growing Graphs

06/07/2021
by   Juan Cervino, et al.
0

Graph neural networks (GNNs) use graph convolutions to exploit network invariances and learn meaningful features from network data. However, on large-scale graphs convolutions incur in high computational cost, leading to scalability limitations. Leveraging the graphon – the limit object of a graph – in this paper we consider the problem of learning a graphon neural network (WNN) – the limit object of a GNN – by training GNNs on graphs sampled Bernoulli from the graphon. Under smoothness conditions, we show that: (i) the expected distance between the learning steps on the GNN and on the WNN decreases asymptotically with the size of the graph, and (ii) when training on a sequence of growing graphs, gradient descent follows the learning direction of the WNN. Inspired by these results, we propose a novel algorithm to learn GNNs on large-scale graphs that, starting from a moderate number of nodes, successively increases the size of the graph during training. This algorithm is benchmarked on both a recommendation system and a decentralized control problem where it is shown to retain comparable performance, to its large-scale counterpart, at a reduced computational cost.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/27/2022

Training Graph Neural Networks on Growing Stochastic Graphs

Graph Neural Networks (GNNs) rely on graph convolutions to exploit meani...
research
04/15/2021

Convolutions for Spatial Interaction Modeling

In many different fields interactions between objects play a critical ro...
research
04/21/2021

GraphTheta: A Distributed Graph Neural Network Learning System With Flexible Training Strategy

Graph neural networks (GNNs) have been demonstrated as a powerful tool f...
research
03/13/2020

Automating Botnet Detection with Graph Neural Networks

Botnets are now a major source for many network attacks, such as DDoS at...
research
01/05/2020

Adaptive fractional order graph neural network

This paper proposes adaptive fractional order graph neural network (AFGN...
research
02/07/2023

Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs

Graph neural networks that model 3D data, such as point clouds or atoms,...
research
10/26/2020

Towards Scale-Invariant Graph-related Problem Solving by Iterative Homogeneous Graph Neural Networks

Current graph neural networks (GNNs) lack generalizability with respect ...

Please sign up or login with your details

Forgot password? Click here to reset