Wide and Deep Graph Neural Network with Distributed Online Learning

by   Zhan Gao, et al.

Graph neural networks (GNNs) are naturally distributed architectures for learning representations from network data. This renders them suitable candidates for decentralized tasks. In these scenarios, the underlying graph often changes with time due to link failures or topology variations, creating a mismatch between the graphs on which GNNs were trained and the ones on which they are tested. Online learning can be leveraged to retrain GNNs at testing time to overcome this issue. However, most online algorithms are centralized and usually offer guarantees only on convex problems, which GNNs rarely lead to. This paper develops the Wide and Deep GNN (WD-GNN), a novel architecture that can be updated with distributed online learning mechanisms. The WD-GNN consists of two components: the wide part is a linear graph filter and the deep part is a nonlinear GNN. At training time, the joint wide and deep architecture learns nonlinear representations from data. At testing time, the wide, linear part is retrained, while the deep, nonlinear one remains fixed. This often leads to a convex formulation. We further propose a distributed online learning algorithm that can be implemented in a decentralized setting. We also show the stability of the WD-GNN to changes of the underlying graph and analyze the convergence of the proposed online learning procedure. Experiments on movie recommendation, source localization and robot swarm control corroborate theoretical findings and show the potential of the WD-GNN for distributed online learning.


page 1

page 2

page 3

page 4


Wide and Deep Graph Neural Networks with Distributed Online Learning

Graph neural networks (GNNs) learn representations from network data wit...

Learning Stable Graph Neural Networks via Spectral Regularization

Stability of graph neural networks (GNNs) characterizes how GNNs react t...

Stochastic Graph Neural Networks

Graph neural networks (GNNs) model nonlinear representations in graph da...

Graphs, Convolutions, and Neural Networks

Network data can be conveniently modeled as a graph signal, where data v...

Stability of Graph Neural Networks to Relative Perturbations

Graph neural networks (GNNs), consisting of a cascade of layers applying...

Fast Learning of Graph Neural Networks with Guaranteed Generalizability: One-hidden-layer Case

Although graph neural networks (GNNs) have made great progress recently ...

Offline-Online Learning of Deformation Model for Cable Manipulation with Graph Neural Networks

Manipulating deformable linear objects by robots has a wide range of app...

Please sign up or login with your details

Forgot password? Click here to reset