Wide and Deep Graph Neural Networks with Distributed Online Learning

06/11/2020
by   Zhan Gao, et al.
0

Graph neural networks (GNNs) learn representations from network data with naturally distributed architectures. This renders them well-suited candidates for decentralized learning since the operations respect the structure imposed by the underlying graph. Oftentimes, this graph support changes with time, whether it is due to link failures or topology changes caused by mobile components. Modifications to the underlying structure create a mismatch between the graphs on which GNNs were trained and the ones on which they are tested. Online learning can be used to retrain the GNNs at test time, overcoming this issue. However, most online learning algorithms are centralized and work on convex objective functions (which GNNs rarely lead to). This paper puts forth the Wide and Deep GNN (WD-GNN), a novel architecture that can be easily updated with distributed online learning mechanisms. The WD-GNN consists of two components: the wide part is a bank of linear graph filters and the deep part is a convolutional GNN. At training time, the joint architecture learns a relevant nonlinear representation from data. At test time, the deep part is left unchanged, while the wide part is retrained online. Since the wide part is linear, the problem becomes convex, and online optimization algorithms can be used. We also propose a distributed online optimization algorithm that updates the wide part at test time, without violating its decentralized nature. We also analyze the stability of the WD-GNN to changes in the underlying topology and derive convergence guarantees for the online retraining procedure. These results indicate the transferability, scalability, and efficiency of the WD-GNN to adapt online to new testing scenarios in a distributed manner. Experiments on the control of robot swarms corroborate the theory and show the potential of the proposed architecture for distributed online learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/19/2021

Wide and Deep Graph Neural Network with Distributed Online Learning

Graph neural networks (GNNs) are naturally distributed architectures for...
04/04/2021

Decentralized Statistical Inference with Unrolled Graph Neural Networks

In this paper, we investigate the decentralized statistical inference pr...
03/08/2020

Graphs, Convolutions, and Neural Networks

Network data can be conveniently modeled as a graph signal, where data v...
10/06/2021

Space-Time Graph Neural Networks

We introduce space-time graph neural network (ST-GNN), a novel GNN archi...
12/29/2020

Decentralized Control with Graph Neural Networks

Dynamical systems consisting of a set of autonomous agents face the chal...
06/04/2020

Stochastic Graph Neural Networks

Graph neural networks (GNNs) model nonlinear representations in graph da...
03/23/2020

Graph Neural Networks for Decentralized Controllers

Dynamical systems comprised of autonomous agents arise in many relevant ...