Community-based Layerwise Distributed Training of Graph Convolutional Networks

12/17/2021
by   Hongyi Li, et al.
8

The Graph Convolutional Network (GCN) has been successfully applied to many graph-based applications. Training a large-scale GCN model, however, is still challenging: Due to the node dependency and layer dependency of the GCN architecture, a huge amount of computational time and memory is required in the training process. In this paper, we propose a parallel and distributed GCN training algorithm based on the Alternating Direction Method of Multipliers (ADMM) to tackle the two challenges simultaneously. We first split GCN layers into independent blocks to achieve layer parallelism. Furthermore, we reduce node dependency by dividing the graph into several dense communities such that each of them can be trained with an agent in parallel. Finally, we provide solutions for all subproblems in the community-based ADMM algorithm. Preliminary results demonstrate that our proposed community-based ADMM training algorithm can lead to more than triple speedup while achieving the best performance compared with state-of-the-art methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/20/2019

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

Graph convolutional network (GCN) has been successfully applied to many ...
research
06/02/2023

Graph Sparsification for GCN Towards Optimal Crop Yield Predictions

In agronomics, predicting crop yield at a per field/county granularity i...
research
02/20/2021

GIST: Distributed Training for Large-Scale Graph Convolutional Networks

The graph convolutional network (GCN) is a go-to solution for machine le...
research
03/30/2020

L^2-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks

Graph convolution networks (GCN) are increasingly popular in many applic...
research
07/09/2021

Automated Graph Learning via Population Based Self-Tuning GCN

Owing to the remarkable capability of extracting effective graph embeddi...
research
03/08/2016

Testing fine-grained parallelism for the ADMM on a factor-graph

There is an ongoing effort to develop tools that apply distributed compu...

Please sign up or login with your details

Forgot password? Click here to reset