GEBT: Drawing Early-Bird Tickets in Graph Convolutional Network Training

03/01/2021
by   Haoran You, et al.
0

Graph Convolutional Networks (GCNs) have emerged as the state-of-the-art deep learning model for representation learning on graphs. However, it remains notoriously challenging to train and inference GCNs over large graph datasets, limiting their application to large real-world graphs and hindering the exploration of deeper and more sophisticated GCN graphs. This is because as the graph size grows, the sheer number of node features and the large adjacency matrix can easily explode the required memory and data movements. To tackle the aforementioned challenge, we explore the possibility of drawing lottery tickets when sparsifying GCN graphs, i.e., subgraphs that largely shrink the adjacency matrix yet are capable of achieving accuracy comparable to or even better than their corresponding full graphs. Specifically, we for the first time discover the existence of graph early-bird (GEB) tickets that emerge at the very early stage when sparsifying GCN graphs, and propose a simple yet effective detector to automatically identify the emergence of such GEB tickets. Furthermore, we develop a generic efficient GCN training framework dubbed GEBT that can significantly boost the efficiency of GCN training by (1) drawing joint early-bird tickets between the GCN graphs and models and (2) enabling simultaneously sparsifying both GCN graphs and models, paving the way for training and inferencing large GCN graphs to handle real-world graph datasets. Experiments on various GCN models and datasets consistently validate our GEB finding and the effectiveness of our GEBT, e.g., our GEBT achieves up to 80.2   85.6 leading to a comparable or even better accuracy as compared to state-of-the-art methods. Code available at https://github.com/RICE-EIC/GEBT

READ FULL TEXT

page 4

page 5

research
12/22/2021

GCoD: Graph Convolutional Network Acceleration via Dedicated Algorithm and Accelerator Co-Design

Graph Convolutional Networks (GCNs) have emerged as the state-of-the-art...
research
02/11/2021

Quartile-based Prediction of Event Types and Event Time in Business Processes using Deep Learning

Deep learning models are now being increasingly used for predictive proc...
research
09/26/2019

Drawing early-bird tickets: Towards more efficient training of deep networks

(Frankle & Carbin, 2019) shows that there exist winning tickets (small b...
research
03/05/2022

Scaling R-GCN Training with Graph Summarization

Training of Relation Graph Convolutional Networks (R-GCN) does not scale...
research
03/23/2023

Extracting real estate values of rental apartment floor plans using graph convolutional networks

Access graphs that indicate adjacency relationships from the perspective...
research
08/03/2023

Unsupervised Multiplex Graph Learning with Complementary and Consistent Information

Unsupervised multiplex graph learning (UMGL) has been shown to achieve s...
research
04/14/2021

Quadratic GCN for Graph Classification

Graph Convolutional Networks (GCNs) have been extensively used to classi...

Please sign up or login with your details

Forgot password? Click here to reset