Local2Global: A distributed approach for scaling representation learning on graphs

01/12/2022
by   Lucas G. S. Jeub, et al.
34

We propose a decentralised "local2global"' approach to graph representation learning, that one can a-priori use to scale any embedding technique. Our local2global approach proceeds by first dividing the input graph into overlapping subgraphs (or "patches") and training local representations for each patch independently. In a second step, we combine the local representations into a globally consistent representation by estimating the set of rigid motions that best align the local representations using information from the patch overlaps, via group synchronization. A key distinguishing feature of local2global relative to existing work is that patches are trained independently without the need for the often costly parameter synchronization during distributed training. This allows local2global to scale to large-scale industrial applications, where the input graph may not even fit into memory and may be stored in a distributed manner. We apply local2global on data sets of different sizes and show that our approach achieves a good trade-off between scale and accuracy on edge reconstruction and semi-supervised classification. We also consider the downstream task of anomaly detection and show how one can use local2global to highlight anomalies in cybersecurity networks.

READ FULL TEXT
POST COMMENT

Comments

Manish123

Nice info


SMM Panel

Authors

page 1

page 2

page 3

page 4

07/26/2021

Local2Global: Scaling global representation learning on graphs via local training

We propose a decentralised "local2global" approach to graph representati...
07/17/2017

graph2vec: Learning Distributed Representations of Graphs

Recent works on representation learning for graph structured data predom...
09/27/2018

Deep Graph Infomax

We present Deep Graph Infomax (DGI), a general approach for learning nod...
04/29/2021

A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection

Anomaly detection, the task of identifying unusual samples in data, ofte...
02/07/2022

Context Autoencoder for Self-Supervised Representation Learning

We present a novel masked image modeling (MIM) approach, context autoenc...
05/17/2019

Neither Global Nor Local: A Hierarchical Robust Subspace Clustering For Image Data

In this paper, we consider the problem of subspace clustering in presenc...
03/20/2022

SimAN: Exploring Self-Supervised Representation Learning of Scene Text via Similarity-Aware Normalization

Recently self-supervised representation learning has drawn considerable ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.