Cleora: A Simple, Strong and Scalable Graph Embedding Scheme

02/03/2021
by   Barbara Rychalska, et al.
0

The area of graph embeddings is currently dominated by contrastive learning methods, which demand formulation of an explicit objective function and sampling of positive and negative examples. This creates a conceptual and computational overhead. Simple, classic unsupervised approaches like Multidimensional Scaling (MSD) or the Laplacian eigenmap skip the necessity of tedious objective optimization, directly exploiting data geometry. Unfortunately, their reliance on very costly operations such as matrix eigendecomposition make them unable to scale to large graphs that are common in today's digital world. In this paper we present Cleora: an algorithm which gets the best of two worlds, being both unsupervised and highly scalable. We show that high quality embeddings can be produced without the popular step-wise learning framework with example sampling. An intuitive learning objective of our algorithm is that a node should be similar to its neighbors, without explicitly pushing disconnected nodes apart. The objective is achieved by iterative weighted averaging of node neigbors' embeddings, followed by normalization across dimensions. Thanks to the averaging operation the algorithm makes rapid strides across the embedding space and usually reaches optimal embeddings in just a few iterations. Cleora runs faster than other state-of-the-art CPU algorithms and produces embeddings of competitive quality as measured on downstream tasks: link prediction and node classification. We show that Cleora learns a data abstraction that is similar to contrastive methods, yet at much lower computational cost. We open-source Cleora under the MIT license allowing commercial use under https://github.com/Synerise/cleora.

READ FULL TEXT
research
11/19/2018

Role action embeddings: scalable representation of network positions

We consider the question of embedding nodes with similar local neighborh...
research
06/30/2020

SCE: Scalable Network Embedding from Sparsest Cut

Large-scale network embedding is to learn a latent representation for ea...
research
03/21/2019

Node Embedding over Temporal Graphs

In this work, we present a method for node embedding in temporal graphs....
research
08/30/2019

Fast and Accurate Network Embeddings via Very Sparse Random Projection

We present FastRP, a scalable and performant algorithm for learning dist...
research
03/30/2023

Efficient distributed representations beyond negative sampling

This article describes an efficient method to learn distributed represen...
research
07/02/2020

Maximizing Cohesion and Separation in Graph Representation Learning: A Distance-aware Negative Sampling Approach

The objective of unsupervised graph representation learning (GRL) is to ...
research
05/20/2020

Understanding Negative Sampling in Graph Representation Learning

Graph representation learning has been extensively studied in recent yea...

Please sign up or login with your details

Forgot password? Click here to reset