Distributed Edge Partitioning for Trillion-edge Graphs

08/16/2019
by   Masatoshi Hanai, et al.
0

We propose Distributed Neighbor Expansion (Distributed NE), a parallel and distributed edge partitioning method that can scale to trillion-edge graphs while providing high partitioning quality. Distributed NE is based on a new heuristic, called parallel expansion, where each partition is constructed in parallel by greedily expanding its edge set from a single vertex in such a way that the increase of the vertex cuts becomes local minimal. We theoretically prove that the proposed method has the upper bound in the partitioning quality. The empirical evaluation with various graphs shows that the proposed method produces higher-quality partitions than the state-of-the-art distributed graph partitioning algorithms. The performance evaluation shows that the space efficiency of the proposed method is an order-of-magnitude better than the existing algorithms, keeping its time efficiency comparable. As a result, Distributed NE can handle a trillion-edge graph using only a few hundreds of machines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/20/2019

Planted Models for k-way Edge and Vertex Expansion

Graph partitioning problems are a central topic of study in algorithms a...
research
08/20/2018

Scalable Edge Partitioning

Edge-centric distributed computations have appeared as a recent techniqu...
research
05/28/2020

Network Partitioning and Avoidable Contention

Network contention frequently dominates the run time of parallel algorit...
research
07/24/2022

CARGO: AI-Guided Dependency Analysis for Migrating Monolithic Applications to Microservices Architecture

Microservices Architecture (MSA) has become a de-facto standard for desi...
research
09/20/2016

An Efficient Method of Partitioning High Volumes of Multidimensional Data for Parallel Clustering Algorithms

An optimal data partitioning in parallel & distributed implementation of...
research
05/02/2021

Sphynx: a parallel multi-GPU graph partitioner for distributed-memory systems

Graph partitioning has been an important tool to partition the work amon...
research
11/17/2014

FGPGA: An Efficient Genetic Approach for Producing Feasible Graph Partitions

Graph partitioning, a well studied problem of parallel computing has man...

Please sign up or login with your details

Forgot password? Click here to reset