Time-Efficient and High-Quality Graph Partitioning for Graph Dynamic Scaling

01/18/2021
by   Masatoshi Hanai, et al.
0

The dynamic scaling of distributed computations plays an important role in the utilization of elastic computational resources, such as the cloud. It enables the provisioning and de-provisioning of resources to match dynamic resource availability and demands. In the case of distributed graph processing, changing the number of the graph partitions while maintaining high partitioning quality imposes serious computational overheads as typically a time-consuming graph partitioning algorithm needs to execute each time repartitioning is required. In this paper, we propose a dynamic scaling method that can efficiently change the number of graph partitions while keeping its quality high. Our idea is based on two techniques: preprocessing and very fast edge partitioning, called graph edge ordering and chunk-based edge partitioning, respectively. The former converts the graph data into an ordered edge list in such a way that edges with high locality are closer to each other. The latter immediately divides the ordered edge list into an arbitrary number of high-quality partitions. The evaluation with the real-world billion-scale graphs demonstrates that our proposed approach significantly reduces the repartitioning time, while the partitioning quality it achieves is on par with that of the best existing static method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/20/2018

Scalable Edge Partitioning

Edge-centric distributed computations have appeared as a recent techniqu...
research
10/29/2021

SDP: Scalable Real-time Dynamic Graph Partitioner

Time-evolving large graph has received attention due to their participat...
research
10/28/2021

Finding a Concise, Precise, and Exhaustive Set of Near Bi-Cliques in Dynamic Graphs

A variety of tasks on dynamic graphs, including anomaly detection, commu...
research
11/17/2014

FGPGA: An Efficient Genetic Approach for Producing Feasible Graph Partitions

Graph partitioning, a well studied problem of parallel computing has man...
research
09/07/2018

A general-purpose hierarchical mesh partitioning method with node balancing strategies for large-scale numerical simulations

Large-scale parallel numerical simulations are essential for a wide rang...
research
05/30/2018

Q-Graph: Preserving Query Locality in Multi-Query Graph Processing

Arising user-centric graph applications such as route planning and perso...
research
02/20/2018

ILP-based Local Search for Graph Partitioning

Computing high-quality graph partitions is a challenging problem with nu...

Please sign up or login with your details

Forgot password? Click here to reset