DeepAI AI Chat
Log In Sign Up

Multi-Dimensional Balanced Graph Partitioning via Projected Gradient Descent

by   Dmitrii Avdiukhin, et al.
Grigory Yaroslavtsev
Indiana University

Motivated by performance optimization of large-scale graph processing systems that distribute the graph across multiple machines, we consider the balanced graph partitioning problem. Compared to the previous work, we study the multi-dimensional variant when balance according to multiple weight functions is required. As we demonstrate by experimental evaluation, such multi-dimensional balance is important for achieving performance improvements for typical distributed graph processing workloads. We propose a new scalable technique for the multidimensional balanced graph partitioning problem. The method is based on applying randomized projected gradient descent to a non-convex continuous relaxation of the objective. We show how to implement the new algorithm efficiently in both theory and practice utilizing various approaches for projection. Experiments with large-scale social networks containing up to hundreds of billions of edges indicate that our algorithm has superior performance compared with the state-of-the-art approaches.


VEBO: A Vertex- and Edge-Balanced Ordering Heuristic to Load Balance Parallel Graph Processing

Graph partitioning drives graph processing in distributed, disk-based an...

Provable Burer-Monteiro factorization for a class of norm-constrained matrix problems

We study the projected gradient descent method on low-rank matrix proble...

Think Locally, Act Globally: Perfectly Balanced Graph Partitioning

We present a novel local improvement scheme for the perfectly balanced g...

TurboGraph++: A Scalable and Fast Graph Analytics System

Existing distributed graph analytics systems are categorized into two ma...

Balanced k-means for Parallel Geometric Partitioning

Mesh partitioning is an indispensable tool for efficient parallel numeri...

Prioritized Restreaming Algorithms for Balanced Graph Partitioning

Balanced graph partitioning is a critical step for many large-scale dist...

FIB: A Method for Evaluation of Feature Impact Balance in Multi-Dimensional Data

Errors might not have the same consequences depending on the task at han...