Optimal Load Balancing and Assessment of Existing Load Balancing Criteria

04/04/2021
by   Anthony Boulmier, et al.
0

Parallel iterative applications often suffer from load imbalance, one of the most critical performance degradation factors. Hence, load balancing techniques are used to distribute the workload evenly to maximize performance. A key challenge is to know when to use load balancing techniques. In general, this is done through load balancing criteria, which trigger load balancing based on runtime application data and/or user-defined information. In the first part of this paper, we introduce a novel, automatic load balancing criterion derived from a simple mathematical model. In the second part, we propose a branch-and-bound algorithm to find the load balancing iterations that lead to the optimal application performance. This algorithm finds the optimal load balancing scenario in quadratic time while, to the best of our knowledge, this has never been addressed in less than an exponential time. Finally, we compare the performance of the scenarios produced by state-of-the-art load balancing criteria relative to the optimal load balancing scenario in synthetic benchmarks and parallel N-body simulations. In the synthetic benchmarks, we observe that the proposed criterion outperforms the other automatic criteria. In the numerical experiments, we show that our new criterion is, on average, 4.9% faster than state-of-the-art load balancing criteria and can outperform them by up to 17.6%. Moreover, we see in the numerical study that the state-of-the-art automatic criteria are at worst 47.4% slower than the optimum and at best 16.5% slower.

READ FULL TEXT
research
09/16/2019

On the Benefits of Anticipating Load Imbalance for Performance Optimization of Parallel Applications

In parallel iterative applications, computational efficiency is essentia...
research
11/30/2018

Dynamic Load Balancing Techniques for Particulate Flow Simulations

Parallel multiphysics simulations often suffer from load imbalances orig...
research
10/16/2012

A Case Study in Complexity Estimation: Towards Parallel Branch-and-Bound over Graphical Models

We study the problem of complexity estimation in the context of parallel...
research
08/25/2021

Towards Informed Partitioning for Load Balancing: a Proof-of-Concept

Most parallel applications suffer from load imbalance, a crucial perform...
research
10/06/2020

Local Deal-Agreement Based Monotonic Distributed Algorithms for Load Balancing in General Graphs

In computer networks, participants may cooperate in processing tasks, so...
research
08/13/2019

uPredict: A User-Level Profiler-Based Predictive Framework for Single VM Applications in Multi-Tenant Clouds

Most existing studies on performance prediction for virtual machines (VM...
research
05/18/2023

Maximal workload, minimal workload, maximal workload difference: optimizing all criteria at once

In a simple model of assigning workers to tasks, every solution that min...

Please sign up or login with your details

Forgot password? Click here to reset