Scalable load balancing in networked systems: A survey of recent advances

06/14/2018
by   Mark van der Boor, et al.
0

The basic load balancing scenario involves a single dispatcher where tasks arrive that must immediately be forwarded to one of N single-server queues. We discuss recent advances on scalable load balancing schemes which provide favorable delay performance when N grows large, and yet only require minimal implementation overhead. Join-the-Shortest-Queue (JSQ) yields vanishing delays as N grows large, as in a centralized queueing arrangement, but involves a prohibitive communication burden. In contrast, power-of-d or JSQ(d) schemes that assign an incoming task to a server with the shortest queue among d servers selected uniformly at random require little communication, but lead to constant delays. In order to examine this fundamental trade-off between delay performance and implementation overhead, we consider JSQ(d(N)) schemes where the diversity parameter d(N) depends on N and investigate what growth rate of d(N) is required to asymptotically match the optimal JSQ performance on fluid and diffusion scale. Stochastic coupling techniques and stochastic-process limits play an instrumental role in establishing the asymptotic optimality. We demonstrate how this methodology carries over to infinite-server settings, finite buffers, multiple dispatchers, servers arranged on graph topologies, and token-based load balancing including the popular Join-the-Idle-Queue (JIQ) scheme. In this way we provide a broad overview of the many recent advances in the field. This survey extends the short review presented at ICM 2018 (arXiv:1712.08555).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/22/2017

Scalable Load Balancing in Networked Systems: Universality Properties and Stochastic Coupling Methods

We present an overview of scalable load balancing algorithms which provi...
research
06/01/2023

Optimal Rate-Matrix Pruning For Large-Scale Heterogeneous Systems

We present an analysis of large-scale load balancing systems, where the ...
research
03/20/2018

Join-Idle-Queue with Service Elasticity: Large-Scale Asymptotics of a Non-monotone System

We consider the model of a token-based joint auto-scaling and load balan...
research
12/14/2020

Optimal Hyper-Scalable Load Balancing with a Strict Queue Limit

Load balancing plays a critical role in efficiently dispatching jobs in ...
research
06/04/2017

Load Balancing in Large-Scale Systems with Multiple Dispatchers

Load balancing algorithms play a crucial role in delivering robust appli...
research
03/24/2017

Optimal Service Elasticity in Large-Scale Distributed Systems

A fundamental challenge in large-scale cloud networks and data centers i...
research
08/09/2022

Learning Mean-Field Control for Delayed Information Load Balancing in Large Queuing Systems

Recent years have seen a great increase in the capacity and parallel pro...

Please sign up or login with your details

Forgot password? Click here to reset