Self-Learning Threshold-Based Load Balancing

10/29/2020
by   Diego Goldsztajn, et al.
0

We consider a large-scale service system where incoming tasks have to be instantaneously dispatched to one out of many parallel server pools. The dispatcher uses a threshold for balancing the load and keeping the maximum number of concurrent tasks across server pools low. We demonstrate that such a policy is optimal on the fluid and diffusion scales for a suitable threshold value, while only involving a small communication overhead. In order to set the threshold optimally, it is important, however, to learn the load of the system, which may be uncertain or even time-varying. For that purpose, we design a control rule for tuning the threshold in an online manner. We provide conditions which guarantee that this adaptive threshold settles at the optimal value, along with estimates for the time until this happens.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/18/2020

Learning and balancing time-varying loads in large-scale systems

Consider a system of n parallel server pools where tasks arrive as a tim...
research
12/16/2021

Utility maximizing load balancing policies

Consider a service system where incoming tasks are instantaneously dispa...
research
12/22/2017

Scalable Load Balancing in Networked Systems: Universality Properties and Stochastic Coupling Methods

We present an overview of scalable load balancing algorithms which provi...
research
09/04/2015

Predicting SLA Violations in Real Time using Online Machine Learning

Detecting faults and SLA violations in a timely manner is critical for t...
research
03/04/2020

LSQ: Load Balancing in Large-Scale Heterogeneous Systems with Multiple Dispatchers

Nowadays, the efficiency and even the feasibility of traditional load-ba...
research
06/06/2022

CARE: Resource Allocation Using Sparse Communication

We propose a new framework for studying effective resource allocation in...
research
04/15/2019

Consistent Dynamic CDN Server Assignment for Online Video Streaming with Optimality Guarantees

Server assignment plays an essential part in Content Delivery Network (C...

Please sign up or login with your details

Forgot password? Click here to reset