Weighted Scheduling of Time-Sensitive Coflows

03/30/2023

∙

by   Olivier Brun, et al.

∙

0

∙

share

Datacenter networks routinely support the data transfers of distributed computing frameworks in the form of coflows, i.e., sets of concurrent flows related to a common task. The vast majority of the literature has focused on the problem of scheduling coflows for completion time minimization, i.e., to maximize the average rate at which coflows are dispatched in the network fabric. However, many modern applications generate coflows dedicated to online services and mission-critical computing tasks which have to comply with specific completion deadlines. In this paper, we introduce 𝚆𝙳𝙲𝚘𝚏𝚕𝚘𝚠, a new algorithm to maximize the weighted number of coflows that complete before their deadline. By combining a dynamic programming algorithm along with parallel inequalities, our heuristic solution performs at once coflow admission control and coflow prioritization, imposing a σ-order on the set of coflows. With extensive simulation, we demonstrate the effectiveness of our algorithm in improving up to 3× more coflows that meet their deadline in comparison the best SotA solution, namely 𝙲𝚂-𝙼𝙷𝙰. Furthermore, when weights are used to differentiate coflow classes, 𝚆𝙳𝙲𝚘𝚏𝚕𝚘𝚠 is able to improve the admission per class up to 4×, while increasing the average weighted coflow admission rate.