Multi-Resource List Scheduling of Moldable Parallel Jobs under Precedence Constraints

by   Lucas Perotin, et al.

The scheduling literature has traditionally focused on a single type of resource (e.g., computing nodes). However, scientific applications in modern High-Performance Computing (HPC) systems process large amounts of data, hence have diverse requirements on different types of resources (e.g., cores, cache, memory, I/O). All of these resources could potentially be exploited by the runtime scheduler to improve the application performance. In this paper, we study multi-resource scheduling to minimize the makespan of computational workflows comprised of parallel jobs subject to precedence constraints. The jobs are assumed to be moldable, allowing the scheduler to flexibly select a variable set of resources before execution. We propose a multi-resource, list-based scheduling algorithm, and prove that, on a system with d types of schedulable resources, our algorithm achieves an approximation ratio of 1.619d+2.545√(d)+1 for any d, and a ratio of d+O(√(d^2)) for large d. We also present improved results for independent jobs and for jobs with special precedence constraints (e.g., series-parallel graphs and trees). Finally, we prove a lower bound of d on the approximation ratio of any list scheduling scheme with local priority considerations. To the best of our knowledge, these are the first approximation results for moldable workflows with multiple resource requirements.



There are no comments yet.


page 1

page 2

page 3

page 4


A log-linear (2+5/6)-approximation algorithm for parallel machine scheduling with a single orthogonal resource

As the gap between compute and I/O performance tends to grow, modern Hig...

Adaptive parallelism with RMI: Idle high-performance computing resources can be completely avoided

In practice, standard scheduling of parallel computing jobs almost alway...

Scheduling Jobs with Random Resource Requirements in Computing Clusters

We consider a natural scheduling problem which arises in many distribute...

Wukong: A Scalable and Locality-Enhanced Framework for Serverless Parallel Computing

Serverless computing is increasingly being used for parallel computing, ...

Scheduling and Trade-off Analysis for Multi-Source Multi-Processor Systems with Divisible Loads

The main goal of parallel processing is to provide users with performanc...

Mitigating inefficient task mappings with an Adaptive Resource-Moldable Scheduler (ARMS)

Efficient runtime task scheduling on complex memory hierarchy becomes in...

Scheduling on Two Types of Resources: a Survey

We study the problem of executing an application represented by a preced...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.