Improving the Effective Utilization of Supercomputer Resources by Adding Low-Priority Containerized Jobs

09/01/2019
by   Julia Dubenskaya, et al.
0

We propose an approach to utilize idle computational resources of supercomputers. The idea is to maintain an additional queue of low-priority non-parallel jobs and execute them in containers, using container migration tools to break the execution down into separate intervals. We propose a container management system that can maintain this queue and interact with the supercomputer scheduler. We conducted a series of experiments simulating supercomputer scheduler and the proposed system. The experiments demonstrate that the proposed system increases the effective utilization of supercomputer resources under most of the conditions, in some cases significantly improving the performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/12/2019

Differential Approximation and Sprinting for Multi-Priority Big Data Engines

Today's big data clusters based on the MapReduce paradigm are capable of...
research
05/22/2019

Two stage cluster for resource optimization with Apache Mesos

As resource estimation for jobs is difficult, users often overestimate t...
research
01/22/2018

Adaptive parallelism with RMI: Idle high-performance computing resources can be completely avoided

In practice, standard scheduling of parallel computing jobs almost alway...
research
05/08/2019

Algorithms for Grey-Weighted Distance Computations

With the increasing size of datasets and demand for real time response f...
research
03/10/2021

A Resourceful Coordination Approach for Multilevel Scheduling

HPC users aim to improve their execution times without particular regard...
research
03/06/2022

Efficient Scheduling for Scalable Bioinformatics Analysis Platform with Microservices

With the advancement of biology and computer science, amount of bioinfor...

Please sign up or login with your details

Forgot password? Click here to reset