Profiling Resource Utilization of Bioinformatics Workflows

05/23/2020
by   Huazeng Deng, et al.
0

We present a software tool, the Container Profiler, that measures and records the resource usage of any containerized task. Our tool profiles the CPU, memory, disk, and network utilization of a containerized job by collecting Linux operating system metrics at the virtual machine, container, and process levels. The Container Profiler can produce utilization snapshots at multiple time points, allowing for continuous monitoring of the resources consumed by a container workflow. To investigate the utility of the Container Profiler we profiled the resource utilization requirements of a multi-stage bioinformatics analytical workflow (RNA sequencing using unique molecular identifiers). We examined the collected profile metrics and confirmed that they were consistent with the expected CPU, disk, network resource utilization patterns for the different stages of the workflow. We also quantified the profiling overhead and found that this was negligible. The Container Profiler is a useful tool that can be used to continuously monitor the resource consumption of long and complex containerized workflows that run locally or on the cloud. This can identify bottlenecks where more resources are needed to improve performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/14/2020

The Impact of Auto-Refactoring Code Smells on the Resource Utilization of Cloud Software

Cloud-based software-as-a-service (SaaS) have gained popularity due to t...
research
10/10/2018

Task Runtime Prediction in Scientific Workflows Using an Online Incremental Learning Approach

Many algorithms in workflow scheduling and resource provisioning rely on...
research
06/27/2023

Predicting the Impact of Batch Refactoring Code Smells on Application Resource Consumption

Automated batch refactoring has become a de-facto mechanism to restructu...
research
11/01/2018

Modeling Conceptual Characteristics of Virtual Machines for CPU Utilization Prediction

Cloud services have grown rapidly in recent years, which provide high fl...
research
01/20/2023

Adaptive Resource Allocation for Workflow Containerization on Kubernetes

In a cloud-native era, the Kubernetes-based workflow engine enables work...
research
03/08/2020

Multiple Regression Particle Swarm Optimization for Host Overload and Under-Load Detection

Detection of overloaded and under-loaded Host approaches in cloud comput...

Please sign up or login with your details

Forgot password? Click here to reset