DeepAI AI Chat
Log In Sign Up

JMS: A workflow management system and web-based cluster front-end for the Torque resource manager

01/26/2015
by   David K. Brown, et al.
0

Motivation: Complex computational pipelines are becoming a staple of modern scientific research. Often these pipelines are resource intensive and require days of computing time. In such cases, it makes sense to run them over distributed computer clusters where they can take advantage of the aggregated resources of many powerful computers. In addition to this, researchers often want to integrate their workflows into their own web servers. In these cases, software is needed to manage the submission of jobs from the web interface to the cluster and then return the results once the job has finished executing. Results: We have developed the Job Management System (JMS), a workflow management system and interface for the Torque resource manager. The JMS provides users with a user-friendly interface for creating complex workflows with multiple stages. It integrates this workflow functionality with Torque, a tool that is used to control and manage batch jobs on distributed computing clusters. The JMS can be used by researchers to build and run complex computational pipelines and provides functionality to include these pipelines in external interfaces. The JMS is currently being used to house a number of structural bioinformatics pipelines at the Research Unit in Bioinformatics (RUBi) at Rhodes University. Availability: The JMS is an open-source project and is freely available at https://github.com/RUBi-ZA/JMS

READ FULL TEXT
07/06/2022

A Kubernetes 'Bridge' operator between cloud and external resources

Many scientific workflows require dedicated compute resources, including...
05/14/2021

Methods Included: Standardizing Computational Reuse and Portability with the Common Workflow Language

A widely used standard for portable multilingual data analysis pipelines...
04/19/2018

An open-source job management framework for parameter-space exploration: OACIS

We present an open-source software framework for parameter-space explora...
09/27/2019

Telescope: an interactive tool for managing large scale analysis from mobile devices

In today's world of big data, computational analysis has become a key dr...
10/30/2022

Hybrid Reusable Computational Analytics Workflow Management with Cloudmesh

In this paper, we summarize our effort to create and utilize a simple fr...
08/07/2018

MaRe: Container-Based Parallel Computing with Data Locality

Application containers are emerging as key components in scientific proc...
07/25/2018

PaPaS: A Portable, Lightweight, and Generic Framework for Parallel Parameter Studies

The current landscape of scientific research is widely based on modeling...