DeepAI AI Chat
Log In Sign Up

Middleware Building Blocks for Workflow Systems

by   Matteo Turilli, et al.

This paper describes a building blocks approach to the design of scientific workflow systems. We discuss RADICAL-Cybertools as one implementation of the building blocks concept, showing how they are designed and developed in accordance with this approach. Four case studies are presented, discussing how RADICAL-Cybertools are integrated with existing workflow, workload, and general purpose computing systems to support the execution of scientific workflows. This paper offers three main contributions: (i) showing the relevance of the design principles of self-sufficiency, interoperability, composability and extensibility for middleware to support scientific workflows on high performance computing machines; (ii) illustrating a set of building blocks that enable multiple points of integration, which results in design flexibility and functional extensibility, as well as providing a level of "unification" in the conceptual reasoning across otherwise very different tools and systems; and (iii) showing how these building blocks have been used to develop and integrate workflow systems.


RADICAL-Cybertools: Middleware Building Blocks for Scalable Science

RADICAL-Cybertools (RCT) are a set of software systems that serve as mid...

Toward Common Components for Open Workflow Systems

The role of scalable high-performance workflows and flexible workflow ma...

Code Farming: A Process for Creating Generic Computational Building Blocks

Motivated by a desire to improve on the current state of the art in gene...

FDBB: Fluid Dynamics Building Blocks

High-performance computing platforms are becoming more and more heteroge...

A Guide to Particle Advection Performance

The performance of particle advection-based flow visualization technique...

Engineering Resilient Collective Adaptive Systems by Self-Stabilisation

Collective adaptive systems are an emerging class of networked computati...

Path Towards Multilevel Evolution of Robots

Multi-level evolution is a bottom-up robotic design paradigm which decom...