An Intermediate Data-driven Methodology for Scientific Workflow Management System to Support Reusability

10/23/2020
by   Debasish Chakroborti, et al.
0

In this thesis first we propose an intermediate data management scheme for a SWfMS. In our second attempt, we explored the possibilities and introduced an automatic recommendation technique for a SWfMS from real-world workflow data (i.e Galaxy [1] workflows) where our investigations show that the proposed technique can facilitate 51 intermediate data of previous workflows and can reduce 74 workflow buildings in a SWfMS. Later we propose an adaptive version of our technique by considering the states of tools in a SWfMS, which shows around 40 reusability for workflows. Consequently, in our fourth study, We have done several experiments for analyzing the performance and exploring the effectiveness of the technique in a SWfMS for various environments. The technique is introduced to emphasize on storing cost reduction, increase data reusability, and faster workflow execution, to the best of our knowledge, which is the first of its kind. Detail architecture and evaluation of the technique are presented in this thesis. We believe our findings and developed system will contribute significantly to the research domain of SWfMSs.

READ FULL TEXT

page 33

page 39

research
10/10/2020

Designing for Recommending Intermediate States in A Scientific Workflow Management System

To process a large amount of data sequentially and systematically, prope...
research
02/14/2022

Gain-loss ratio of storing intermediate data from workflows

Sequentially, the systematic processing of a significant amount of data ...
research
05/29/2021

WfCommons: A Framework for Enabling Scientific Workflow Research and Development

Scientific workflows are a cornerstone of modern scientific computing. T...
research
05/15/2017

A data-driven workflow for predicting horizontal well production using vertical well logs

In recent work, data-driven sweet spotting technique for shale plays pre...
research
07/11/2023

SecFlow: Adaptive Security-Aware Workflow Management System in Multi-Cloud Environments

In this paper, we propose an architecture for a security-aware workflow ...
research
04/22/2020

Exploring Trade-offs in Dynamic Task Triggering for Loosely Coupled Scientific Workflows

In order to achieve near-time insights, scientific workflows tend to be ...

Please sign up or login with your details

Forgot password? Click here to reset