Workflows Community Summit: Tightening the Integration between Computing Facilities and Scientific Workflows

01/19/2022
by   Rafael Ferreira da Silva, et al.
0

The importance of workflows is highlighted by the fact that they have underpinned some of the most significant discoveries of the past decades. Many of these workflows have significant computational, storage, and communication demands, and thus must execute on a range of large-scale computer systems, from local clusters to public clouds and upcoming exascale HPC platforms. Historically, infrastructures for workflow execution consisted of complex, integrated systems, developed in-house by workflow practitioners with strong dependencies on a range of legacy technologies. Due to the increasing need to support workflows, dedicated workflow systems were developed to provide abstractions for creating, executing, and adapting workflows conveniently and efficiently while ensuring portability. While these efforts are all worthwhile individually, there are now hundreds of independent workflow systems. The resulting workflow system technology landscape is fragmented, which may present significant barriers for future workflow users due to many seemingly comparable, yet usually mutually incompatible, systems that exist. In order to tackle some of these challenges, the DOE-funded ExaWorks and NSF-funded WorkflowsRI projects have organized in 2021 a series of events entitled the "Workflows Community Summit". The third edition of the “Workflows Community Summit" explored workflows challenges and opportunities from the perspective of computing centers and facilities. The third summit brought together a small group of facilities representatives with the aim to understand how workflows are currently being used at each facility, how facilities would like to interact with workflow developers and users, how workflows fit with facility roadmaps, and what opportunities there are for tighter integration between facilities and workflows. More information at: https://workflowsri.org/summits/facilities/

READ FULL TEXT

page 1

page 4

research
03/16/2021

Workflows Community Summit: Bringing the Scientific Workflows Community Together

Scientific workflows have been used almost universally across scientific...
research
06/09/2021

Workflows Community Summit: Advancing the State-of-the-art of Scientific Workflows Management Systems Research and Development

Scientific workflows are a cornerstone of modern scientific computing, a...
research
10/05/2021

A Community Roadmap for Scientific Workflows Research and Development

The landscape of workflow systems for scientific applications is notorio...
research
12/15/2021

or2yw: Modeling and Visualizing OpenRefineHistories as YesWorkflow Diagrams

OpenRefine is a popular open-source data cleaning tool. It allows users ...
research
06/13/2022

Towards an Approximation-Aware Computational Workflow Framework for Accelerating Large-Scale Discovery Tasks

The use of approximation is fundamental in computational science. Almost...
research
02/14/2019

Theory-plus-code documentation of the DEPAM workflow for soundscape description

In the Big Data era, the community of PAM faces strong challenges, inclu...
research
05/27/2021

RADICAL-Pilot and Parsl: Executing Heterogeneous Workflows on HPC Platforms

Executing scientific workflows with heterogeneous tasks on HPC platforms...

Please sign up or login with your details

Forgot password? Click here to reset