Methods Included: Standardizing Computational Reuse and Portability with the Common Workflow Language

05/14/2021
by   Michael R. Crusoe, et al.
0

A widely used standard for portable multilingual data analysis pipelines would enable considerable benefits to scholarly publication reuse, research/industry collaboration, regulatory cost control, and to the environment. Published research that used multiple computer languages for their analysis pipelines would include a complete and reusable description of that analysis that is runnable on a diverse set of computing environments. Researchers would be able to easier collaborate and reuse these pipelines, adding or exchanging components regardless of programming language used; collaborations with and within the industry would be easier; approval of new medical interventions that rely on such pipelines would be faster. Time will be saved and environmental impact would also be reduced, as these descriptions contain enough information for advanced optimization without user intervention. Workflows are widely used in data analysis pipelines, enabling innovation and decision-making for the modern society. In many domains the analysis components are numerous and written in multiple different computer languages by third parties. However, lacking a standard for reusable and portable multilingual workflows, then reusing published multilingual workflows, collaborating on open problems, and optimizing their execution would be severely hampered. Moreover, only a standard for multilingual data analysis pipelines that was widely used would enable considerable benefits to research-industry collaboration, regulatory cost control, and to preserving the environment. Prior to the start of the CWL project, there was no standard for describing multilingual analysis pipelines in a portable and reusable manner. Even today / currently, although there exist hundreds of single-vendor and other single-source systems that run workflows, none is a general, community-driven, and consensus-built standard.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/26/2015

JMS: A workflow management system and web-based cluster front-end for the Torque resource manager

Motivation: Complex computational pipelines are becoming a staple of mod...
research
06/04/2023

A Technical Report for Polyglot-Ko: Open-Source Large-Scale Korean Language Models

Polyglot is a pioneering project aimed at enhancing the non-English lang...
research
08/11/2018

MARVIN: An Open Machine Learning Corpus and Environment for Automated Machine Learning Primitive Annotation and Execution

In this demo paper, we introduce the DARPA D3M program for automatic mac...
research
04/11/2022

Adapting BigScience Multilingual Model to Unseen Languages

We benchmark different strategies of adding new languages (German and Ko...
research
04/16/2020

A Workflow Manager for Complex NLP and Content Curation Pipelines

We present a workflow manager for the flexible creation and customisatio...
research
08/03/2023

DaphneSched: A Scheduler for Integrated Data Analysis Pipelines

DAPHNE is a new open-source software infrastructure designed to address ...

Please sign up or login with your details

Forgot password? Click here to reset