Viash: from scripts to pipelines

10/21/2021
by   Robrecht Cannoodt, et al.
0

Most bioinformatics pipelines consist of software components that are tightly coupled to the logic of the pipeline itself. This limits reusability of the individual components in the pipeline or introduces maintenance overhead when they need to be reimplemented in multiple pipelines. We introduce Viash, a tool for speeding up development of robust pipelines through "code-first" prototyping, separation of concerns and code generation of modular pipeline components. By decoupling the component functionality from the pipeline logic, component functionality becomes fully pipeline-agnostic, and conversely the resulting pipelines are agnostic towards specific component requirements. This separation of concerns improves reusability of components and facilitates multidisciplinar and pan-organisational collaborations. It has been applied in a variety of projects, from proof-of-concept pipelines to supporting an international data science competition. Viash is available as an open-source project at https://github.com/viash-io/viash and documentation is available at https://viash.io.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/02/2021

The Art and Practice of Data Science Pipelines: A Comprehensive Study of Data Science Pipelines In Theory, In-The-Small, and In-The-Large

Increasingly larger number of software systems today are including data ...
research
01/03/2018

Prediction of corrosions in Gas and Oil pipelines based on the theory of records

Predictions of corrosions in pipelines are valuable. Based on the availa...
research
05/05/2022

Replicating Data Pipelines with GrimoireLab

In this paper, we present our MSR Hackathon 2022 project that replicates...
research
10/18/2019

PyTorchPipe: a framework for rapid prototyping of pipelines combining language and vision

Access to vast amounts of data along with affordable computational power...
research
04/16/2021

Pylot: A Modular Platform for Exploring Latency-Accuracy Tradeoffs in Autonomous Vehicles

We present Pylot, a platform for autonomous vehicle (AV) research and de...
research
02/25/2019

Quantifying error contributions of computational steps, algorithms and hyperparameter choices in image classification pipelines

Data science relies on pipelines that are organized in the form of inter...
research
08/28/2023

Towards Evolution Capabilities in Data Pipelines

Evolutionary change over time in the context of data pipelines is certai...

Please sign up or login with your details

Forgot password? Click here to reset