The BioExcel methodology for developing dynamic, scalable, reliable and portable computational biomolecular workflows

08/30/2022
by   Jorge Ejarque, et al.
0

Developing complex biomolecular workflows is not always straightforward. It requires tedious developments to enable the interoperability between the different biomolecular simulation and analysis tools. Moreover, the need to execute the pipelines on distributed systems increases the complexity of these developments. To address these issues, we propose a methodology to simplify the implementation of these workflows on HPC infrastructures. It combines a library, the BioExcel Building Blocks (BioBBs), that allows scientists to implement biomolecular pipelines as Python scripts, and the PyCOMPSs programming framework which allows to easily convert Python scripts into task-based parallel workflows executed in distributed computing systems such as HPC clusters, clouds, containerized platforms, etc. Using this methodology, we have implemented a set of computational molecular workflows and we have performed several experiments to validate its portability, scalability, reliability and malleability.

READ FULL TEXT

page 3

page 5

page 9

research
12/01/2020

Python Workflows on HPC Systems

The recent successes and wide spread application of compute intensive ma...
research
10/17/2018

Asynchronous Execution of Python Code on Task Based Runtime Systems

Despite advancements in the areas of parallel and distributed computing,...
research
08/31/2022

pPython for Parallel Python Programming

pPython seeks to provide a parallel capability that provides good speed-...
research
09/02/2018

A Serverless Tool for Platform Agnostic Computational Experiment Management

Neuroscience has been carried into the domain of big data and high perfo...
research
08/04/2022

A Container-Based Workflow for Distributed Training of Deep Learning Algorithms in HPC Clusters

Deep learning has been postulated as a solution for numerous problems in...
research
04/20/2021

ds-array: A Distributed Data Structure for Large Scale Machine Learning

Machine learning has proved to be a useful tool for extracting knowledge...
research
11/06/2019

zksk: A Library for Composable Zero-Knowledge Proofs

Zero-knowledge proofs are an essential building block in many privacy-pr...

Please sign up or login with your details

Forgot password? Click here to reset