Dynamo – Handling Scientific Data Across Sites and Storage Media

03/25/2020
by   Yutaro Iiyama, et al.
0

Dynamo is a full-stack software solution for scientific data management. Dynamo's architecture is modular, extensible, and customizable, making the software suitable for managing data in a wide range of installation scales, from a few terabytes stored at a single location to hundreds of petabytes distributed across a worldwide computing grid. This article documents the core system design of Dynamo and describes the applications that implement various data management tasks. A brief report is also given on the operational experiences of the system at the CMS experiment at the CERN Large Hadron Collider and at a small scale analysis facility.

READ FULL TEXT

page 4

page 8

page 10

research
02/03/2022

Astronomical data organization, management and access in Scientific Data Lakes

The data volumes stored in telescope archives is constantly increasing d...
research
05/30/2019

Dashboard Task Monitor for Managing ATLAS User Analysis on the Grid

The organization of the distributed user analysis on the Worldwide LHC C...
research
10/26/2018

Federating distributed storage for clouds in ATLAS

Input data for applications that run in cloud computing centres can be s...
research
02/26/2019

Rucio - Scientific Data Management

Rucio is an open source software framework that provides scientific coll...
research
10/10/2020

AstroDS – A Distributed Storage for Astrophysics of Cosmic Rays. Current Status

Currently, the processing of scientific data in astroparticle physics is...
research
07/14/2022

Toward a Framework for Integrative, FAIR, and Reproducible Management of Data on the Dynamic Balance of Microbial Communities

The increasing volumes of data produced by high-throughput instruments c...
research
04/26/2023

Architecting Complex, Long-Lived Scientific Software

Software is a critical aspect of large-scale science, providing essentia...

Please sign up or login with your details

Forgot password? Click here to reset