Distributed data storage for modern astroparticle physics experiments

07/16/2019
by   Alexander Kryukov, et al.
0

The German-Russian Astroparticle Data Life Cycle Initiative is an international project launched in 2018. The Initiative aims to develop technologies that provide a unified approach to data management, as well as to demonstrate their applicability on the example of two large astrophysical experiments - KASCADE and TAIGA. One of the key points of the project is the development of a distributed storage, which, on the one hand, will allow data of several experiments to be combined into a single repository with unified interface, and on the other hand, will provide data to all participants of experimental groups for multi-messenger analysis. Our approach to storage design is based on the single write-multiple read (SWMR) model for accessing raw or centrally processed data for further analysis. The main feature of the distributed storage is the ability to extract data either as a collection of files or as aggregated events from different sources. In the last case the storage provides users with a special service that aggregates data from different storages into a single sample. Thanks to this feature, multi-messenger methods used for more sophisticated data exploration can be applied. Users can use both Web-interface and Application Programming Interface (API) for accessing the storage. In this paper we describe the architecture of a distributed data storage for astroparticle physics and discuss the current status of our work.

READ FULL TEXT
research
08/05/2019

Data Aggregation In The Astroparticle Physics Distributed Data Storage

German-Russian Astroparticle Data Life Cycle Initiative is an internatio...
research
10/10/2020

AstroDS – A Distributed Storage for Astrophysics of Cosmic Rays. Current Status

Currently, the processing of scientific data in astroparticle physics is...
research
11/06/2018

Architecture of Distributed Data Storage for Astroparticle Physics

For the successful development of the astrophysics and, accordingly, for...
research
12/05/2018

A distributed data warehouse system for astroparticle physics

A distributed data warehouse system is one of the actual issues in the f...
research
07/04/2019

Development of a data infrastructure for a global data and analysis center in astroparticle physics

Nowadays astroparticle physics faces a rapid data volume increase. Meanw...
research
08/01/2017

Performance Measurements of Supercomputing and Cloud Storage Solutions

Increasing amounts of data from varied sources, particularly in the fiel...
research
07/15/2021

Exploring Object Stores for High-Energy Physics Data Storage

Over the last two decades, ROOT TTree has been used for storing over one...

Please sign up or login with your details

Forgot password? Click here to reset