Meld: Exploring the Feasibility of a Framework-less Framework

08/31/2023
by   Kyle J. Knoepfel, et al.
0

HEP data-processing frameworks are essential ingredients in getting from raw data to physics results. But they are often tricky to use well, and they present a significant learning barrier for the beginning HEP physicist. In addition, existing frameworks typically support rigid, collider-based data models, which do not map well to neutrino-physics experiments like DUNE. Neutrino physicists thus expend significant effort working around framework limitations instead of using a framework that directly supports their needs. Presented here is Meld, a Fermilab R D project, which intends to address these limitations. By leveraging modern C++ capabilities, state-of-the-art concurrency libraries, and a flexible data model, it is possible for beginning (and seasoned) HEP physicists to execute framework programs easily and efficiently, with minimal coupling to framework-specific constructs. Meld aims to directly support the frameworks needs of neutrino experiments like DUNE as well as the more common collider-based experiments.

READ FULL TEXT

page 6

page 7

research
03/16/2022

Evolution of HEP Processing Frameworks

HEP data-processing software must support the disparate physics needs of...
research
05/20/2021

Towards an Arrow-native Storage System

With the ever-increasing dataset sizes, several file formats like Parque...
research
04/12/2022

Skyhook: Towards an Arrow-Native Storage System

With the ever-increasing dataset sizes, several file formats such as Par...
research
12/12/2021

In-Memory Indexed Caching for Distributed Data Processing

Powerful abstractions such as dataframes are only as efficient as their ...
research
07/11/2022

HEGrid: A High Efficient Multi-Channel Radio Astronomical Data Gridding Framework in Heterogeneous Computing Environments

The challenge to fully exploit the potential of existing and upcoming sc...
research
12/06/2018

K-Pg: Shared State in Differential Dataflows

Many of the most popular scalable data-processing frameworks are fundame...
research
06/22/2022

Exploring user needs in relation to algorithmically constructed classifications of publications

Algorithmic classification of research publications has been created to ...

Please sign up or login with your details

Forgot password? Click here to reset