Performance Comparison of DAOS and Lustre for Object Data Storage Approaches

11/16/2022
by   Nicolau Manubens, et al.
0

High-performance object stores are an emerging technology which offers an alternative solution in the field of HPC storage, with potential to address long-standing scalability issues in traditional distributed POSIX file systems due to excessive consistency assurance and metadata prescriptiveness. In this paper we assess the performance of storing object-like data within a standard file system, where the configuration and access mechanisms have not been optimised for object access behaviour, and compare with and investigate the benefits of using an object storage system. Whilst this approach is not exploiting the file system in a standard way, this work allows us to investigate whether the underlying storage technology performance is more or less important than the software interface and infrastructure a file system or object store provides.

READ FULL TEXT
research
07/06/2018

Exploring Scientific Application Performance Using Large Scale Object Storage

One of the major performance and scalability bottlenecks in large scient...
research
08/14/2022

DAOS as HPC Storage, a view from Numerical Weather Prediction

Novel object storage solutions potentially address long-standing scalabi...
research
01/21/2023

Auditing Lustre file system

With the increasing time, we are facing massive demand for the increasin...
research
05/16/2018

Client-side Straggler-Aware I/O Scheduler for Object-based Parallel File Systems

Object-based parallel file systems have emerged as promising storage sol...
research
12/06/2022

DisTRaC: Accelerating High Performance Compute Processing for Temporary Data Storage

High Performance Compute (HPC) clusters often produce intermediate files...
research
01/24/2019

SimFS: A Simulation Data Virtualizing File System Interface

Nowadays simulations can produce petabytes of data to be stored in paral...
research
08/01/2017

Performance Measurements of Supercomputing and Cloud Storage Solutions

Increasing amounts of data from varied sources, particularly in the fiel...

Please sign up or login with your details

Forgot password? Click here to reset