Exploring Scientific Application Performance Using Large Scale Object Storage

07/06/2018
by   Steven Wei Der Chien, et al.
0

One of the major performance and scalability bottlenecks in large scientific applications is parallel reading and writing to supercomputer I/O systems. The usage of parallel file systems and consistency requirements of POSIX, that all the traditional HPC parallel I/O interfaces adhere to, pose limitations to the scalability of scientific applications. Object storage is a widely used storage technology in cloud computing and is more frequently proposed for HPC workload to address and improve the current scalability and performance of I/O in scientific applications. While object storage is a promising technology, it is still unclear how scientific applications will use object storage and what the main performance benefits will be. This work addresses these questions, by emulating an object storage used by a traditional scientific application and evaluating potential performance benefits. We show that scientific applications can benefit from the usage of object storage on large scales.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/16/2022

Performance Comparison of DAOS and Lustre for Object Data Storage Approaches

High-performance object stores are an emerging technology which offers a...
research
09/05/2021

Assessing the Use Cases of Persistent Memory in High-Performance Scientific Computing

As the High Performance Computing world moves towards the Exa-Scale era,...
research
12/01/2017

DAOS for Extreme-scale Systems in Scientific Applications

Exascale I/O initiatives will require new and fully integrated I/O model...
research
02/13/2017

Data-Intensive Supercomputing in the Cloud: Global Analytics for Satellite Imagery

We present our experiences using cloud computing to support data-intensi...
research
03/03/2021

VELOC: VEry Low Overhead Checkpointing in the Age of Exascale

Checkpointing large amounts of related data concurrently to stable stora...
research
05/16/2018

Client-side Straggler-Aware I/O Scheduler for Object-based Parallel File Systems

Object-based parallel file systems have emerged as promising storage sol...
research
08/14/2022

DAOS as HPC Storage, a view from Numerical Weather Prediction

Novel object storage solutions potentially address long-standing scalabi...

Please sign up or login with your details

Forgot password? Click here to reset