DAOS as HPC Storage, a view from Numerical Weather Prediction
share
Novel object storage solutions potentially address long-standing scalability issues with POSIX file systems, and Storage Class Memory (SCM) offers promising performance characteristics for data-intensive use cases. Intel's Distributed Asynchronous Object Store (DAOS) is an emerging high-performance object store which can leverage SCM and NVMe devices. It has been gaining traction after scoring top positions in the I/O 500 benchmark. Numerical Weather Prediction (NWP) simulations are sensitive to I/O performance and scaling, and their output resolution and diversity is expected to increase significantly in the near future. In this work, we present a preliminary assessment of DAOS in conjunction with SCM on a research HPC system and evaluate its potential use as HPC storage at a world-leading weather forecasting centre. We demonstrate DAOS can provide the required performance, with bandwidth scaling linearly with additional SCM nodes in most cases, although choices in configuration and application design can impact achievable bandwidth. We describe a new I/O benchmark and associated metrics that address object storage performance from application-derived workloads that can be utilised to explore real-world performance for this new class of storage systems
READ FULL TEXT