I/O Workload Management for All-Flash Datacenter Storage Systems Based on Total Cost of Ownership

09/16/2018
by   Zhengyu Yang, et al.
0

Recently, the capital expenditure of flash-based Solid State Driver (SSDs) keeps declining and the storage capacity of SSDs keeps increasing. As a result, all-flash storage systems have started to become more economically viable for large shared storage installations in datacenters, where metrics like Total Cost of Ownership (TCO) are of paramount importance. On the other hand, flash devices suffer from write amplification, which, if unaccounted, can substantially increase the TCO of a storage system. In this paper, we first develop a TCO model for datacenter all-flash storage systems, and then plug a Write Amplification model (WAF) of NVMe SSDs we build based on empirical data into this TCO model. Our new WAF model accounts for workload characteristics like write rate and percentage of sequential writes. Furthermore, using both the TCO and WAF models as the optimization criterion, we design new flash resource management schemes (MINTCO) to guide datacenter managers to make workload allocation decisions under the consideration of TCO for SSDs. Based on that, we also develop MINTCO-RAID to support RAID SSDs and MINTCO-OFFLINE to optimize the offline workload-disk deployment problem during the initialization phase. Experimental results show that MINTCO can reduce the TCO and keep relatively high throughput and space utilization of the entire datacenter storage resources.

READ FULL TEXT

page 11

page 12

page 13

research
06/07/2023

An Analytical Model-based Capacity Planning Approach for Building CSD-based Storage Systems

The data movement in large-scale computing facilities (from compute node...
research
04/22/2020

Breaking Down Memory Walls: Adaptive Memory Management in LSM-based Storage Systems (Extended Version)

Log-Structured Merge-trees (LSM-trees) have been widely used in modern N...
research
07/04/2019

Automating Distributed Tiered Storage Management in Cluster Computing

Data-intensive platforms such as Hadoop and Spark are routinely used to ...
research
01/30/2019

Multi-Tier Buffer Management and Storage System Design for Non-Volatile Memory

The design of the buffer manager in database management systems (DBMSs) ...
research
01/12/2022

Enlightening Flash Storage to Stream Writes by Objects

For a write request, today flash storage cannot distinguish the logical ...
research
04/11/2023

An Empirical Evaluation of Columnar Storage Formats

Columnar storage is one of the core components of a modern data analytic...
research
02/27/2022

Past, Present and Future of Hadoop: A Survey

In this paper, a technology for massive data storage and computing named...

Please sign up or login with your details

Forgot password? Click here to reset