On Distributed Storage Allocations of Large Files for Maximum Service Rate

08/08/2018
by   Pei Peng, et al.
0

Allocation of (redundant) file chunks throughout a distributed storage system affects important performance metrics such as the probability of file recovery, data download time, or the service rate of the system under a given data access model. This paper is concerned with the service rate under the assumption that the stored data is large and its download time is not negligible. We focus on quasi-uniform storage allocations and provide a service rate analysis for two common data access models. We find that the optimal allocation varies in accordance with different system parameters. This was not the case under the assumption that the download time does not scale with the size of data, where the minimal spreading allocation was previously found to be universally optimal.

READ FULL TEXT
research
02/08/2021

Distributed Storage Allocations for Optimal Service Rates

Redundant storage maintains the performance of distributed systems under...
research
01/08/2019

Service Rate Region of Content Access from Erasure Coded Storage

We consider storage systems in which K files are stored over N nodes. A ...
research
07/27/2020

FASTA/Q Data Compressors for MapReduce-Hadoop Genomics:Space and Time Savings Made Easy – Version 1

Motivation: Storage of genomic data is a major cost for the Life Science...
research
05/11/2022

Studying Scientific Data Lifecycle in On-demand Distributed Storage Caches

The XRootD system is used to transfer, store, and cache large datasets f...
research
03/07/2023

The Service Rate Region Polytope

We investigate the properties of a family of polytopes that naturally ar...
research
02/18/2020

Characterizing Synchronous Writes in Stable Memory Devices

Distributed algorithms that operate in the fail-recovery model rely on t...
research
01/24/2019

SimFS: A Simulation Data Virtualizing File System Interface

Nowadays simulations can produce petabytes of data to be stored in paral...

Please sign up or login with your details

Forgot password? Click here to reset