CFS: A Distributed File System for Large Scale Container Platforms

11/08/2019
by   Haifeng Liu, et al.
0

We propose CFS, a distributed file system for large scale container platforms. CFS supports both sequential and random file accesses with optimized storage for both large files and small files, and adopts different replication protocols for different write scenarios to improve the replication performance. It employs a metadata subsystem to store and distribute the file metadata across different storage nodes based on the memory usage. This metadata placement strategy avoids the need of data rebalancing during capacity expansion. CFS also provides POSIX-compliant APIs with relaxed semantics and metadata atomicity to improve the system performance. We performed a comprehensive comparison with Ceph, a widely-used distributed file system on container platforms. Our experimental results show that, in testing 7 commonly used metadata operations, CFS gives around 3 times performance boost on average. In addition, CFS exhibits better random-read/write performance in highly concurrent environments with multiple clients and processes.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/26/2017

Analyzing IO Amplification in Linux File Systems

We present the first systematic analysis of read, write, and space ampli...
research
03/14/2019

Hadoop Perfect File: A fast access container for small files with direct in disc metadata access

Storing and processing massive small files is one of the major challenge...
research
02/14/2020

Deploying large fixed file datasets with SquashFS and Singularity

Shared high-performance computing (HPC) platforms, such as those provide...
research
06/20/2023

λFS: A Scalable and Elastic Distributed File System Metadata Service using Serverless Functions

The metadata service (MDS) sits on the critical path for distributed fil...
research
08/16/2020

Dependability Evaluation of Middleware Technology for Large-scale Distributed Caching

Distributed caching systems (e.g., Memcached) are widely used by service...
research
01/24/2023

FUSEE: A Fully Memory-Disaggregated Key-Value Store (Extended Version)

Distributed in-memory key-value (KV) stores are embracing the disaggrega...
research
10/05/2020

Metadata-Based Detection of Child Sexual Abuse Material

In the last decade, the scale of creation and distribution of child sexu...

Please sign up or login with your details

Forgot password? Click here to reset