A distributed file system for a wide-area high performance computing infrastructure

01/01/2010
by   Edward Walker, et al.
0

We describe our work in implementing a wide-area distributed file system for the NSF TeraGrid. The system, called XUFS, allows private distributed name spaces to be created for transparent access to personal files across over 9000 computer nodes. XUFS builds on many principles from prior distributed file systems research, but extends key design goals to support the workflow of computational science researchers. Specifically, XUFS supports file access from the desktop to the wide-area network seamlessly, survives transient disconnected operations robustly, and demonstrates comparable or better throughput than some current high performance file systems on the wide-area network.

READ FULL TEXT
research
02/14/2020

Deploying large fixed file datasets with SquashFS and Singularity

Shared high-performance computing (HPC) platforms, such as those provide...
research
04/17/2018

Deep Learning on Operational Facility Data Related to Large-Scale Distributed Area Scientific Workflows

Distributed computing platforms provide a robust mechanism to perform la...
research
12/16/2019

Proof of file access in a private P2P network using blockchain

While sharing files in a peer-to-peer (P2P) system significantly increas...
research
07/07/2021

How Big Are Peoples' Computer Files? File Size Distributions Among User-managed Collections

Improving file management interfaces and optimising system performance r...
research
06/16/2020

Distributed File System for an Edge-Based Environment

Recent developments in the industry of personal computing led to a great...
research
04/11/2023

Manuscript of a method for improving wear in intermittently computing file systems

For the first time, the repeated wear phenomenon of high-frequency power...
research
12/27/2018

Extending TCP for Accelerating Replication on Cluster File Systems over SDNs

This paper explores the changes required of TCP to efficiently support c...

Please sign up or login with your details

Forgot password? Click here to reset