RFaaS: RDMA-Enabled FaaS Platform for Serverless High-Performance Computing

06/25/2021
by   Marcin Copik, et al.
0

The rigid MPI programming model and batch scheduling dominate high-performance computing. While clouds brought new levels of elasticity into the world of computing, supercomputers still suffer from low resource utilization rates. To enhance supercomputing clusters with the benefits of serverless computing, a modern cloud programming paradigm for pay-as-you-go execution of stateless functions, we present rFaaS, the first RDMA-aware Function-as-a-Service (FaaS) platform. With hot invocations and decentralized function placement, we overcome the major performance limitations of FaaS systems and provide low-latency remote invocations in multi-tenant environments. We evaluate the new serverless system through a series of microbenchmarks and show that remote functions execute with negligible performance overheads. We demonstrate how serverless computing can bring elastic resource management into MPI-based high-performance applications. Overall, our results show that MPI applications can benefit from modern cloud programming paradigms to guarantee high performance at lower resource costs.

READ FULL TEXT

Authors

page 3

page 4

page 7

09/16/2020

Extending SLURM for Dynamic Resource-Aware Adaptive Batch Scheduling

With the growing constraints on power budget and increasing hardware fai...
11/12/2018

Comparing Spark vs MPI/OpenMP On Word Count MapReduce

Spark provides an in-memory implementation of MapReduce that is widely u...
08/14/2019

Serverless Supercomputing: High Performance Function as a Service for Science

Growing data volumes and velocities are driving exciting new methods acr...
09/07/2019

Elastic Remote Methods

For distributed applications to take full advantage of cloud computing s...
05/28/2021

Performance Evaluation of Snapshot Methods to Warm the Serverless Cold Start

The serverless computing model strengthens the cloud computing tendency ...
08/04/2021

Two-Chains: High Performance Framework for Function Injection and Execution

Some important problems, such as semantic graph analysis, require large-...
05/13/2018

Building Near-Real-Time Processing Pipelines with the Spark-MPI Platform

Advances in detectors and computational technologies provide new opportu...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.