Faabric: Fine-Grained Distribution of Scientific Workloads in the Cloud

02/22/2023
by   Simon Shillaker, et al.
0

With their high parallelism and resource needs, many scientific applications benefit from cloud deployments. Today, scientific applications are executed on dedicated pools of VMs, resulting in resource fragmentation: users pay for underutilised resources, and providers cannot reallocate unused resources between applications. While serverless cloud computing could address these issues, its programming model is incompatible with the use of shared memory and message passing in scientific applications: serverless functions do not share memory directly on the same VM or support message passing semantics when scheduling functions dynamically. We describe Faabric, a new serverless cloud runtime that transparently distributes applications with shared memory and message passing across VMs. Faabric achieves this by scheduling computation in a fine-grained (thread/process) fashion through a new execution abstraction called Granules. To support shared memory, Granules are isolated using WebAssembly but share memory directly; to support message passing, Granules offer asynchronous point-to-point communication. Faabric schedules Granules to meet an application's parallelism needs. It also synchronises changes to Granule's shared memory, and migrates Granules to improve locality.

READ FULL TEXT
research
05/15/2023

FMI: Fast and Cheap Message Passing for Serverless Functions

Serverless functions provide elastic scaling and a fine-grained billing ...
research
02/11/2020

Back to Futures

We briefly introduce the semi-axiomatic sequent calculus for linear logi...
research
05/14/2021

Impossibility of Strongly-Linearizable Message-Passing Objects via Simulation by Single-Writer Registers

A key way to construct complex distributed systems is through modular co...
research
12/20/2020

Optimal Resilience in Systems that Mix Shared Memory and Message Passing

We investigate the minimal number of failures that can partition a syste...
research
12/12/2012

Distributed Planning in Hierarchical Factored MDPs

We present a principled and efficient planning algorithm for collaborati...
research
09/16/2020

A FaaS File System for Serverless Computing

Serverless computing with cloud functions is quickly gaining adoption, b...
research
05/16/2018

Towards Complex Artificial Life

An object-oriented combinator chemistry was used to construct an artific...

Please sign up or login with your details

Forgot password? Click here to reset