SeBS: A Serverless Benchmark Suite for Function-as-a-Service Computing

12/28/2020
by   Marcin Copik, et al.
0

Function-as-a-Service (FaaS) is one of the most promising directions for the future of cloud services, and serverless functions have immediately become a new middleware for building scalable and cost-efficient microservices and applications. However, the quickly moving technology hinders reproducibility, and the lack of a standardized benchmarking suite leads to ad-hoc solutions and microbenchmarks being used in serverless research, further complicating metaanalysis and comparison of research solutions. To address this challenge, we propose the Serverless Benchmark Suite: the first benchmark for FaaS computing that systematically covers a wide spectrum of cloud resources and applications. Our benchmark consists of the specification of representative workloads, the accompanying implementation and evaluation infrastructure, and the evaluation methodology that facilitates reproducibility and enables interpretability. We demonstrate that the abstract model of a FaaS execution environment ensures the applicability of our benchmark to multiple commercial providers such as AWS, Azure, and Google Cloud. Our work facilities experimental evaluation of serverless systems, and delivers a standardized, reliable and evolving evaluation methodology of performance, efficiency, scalability and reliability of middleware FaaS platforms.

READ FULL TEXT

Authors

page 2

page 3

02/22/2022

SupermarQ: A Scalable Quantum Benchmark Suite

The emergence of quantum computers as a new computational paradigm has b...
04/03/2020

Using HEP experiment workflows for the benchmarking and accounting of WLCG computing resources

Benchmarking of CPU resources in WLCG has been based on the HEP-SPEC06 (...
06/05/2020

FaaSdom: A Benchmark Suite for Serverless Computing

Serverless computing has become a major trend among cloud providers. Wit...
08/04/2017

Boosting Metrics for Cloud Services Evaluation -- The Last Mile of Using Benchmark Suites

Benchmark suites are significant for evaluating various aspects of Cloud...
10/22/2021

Towards Demystifying Intra-Function Parallelism in Serverless Computing

Serverless computing offers a pay-per-use model with high elasticity and...
04/25/2022

Streaming vs. Functions: A Cost Perspective on Cloud Event Processing

In cloud event processing, data generated at the edge is processed in re...
04/07/2020

Function-as-a-Service Performance Evaluation: A Multivocal Literature Review

Function-as-a-Service (FaaS) is one form of the serverless cloud computi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.