SeBS: A Serverless Benchmark Suite for Function-as-a-Service Computing

by   Marcin Copik, et al.

Function-as-a-Service (FaaS) is one of the most promising directions for the future of cloud services, and serverless functions have immediately become a new middleware for building scalable and cost-efficient microservices and applications. However, the quickly moving technology hinders reproducibility, and the lack of a standardized benchmarking suite leads to ad-hoc solutions and microbenchmarks being used in serverless research, further complicating metaanalysis and comparison of research solutions. To address this challenge, we propose the Serverless Benchmark Suite: the first benchmark for FaaS computing that systematically covers a wide spectrum of cloud resources and applications. Our benchmark consists of the specification of representative workloads, the accompanying implementation and evaluation infrastructure, and the evaluation methodology that facilitates reproducibility and enables interpretability. We demonstrate that the abstract model of a FaaS execution environment ensures the applicability of our benchmark to multiple commercial providers such as AWS, Azure, and Google Cloud. Our work facilities experimental evaluation of serverless systems, and delivers a standardized, reliable and evolving evaluation methodology of performance, efficiency, scalability and reliability of middleware FaaS platforms.



page 2

page 3


SupermarQ: A Scalable Quantum Benchmark Suite

The emergence of quantum computers as a new computational paradigm has b...

Using HEP experiment workflows for the benchmarking and accounting of WLCG computing resources

Benchmarking of CPU resources in WLCG has been based on the HEP-SPEC06 (...

FaaSdom: A Benchmark Suite for Serverless Computing

Serverless computing has become a major trend among cloud providers. Wit...

Boosting Metrics for Cloud Services Evaluation -- The Last Mile of Using Benchmark Suites

Benchmark suites are significant for evaluating various aspects of Cloud...

Towards Demystifying Intra-Function Parallelism in Serverless Computing

Serverless computing offers a pay-per-use model with high elasticity and...

Streaming vs. Functions: A Cost Perspective on Cloud Event Processing

In cloud event processing, data generated at the edge is processed in re...

Function-as-a-Service Performance Evaluation: A Multivocal Literature Review

Function-as-a-Service (FaaS) is one form of the serverless cloud computi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.