ESPBench: The Enterprise Stream Processing Benchmark

03/11/2021
by   Guenter Hesse, et al.
0

Growing data volumes and velocities in fields such as Industry 4.0 or the Internet of Things have led to the increased popularity of data stream processing systems. Enterprises can leverage these developments by enriching their core business data and analyses with up-to-date streaming data. Comparing streaming architectures for these complex use cases is challenging, as existing benchmarks do not cover them. ESPBench is a new enterprise stream processing benchmark that fills this gap. We present its architecture, the benchmarking process, and the query workload. We employ ESPBench on three state-of-the-art stream processing systems, Apache Spark, Apache Flink, and Hazelcast Jet, using provided query implementations developed with Apache Beam. Our results highlight the need for the provided ESPBench toolkit that supports benchmark execution, as it enables query result validation and objective latency measures.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 8

07/18/2019

Quantitative Impact Evaluation of an Abstraction Layer for Data Stream Processing Systems

With the demand to process ever-growing data volumes, a variety of new d...
09/01/2020

Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures

Distributed stream processing engines are designed with a focus on scala...
07/22/2019

Application of Data Stream Processing Technologies in Industry 4.0 – What is Missing?

Industry 4.0 is becoming more and more important for manufacturers as th...
08/27/2018

Piecewise Linear Approximation in Data Streaming: Algorithmic Implementations and Experimental Analysis

Piecewise Linear Approximation (PLA) is a well-established tool to reduc...
04/12/2018

BigSR: an empirical study of real-time expressive RDF stream reasoning on modern Big Data platforms

The trade-off between language expressiveness and system scalability (E&...
10/06/2020

Move Fast and Meet Deadlines: Fine-grained Real-time Stream Processing with Cameo

Resource provisioning in multi-tenant stream processing systems faces th...
09/21/2020

Towards application-specific query processing systems

Database systems use query processing subsystems for enabling efficient ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.