The Art of the Meta Stream Protocol: Torrents of Streams

by   Christophe De Troyer, et al.

The rise of streaming libraries such as Akka Stream, Reactive Extensions, and LINQ popularized the declarative functional style of data processing. The stream paradigm offers concise syntax to write down processing pipelines to consume the vast amounts of real-time data available today. These libraries offer the programmer a domain specific language (DSL) embedded in the host language to describe data streams. These libraries however, all suffer from extensibility issues. The semantics of a stream is hard-coded into the DSL language and cannot be changed by the user of the library. We introduce an approach to modify the semantics of a streaming library by means of meta-programming at both run-time and compile-time, and showcase its generality. We show that the expressiveness of the meta-facilities is strong enough to enable push and pull semantics, error handling, parallelism, and operator fusion. We evaluate our work by implementing the identified shortcomings in terms of a novel stream meta-architecture and show that its design and architecture adhere to the design principles of a meta-level architecture. The state of the art offers plenty of choice to programmers regarding reactive stream processing libraries. Expressing reactive systems is otherwise difficult to do in general purpose languages. Extensibility and fine-tuning should be possible in these libraries to ensure a broad variety of applications can be expressed within this single DSL.



There are no comments yet.


page 7

page 14

page 30

page 32

page 33

page 34

page 35

page 36


SecureStreams: A Reactive Middleware Framework for Secure Data Stream Processing

The growing adoption of distributed data processing frameworks in a wide...

Fast Concurrent Data Sketches

Data sketches are approximate succinct summaries of long streams. They a...

Juniper: A Functional Reactive Programming Language for the Arduino

This paper presents the design and implementation of Juniper: a function...

Reactive Probabilistic Programming

Synchronous reactive languages were introduced for designing and impleme...

Towards Ideal Semantics for Analyzing Stream Reasoning

The rise of smart applications has drawn interest to logical reasoning o...

Lazy Stream Programming in Prolog

In recent years, stream processing has become a prominent approach for i...

mPyPl: Python Monadic Pipeline Library for Complex Functional Data Processing

In this paper, we present a new Python library called mPyPl, which is in...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.