Timestamp tokens: a better coordination primitive for data-processing systems

10/12/2022
by   Andrea Lattuada, et al.
0

Distributed data processing systems have advanced through models that expose more and more opportunities for concurrency within a computation. The scheduling of these increasingly sophisticated models has become the bottleneck for improved throughput and reduced latency. We present a new coordination primitive for dataflow systems, the timestamp token, which minimizes the volume of information shared between the computation and host system, without surrendering precision about concurrency. Several projects have now used timestamp tokens, and were able to explore computational idioms that could not be expressed easily, if at all, in other platforms. Importantly, these projects did not need to design and implement whole systems to support their research.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/15/2022

Tokens and Distributed Ledgers in Digital Payment Systems

In this article, we consider the roles of tokens and distributed ledgers...
research
10/26/2022

A case for disaggregation of ML data processing

Machine Learning (ML) computation requires feeding input data for the mo...
research
04/25/2018

Processing Database Joins over a Shared-Nothing System of Multicore Machines

To process a large volume of data, modern data management systems use a ...
research
03/15/2022

Towards Distributed Coordination for Fog Platforms

Distributed fog and edge applications communicate over unreliable networ...
research
08/08/2023

The Vulnerable Nature of Decentralized Governance in DeFi

Decentralized Finance (DeFi) platforms are often governed by Decentraliz...
research
12/13/2021

Past, Present and Future of Computational Storage: A Survey

We live in a data-centric world where we are heading to generate close t...

Please sign up or login with your details

Forgot password? Click here to reset