DeepAI AI Chat
Log In Sign Up

To Ship or Not to (Function) Ship (Extended version)

by   Feilong Liu, et al.
The Ohio State University

Sampling is often used to reduce query latency for interactive big data analytics. The established parallel data processing paradigm relies on function shipping, where a coordinator dispatches queries to worker nodes and then collects the results. The commoditization of high-performance networking makes data shipping possible, where the coordinator directly reads data in the workers' memory using RDMA while workers process other queries. In this work, we explore when to use function shipping or data shipping for interactive query processing with sampling. Whether function shipping or data shipping should be preferred depends on the amount of data transferred, the current CPU utilization, the sampling method and the number of queries executed over the data set. The results show that data shipping is up to 6.5x faster when performing clustered sampling with heavily-utilized workers.


page 1

page 2

page 3

page 4


Approximate Query Processing for Group-By Queries based on Conditional Generative Models

The Group-By query is an important kind of query, which is common and wi...

Reinforced Approximate Exploratory Data Analysis

Exploratory data analytics (EDA) is a sequential decision making process...

EntropyDB: A Probabilistic Approach to Approximate Query Processing

We present EntropyDB, an interactive data exploration system that uses a...

Starling: A Scalable Query Engine on Cloud Function Services

Much like on-premises systems, the natural choice for running database a...

Lambada: Interactive Data Analytics on Cold Data using Serverless Cloud Infrastructure

The promise of ultimate elasticity and operational simplicity of serverl...

Similarity Driven Approximation for Text Analytics

Text analytics has become an important part of business intelligence as ...

Approximating Aggregated SQL Queries With LSTM Networks

Despite continuous investments in data technologies, the latency of quer...