Scalable Model-Based Management of Correlated Dimensional Time Series in ModelarDB

03/25/2019
by   Søren Kejser Jensen, et al.
0

To monitor critical infrastructure, high quality sensors sampled at a high frequency are increasingly installed. However, due to the big amounts of data produced, only simple aggregates are stored. This removes outliers and hides fluctuations that could indicate problems. As a solution we propose compressing time series with dimensions using a model-based method we name Multi-model Group Compression (MMGC). MMGC adaptively compresses groups of correlated time series with dimensions using an extensible set of models within a user-defined error bound (possibly zero). To partition time series into groups, we propose a set of primitives for efficiently describing correlation for data sets of varying sizes. We also propose efficient query processing algorithms for executing multi-dimensional aggregate queries on models instead of data points. Last, we provide an open-source implementation of our methods as extensions to the model-based Time Series Management System (TSMS) ModelarDB. ModelarDB interfaces with the stock versions of Apache Spark and Apache Cassandra and thus can reuse existing infrastructure. Through an evaluation we show that, compared to widely used systems, our extended ModelarDB provides up to 11 times faster ingestion due to high compression, 65 times better compression due to the adaptivity of MMGC, 92 times faster aggregate queries as they are executed on models, and close to linear scalability while also being extensible and supporting online query processing.

READ FULL TEXT
research
06/29/2023

TimeClave: Oblivious In-enclave Time series Processing System

Cloud platforms are widely adopted by many systems, such as time series ...
research
05/27/2018

Measuring Congruence on High Dimensional Time Series

A time series is a sequence of data items; typical examples are videos, ...
research
01/07/2019

A Compact Representation of Raster Time Series

The raster model is widely used in Geographic Information Systems to rep...
research
11/08/2018

TimeCrypt: A Scalable Private Time Series Data Store

We present TimeCrypt, an efficient and scalable system that augments tim...
research
03/12/2021

Visualising Deep Network's Time-Series Representations

Despite the popularisation of the machine learning models, more often th...
research
08/14/2018

Plato: Approximate Analytics over Compressed Time Series with Tight Deterministic Error Guarantees

Plato provides sound and tight deterministic error guarantees for approx...
research
03/02/2020

Bridging the Gap Between Theory and Practice on Insertion-Intensive Database

With the prevalence of online platforms, today, data is being generated ...

Please sign up or login with your details

Forgot password? Click here to reset