Concept-oriented model: Modeling and processing data using functions

11/17/2019
by   Alexandr Savinov, et al.
0

We describe a new logical data model, called the concept-oriented model (COM). It uses mathematical functions as first-class constructs for data representation and data processing as opposed to using exclusively sets in conventional set-oriented models. Functions and function composition are used as primary semantic units for describing data connectivity instead of relations and relation composition (join), respectively. Grouping and aggregation are also performed by using (accumulate) functions providing an alternative to group-by and reduce operations. This model was implemented in an open source data processing toolkit examples of which are used to illustrate the model and its operations. The main benefit of this model is that typical data processing tasks become simpler and more natural when using functions in comparison to adopting sets and set operations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/31/2020

On the importance of functions in data modeling

In this paper we argue that representing entity properties by tuple attr...
research
08/05/2021

An Abstract View of Big Data Processing Programs

This paper proposes a model for specifying data flow based parallel data...
research
06/16/2021

mPyPl: Python Monadic Pipeline Library for Complex Functional Data Processing

In this paper, we present a new Python library called mPyPl, which is in...
research
08/19/2022

Resource Allocation in Serverless Query Processing

Data lakes hold a growing amount of cold data that is infrequently acces...
research
03/07/2023

A Step Toward Deep Online Aggregation (Extended Version)

For exploratory data analysis, it is often desirable to know what answer...
research
03/07/2016

Position paper: Towards an observer-oriented theory of shape comparison

In this position paper we suggest a possible metric approach to shape co...
research
12/18/2017

Error-Tolerant Big Data Processing

Real-world data contains various kinds of errors. Before analyzing data,...

Please sign up or login with your details

Forgot password? Click here to reset