Theoretical Model and Practical Considerations for Data Lineage Reconstruction

01/30/2020

∙

by   Egor Pushkin, et al.

∙

0

∙

share

We live in a world driven by data. The amount of it outgrows anyone's ability to oversee it or even observe its scope. Along with all the advances in the space of data management, there is still a significant lack of formalism and standardization around defining data ecosystems and processes occurring within those. In order to address the issue we propose a notation for data flow modeling and evaluate some of the most common applications of it based on real-world use cases. To facilitate future work, we provide detailed reference of the data model we defined and consider potential programming paradigms.