Reactive Liquid: Optimized Liquid Architecture for Elastic and Resilient Distributed Data Processing

02/15/2019
by   Seyed Esmaeil Mirvakili, et al.
0

Today's most prominent IT companies are built on the extraction of insight from data, and data processing has become crucial in data-intensive businesses. Nevertheless, the size of data which should be processed is growing significantly fast. The pace of the data growing has changed the nature of data processing. Today, data-intensive industries demand highly scalable and fault tolerant data processing architectures which can handle the massive amount of data. In this paper, we presented a distributed architecture for elastic and resilient data processing based on the Liquid which is a nearline and offline big data architecture. We used the Reactive Manifesto to design the architecture highly reactive to workload changes and failures. We evaluate our architecture by drawing some numerical comparisons between our architecture prototype and the Liquid prototype. The performed evaluation shows that our architecture can be more scalable against workload and more resilient against failures than the Liquid architecture is.

READ FULL TEXT
research
07/06/2017

A Survey on Geographically Distributed Big-Data Processing using MapReduce

Hadoop and Spark are widely used distributed processing frameworks for l...
research
01/07/2021

Towards a Smart Data Processing and Storage Model

In several domains it is crucial to store and manipulate data whose orig...
research
10/22/2021

RDD-Eclat: Approaches to Parallelize Eclat Algorithm on Spark RDD Framework (Extended Version)

Frequent itemset mining (FIM) is a highly computational and data intensi...
research
12/13/2019

RDD-Eclat: Approaches to Parallelize Eclat Algorithm on Spark RDD Framework

Initially, a number of frequent itemset mining (FIM) algorithms have bee...
research
05/04/2018

SecureStreams: A Reactive Middleware Framework for Secure Data Stream Processing

The growing adoption of distributed data processing frameworks in a wide...
research
04/11/2020

Adaptive HTAP through Elastic Resource Scheduling

Modern Hybrid Transactional/Analytical Processing (HTAP) systems use an ...
research
11/06/2017

Game Theoretic Approaches to Massive Data Processing in Wireless Networks

Wireless communication networks are becoming highly virtualized with two...

Please sign up or login with your details

Forgot password? Click here to reset