AlertMix: A Big Data platform for multi-source streaming data

06/18/2018
by   Ayush Singhal, et al.
0

The demand for stream processing is increasing at an unprecedented rate. Big data is no longer limited to processing of big volumes of data. In most real-world scenarios, the need for processing stream data as it comes can only meet the business needs. It is required for trading, fraud detection, system monitoring, product maintenance and of course social media data such as Twitter and YouTube videos. In such cases, a "too late architecture" that focuses on batch processing cannot realize the use cases. In this article, we present an end to end Big data platform called AlertMix for processing multi-source streaming data. Its architecture and how various Big data technologies are utilized are explained in this work. We present the performance of our platform on real live streaming data which is currently handled by the platform.

READ FULL TEXT

Authors

page 4

page 7

09/14/2017

Scalable real-time processing with Spark Streaming: implementation and design of a Car Information System

Streaming data processing is a hot topic in big data these days, because...
01/25/2019

A quality model for evaluating and choosing a stream processing framework architecture

Today, we have to deal with many data (Big data) and we need to make dec...
12/08/2019

A study on Modern Messaging Systems- Kafka, RabbitMQ and NATS Streaming

Distributed messaging systems form the core of big data streaming, cloud...
08/31/2017

Identifying Unsafe Videos on Online Public Media using Real-time Crowdsourcing

Due to the significant growth of social networking and human activities ...
09/29/2019

Context agnostic trajectory prediction based on λ-architecture

Predicting the next position of movable objects has been a problem for a...
08/09/2021

Towards a Generic Multimodal Architecture for Batch and Streaming Big Data Integration

Big Data are rapidly produced from various heterogeneous data sources. T...
02/01/2019

Incremental Techniques for Large-Scale Dynamic Query Processing

Many applications from various disciplines are now required to analyze f...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.