DeepAI AI Chat
Log In Sign Up

ACCORDANT: A Domain Specific Model and DevOpsApproach for Big Data Analytics Architectures

11/16/2020
by   Camilo Castellanos, et al.
0

Big data analytics (BDA) applications use machine learning algorithms to extract valuable insights from large, fast, and heterogeneous data sources. New software engineering challenges for BDA applications include ensuring performance levels of data-driven algorithms even in the presence of large data volume, velocity, and variety (3Vs). BDA software complexity frequently leads to delayed deployments, longer development cycles and challenging performance assessment. This paper proposes a Domain-Specific Model (DSM), and DevOps practices to design, deploy, and monitor performance metrics in BDA applications. Our proposal includes a design process, and a framework to define architectural inputs, software components, and deployment strategies through integrated high-level abstractions to enable QS monitoring. We evaluate our approach with four use cases from different domains to demonstrate a high level of generalization. Our results show a shorter deployment and monitoring times, and a higher gain factor per iteration compared to similar approaches.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/01/2018

Big Data Dwarfs: Towards Fully Understanding Big Data Analytics Workloads

Though the big data benchmark suites like BigDataBench and CloudSuite ha...
03/06/2021

EVEREST: A design environment for extreme-scale big data analytics on heterogeneous platforms

High-Performance Big Data Analytics (HPDA) applications are characterize...
10/27/2020

Big Data Science

In ever more disciplines, science is driven by data, which leads to data...
02/24/2016

A Survey on Domain-Specific Languages for Machine Learning in Big Data

The amount of data generated in the modern society is increasing rapidly...
04/14/2018

Data Analytics Service Composition and Deployment on Edge Devices

Data analytics on edge devices has gained rapid growth in research, indu...
10/24/2017

Fast and Scalable Distributed Deep Convolutional Autoencoder for fMRI Big Data Analytics

In recent years, analyzing task-based fMRI (tfMRI) data has become an es...
12/26/2018

Large Multistream Data Analytics for Monitoring and Diagnostics in Manufacturing Systems

The high-dimensionality and volume of large scale multistream data has i...