Resource Management and Scheduling for Big Data Applications in Cloud Computing Environments

12/03/2018
by   Muhammed Tawfiqul Islam, et al.
0

This chapter presents software architectures of the big data processing platforms. It will provide an in-depth knowledge on resource management techniques involved while deploying big data processing systems on cloud environment. It starts from the very basics and gradually introduce the core components of resource management which we have divided in multiple layers. It covers the state-of-art practices and researches done in SLA-based resource management with a specific focus on the job scheduling mechanisms.

READ FULL TEXT
research
01/19/2018

A hybrid architecture for astronomical computing

With many large science equipment constructing and putting into use, ast...
research
08/11/2020

DV-ARPA: Data Variety Aware Resource Provisioning for Big Data Processing in Accumulative Applications

In Cloud Computing, the resource provisioning approach used has a great ...
research
07/05/2022

Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing

Big data processing at the production scale presents a highly complex en...
research
11/04/2022

Rethinking Storage Management for Data Processing Pipelines in Cloud Data Centers

Data processing frameworks such as Apache Beam and Apache Spark are used...
research
04/10/2019

R-Storm: Resource-Aware Scheduling in Storm

The era of big data has led to the emergence of new systems for real-tim...
research
12/05/2016

Support vector regression model for BigData systems

Nowadays Big Data are becoming more and more important. Many sectors of ...
research
08/05/2023

Resource Management for GPT-based Model Deployed on Clouds: Challenges, Solutions, and Future Directions

The widespread adoption of the large language model (LLM), e.g. Generati...

Please sign up or login with your details

Forgot password? Click here to reset