Affinity Scheduling and the Applications on Data Center Scheduling with Data Locality

05/09/2017
by   Mohammadamir Kavousi, et al.
0

MapReduce framework is the de facto standard in Hadoop. Considering the data locality in data centers, the load balancing problem of map tasks is a special case of affinity scheduling problem. There is a huge body of work on affinity scheduling, proposing heuristic algorithms which try to increase data locality in data centers like Delay Scheduling and Quincy. However, not enough attention has been put on theoretical guarantees on throughput and delay optimality of such algorithms. In this work, we present and compare different algorithms and discuss their shortcoming and strengths. To the best of our knowledge, most data centers are using static load balancing algorithms which are not efficient in any ways and results in wasting the resources and causing unnecessary delays for users.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/23/2017

GB-PANDAS: Throughput and heavy-traffic optimality analysis for affinity scheduling

Dynamic affinity scheduling has been an open problem for nearly three de...
research
03/31/2019

The Power of d Choices in Scheduling for Data Centers with Heterogeneous Servers

MapReduce framework is the de facto in big data and its applications whe...
research
01/13/2019

Blind GB-PANDAS: A Blind Throughput-Optimal Load Balancing Algorithm for Affinity Scheduling

Dynamic affinity load balancing of multi-type tasks on multi-skilled ser...
research
03/31/2021

Scheduling Applications on Containers Based on Dependency of The Applications

Cloud computing technology has been one of the most critical development...
research
08/24/2018

Hybrid Job-driven Scheduling for Virtual MapReduce Clusters

It is cost-efficient for a tenant with a limited budget to establish a v...
research
02/26/2020

A Disk Scheduling Algorithm Based on ANT Colony Optimization

Audio, animations and video belong to a class of data known as delay sen...
research
12/12/2016

Geographical Load Balancing across Green Datacenters

"Geographic Load Balancing" is a strategy for reducing the energy cost o...

Please sign up or login with your details

Forgot password? Click here to reset