Optimizing Co-flows Scheduling and Routing in Data Centre Networks for Big Data Applications

by   Sanaa Hamid Mohamed, et al.

This paper optimizes the scheduling and routing of the co-flows of MapReduce shuffling phase in state-of-the-art and proposed Passive Optical Networking (PON)-based Data Centre Network (DCN) architectures. A time-slotted Mixed Integer Linear Programming (MILP) model is developed and used for the optimization with the objective of minimizing either the total energy consumption or the completion time. The DCN architectures include four state-of-the-art electronic switching architectures which are spine-leaf, Fat-tree, BCube, and DCell data centres. The proposed PON-based DCN architectures include two designs that utilize ports in Optical Line Terminal (OLT) line cards for inter and possibly intra data centre networking in addition to passive interconnects for the intra data centre networking between different PON groups (i.e. racks) within a PON cell (i.e. number of PON groups connected to a single OLT port). The first design is a switch-centric design that uses two Arrayed Waveguide Grating Routers (AWGRs) and the second is a server-centric design. The study also considers different traffic patterns defined according to the distribution of map and reduce tasks in the servers and data skewness.


page 1

page 18


Energy Efficiency of Server-Centric PON Data Center Architecture for Fog Computing

In this paper, we utilize Mixed Integer Linear Programming (MILP) models...

Experimental Evaluation of Server Centric Passive Optical Network Based Data Centre Architecture

Passive optical networks (PON) technology has recently been proposed as ...

Optimized Passive Optical Networks with Cascaded-AWGRs for Data Centers

The use of Passive Optical Networks (PONs) in modern and future data cen...

Experimental Evaluation of Passive Optical Network Based Data Centre Architecture

Passive optical networks (PON) technology is increasingly becoming an at...

A Fast-Convergence Routing of the Hot-Potato

Interactions between the intra- and inter-domain routing protocols recei...

Energy Efficient and Resilient Infrastructure for Fog Computing Health Monitoring Applications

In this paper, we propose a resilient energy efficient and fog computing...

Coflow Scheduling in Data Centers: Routing and Bandwidth Allocation

In distributed computing frameworks like MapReduce, Spark, and Dyrad, a ...