DeepAI AI Chat
Log In Sign Up

Heterogeneous Coded Computation across Heterogeneous Workers

04/16/2019
by   Yuxuan Sun, et al.
Imperial College London
0

Coded distributed computing framework enables large-scale machine learning (ML) models to be trained efficiently in a distributed manner, while mitigating the straggler effect. In this work, we consider a multi-task assignment problem in a coded distributed computing system, where multiple masters, each with a different matrix multiplication task, assign computation tasks to workers with heterogeneous computing capabilities. Both dedicated and probabilistic worker assignment models are considered, with the objective of minimizing the average completion time of all computations. For dedicated worker assignment, greedy algorithms are proposed and the corresponding optimal load allocation is derived based on the Lagrange multiplier method. For probabilistic assignment, successive convex approximation method is used to solve the non-convex optimization problem. Simulation results show that the proposed algorithms reduce the completion time by 60 unbalanced coded scheme.

READ FULL TEXT
04/16/2019

Heterogeneous Computation across Heterogeneous Workers

Coded distributed computing framework enables large-scale machine learni...
09/23/2021

Coded Computation across Shared Heterogeneous Workers with Communication Delay

Distributed computing enables large-scale computation tasks to be proces...
12/29/2019

On Batch-Processing Based Coded Computing for Heterogeneous Distributed Computing Systems

In recent years, coded distributed computing (CDC) has attracted signifi...
11/22/2017

Combating Computational Heterogeneity in Large-Scale Distributed Computing via Work Exchange

Owing to data-intensive large-scale applications, distributed computatio...
01/20/2020

Bivariate Polynomial Coding for Exploiting Stragglers in Heterogeneous Coded Computing Systems

Polynomial coding has been proposed as a solution to the straggler mitig...
08/12/2020

Coded Elastic Computing on Machines with Heterogeneous Storage and Computation Speed

We study the optimal design of heterogeneous Coded Elastic Computing (CE...
12/17/2021

Coded Consensus Monte Carlo: Robust One-Shot Distributed Bayesian Learning with Stragglers

This letter studies distributed Bayesian learning in a setting encompass...