A Fundamental Storage-Communication Tradeoff in Distributed Computing with Straggling Nodes

01/23/2019
by   Qifa Yan, et al.
0

The optimal storage-computation tradeoff is characterized for a MapReduce-like distributed computing system with straggling nodes, where only a part of the nodes can be utilized to compute the desired output functions. The result holds for arbitrary output functions and thus generalizes previous results that restricted to linear functions. Specifically, in this work, we propose a new information-theoretical converse and a new matching coded computing scheme, that we call coded computing for straggling systems (CCS).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/26/2018

A Storage-Computation-Communication Tradeoff for Distributed Computing

This paper investigates distributed computing systems where computations...
research
01/10/2023

A Fundamental Tradeoff Among Storage, Computation, and Communication for Distributed Computing over Star Network

Coded distributed computing can alleviate the communication load by leve...
research
07/25/2021

Low-bandwidth recovery of linear functions of Reed-Solomon-encoded data

We study the problem of efficiently computing on encoded data. More spec...
research
01/10/2022

Successive Approximation Coding for Distributed Matrix Multiplication

Coded distributed computing was recently introduced to mitigate the effe...
research
01/17/2022

Universal Coded Distributed Computing For MapReduce Frameworks

Coded distributed computing (CDC) can trade extra computing power to red...
research
10/29/2019

The Tradeoff Between Coverage and Computation in Wireless Networks

We consider a distributed edge computing scenario consisting of several ...
research
09/07/2022

Explicit Low-Bandwidth Evaluation Schemes for Weighted Sums of Reed-Solomon-Coded Symbols

Motivated by applications in distributed storage, distributed computing,...

Please sign up or login with your details

Forgot password? Click here to reset