Coded Computing for Distributed Graph Analytics

01/17/2018
by   Saurav Prakash, et al.
0

Many distributed graph computing systems have been developed recently for efficient processing of massive graphs. These systems require many messages to be exchanged among computing machines at each step of the computation, making communication bandwidth a major performance bottleneck. We present a coded computing framework that systematically injects redundancy in the computation phase to enable coding opportunities in the communication phase thus reducing the communication load substantially. Specifically, we propose coded schemes that enable an inverse-linear trade-off (asymptotically) between computation load and average communication load for three popular random graphs -- Erdös-Rényi (ER), random bi-partite (RB), stochastic block model (SBM). The proposed scheme for ER graph is shown to be optimal asymptotically as the graph size n→∞. For finite n, we demonstrate via numerical analysis that for a given computation load r, i.e. when each graph node is carefully stored at r servers, the proposed scheme slashes the average communication load by (nearly) r.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/03/2018

Wireless MapReduce Distributed Computing

Motivated by mobile edge computing and wireless data centers, we study a...
research
05/13/2020

Improved Computation-Communication Trade-Off for Coded Distributed Computing using Linear Dependence of Intermediate Values

In large scale distributed computing systems, communication overhead is ...
research
02/27/2019

Coded Distributed Computing with Heterogeneous Function Assignments

Coded distributed computing (CDC) introduced by Li et. al. is an effecti...
research
05/05/2018

Compressed Coded Distributed Computing

Communication overhead is one of the major performance bottlenecks in la...
research
08/19/2019

Heterogeneous Coded Distributed Computing: Joint Design of File Allocation and Function Assignment

This paper studies the computation-communication tradeoff in a heterogen...
research
06/26/2022

Multi-Access Distributed Computing

Coded distributed computing (CDC) is a new technique proposed with the p...
research
02/09/2018

Communication-Computation Efficient Gradient Coding

This paper develops coding techniques to reduce the running time of dist...

Please sign up or login with your details

Forgot password? Click here to reset