Sparse Matrix Multiplication with Bandwidth Restricted All-to-All Communication
share
We show how to multiply two n × n matrices over semirings in the Congested Clique model, where n nodes synchronously communicate in an all-to-all manner using O( n)-bit messages, within a round complexity that depends on the number of non-zero elements in the input matrices. By leveraging the sparsity of the input matrices, our algorithm reduces communication costs and thus improves upon the state-of-the-art for matrices with o(n^2) nonzero elements. Moreover, our algorithm exhibits the additional strength of surpassing previous solutions also in the case where only one of the two matrices is such. Particularly, this allows to efficiently raise a sparse matrix to a power greater than 2. As applications, we show how to speed up the computation on non-dense graphs of 3- and 4-cycle counting, as well as of all-pairs-shortest-paths. Our algorithmic contribution is a new deterministic method of restructuring the input matrices in a sparsity-aware manner, which assigns each node with element-wise multiplication tasks that are not necessarily consecutive but guarantee a balanced element distribution, providing for communication-efficient multiplication. As such, our technique may be useful in additional computational models.
READ FULL TEXT
