Stark: Fast and Scalable Strassen's Matrix Multiplication using Apache Spark

by   Chandan Misra, et al.

This paper presents a new fast, highly scalable distributed matrix multiplication algorithm on Apache Spark, called Stark, based on Strassen's matrix multiplication algorithm. Stark preserves Strassen's 7 multiplications scheme in a distributed environment and thus achieves faster execution. It is based on two new ideas; it creates a recursion tree of computation where each level of such tree corresponds to division and combination of distributed matrix blocks in the form of Resilient Distributed Datasets(RDDs); It processes each divide and combine step in parallel and memorize the sub-matrices by intelligently tagging matrix blocks in it. To the best of our knowledge, Stark is the first Strassen's implementation in Spark platform. We show experimentally that Stark has a strong scalability with increasing matrix size enabling us to multiply two (16384 x 16384) matrices with 28 clock time than Marlin and MLLib respectively, state-of-the-art matrix multiplication approaches based on Spark.



There are no comments yet.


page 6

page 16

page 17


Approximate Weighted CR Coded Matrix Multiplication

One of the most common, but at the same time expensive operations in lin...

Inv-ASKIT: A Parallel Fast Diret Solver for Kernel Matrices

We present a parallel algorithm for computing the approximate factorizat...

Distributed-memory ℋ-matrix Algebra I: Data Distribution and Matrix-vector Multiplication

We introduce a data distribution scheme for ℋ-matrices and a distributed...

A parallel structured divide-and-conquer algorithm for symmetric tridiagonal eigenvalue problems

In this paper, a parallel structured divide-and-conquer (PSDC) eigensolv...

Localized sketching for matrix multiplication and ridge regression

We consider sketched approximate matrix multiplication and ridge regress...

JAMPI: efficient matrix multiplication in Spark using Barrier Execution Mode

The new barrier mode in Apache Spark allows embedding distributed deep l...

MatRox: A Model-Based Algorithm with an Efficient Storage Format for Parallel HSS-Structured Matrix Approximations

We present MatRox, a novel model-based algorithm and implementation of H...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.