Coded Alternating Least Squares for Straggler Mitigation in Distributed Recommendations

05/08/2021
by   Siyuan Wang, et al.
0

Matrix factorization is an important representation learning algorithm, e.g., recommender systems, where a large matrix can be factorized into the product of two low dimensional matrices termed as latent representations. This paper investigates the problem of matrix factorization in distributed computing systems with stragglers, those compute nodes that are slow to return computation results. A computation procedure, called coded Alternative Least Square (ALS), is proposed for mitigating the effect of stragglers in such systems. The coded ALS algorithm iteratively computes two low dimensional latent matrices by solving various linear equations, with the Entangled Polynomial Code (EPC) as a building block. We theoretically characterize the maximum number of stragglers that the algorithm can tolerate (or the recovery threshold) in relation to the redundancy of coding (or the code rate). In addition, we theoretically show the computation complexity for the coded ALS algorithm and conduct numerical experiments to validate our design.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/20/2020

On Application of Block Kaczmarz Methods in Matrix Factorization

Matrix factorization techniques compute low-rank product approximations ...
research
01/21/2019

Polar Coded Distributed Matrix Multiplication

We propose a polar coding mechanism for distributed matrix multiplicatio...
research
04/15/2019

Distributed Matrix Multiplication Using Speed Adaptive Coding

While performing distributed computations in today's cloud-based platfor...
research
11/09/2020

MatRec: Matrix Factorization for Highly Skewed Dataset

Recommender systems is one of the most successful AI technologies applie...
research
08/08/2023

Preserving Sparsity and Privacy in Straggler-Resilient Distributed Matrix Computations

Existing approaches to distributed matrix computations involve allocatin...
research
07/13/2022

Secure Linear MDS Coded Matrix Inversion

A cumbersome operation in many scientific fields, is inverting large ful...
research
09/27/2019

Identifying Low-Dimensional Structures in Markov Chains: A Nonnegative Matrix Factorization Approach

A variety of queries about stochastic systems boil down to study of Mark...

Please sign up or login with your details

Forgot password? Click here to reset