Efficient GPU implementation of randomized SVD and its applications

by   Łukasz Struski, et al.
Jagiellonian University

Matrix decompositions are ubiquitous in machine learning, including applications in dimensionality reduction, data compression and deep learning algorithms. Typical solutions for matrix decompositions have polynomial complexity which significantly increases their computational cost and time. In this work, we leverage efficient processing operations that can be run in parallel on modern Graphical Processing Units (GPUs), predominant computing architecture used e.g. in deep learning, to reduce the computational burden of computing matrix decompositions. More specifically, we reformulate the randomized decomposition problem to incorporate fast matrix multiplication operations (BLAS-3) as building blocks. We show that this formulation, combined with fast random number generators, allows to fully exploit the potential of parallel processing implemented in GPUs. Our extensive evaluation confirms the superiority of this approach over the competing methods and we release the results of this research as a part of the official CUDA implementation (https://docs.nvidia.com/cuda/cusolver/index.html).


page 1

page 2

page 3

page 4


Randomized Matrix Decompositions using R

Matrix decompositions are fundamental tools in the area of applied mathe...

CPQR-based randomized algorithms for generalized CUR decompositions

Based on the column pivoted QR decomposition, we propose some randomized...

What if Neural Networks had SVDs?

Various Neural Networks employ time-consuming matrix operations like mat...

Mixed-Precision Random Projection for RandNLA on Tensor Cores

Random projection can reduce the dimension of data while capturing its s...

Computing techniques

This lecture aims at providing a user's perspective on the main concepts...

Improving Energy Saving of One-sided Matrix Decompositions on CPU-GPU Heterogeneous Systems

One-sided dense matrix decompositions (e.g., Cholesky, LU, and QR) are t...

Spectral approximations in machine learning

In many areas of machine learning, it becomes necessary to find the eige...

Please sign up or login with your details

Forgot password? Click here to reset