A Unified Iteration Space Transformation Framework for Sparse and Dense Tensor Algebra

12/28/2019
by   Ryan Senanayake, et al.
0

We address the problem of optimizing mixed sparse and dense tensor algebra in a compiler. We show that standard loop transformations, such as strip-mining, tiling, collapsing, parallelization and vectorization, can be applied to irregular loops over sparse iteration spaces. We also show how these transformations can be applied to the contiguous value arrays of sparse tensor data structures, which we call their position space, to unlock load-balanced tiling and parallelism. We have prototyped these concepts in the open-source TACO system, where they are exposed as a scheduling API similar to the Halide domain-specific language for dense computations. Using this scheduling API, we show how to optimize mixed sparse/dense tensor algebra expressions, how to generate load-balanced code by scheduling sparse tensor algebra in position space, and how to generate sparse tensor algebra GPU code. Our evaluation shows that our transformations let us generate good code that is competitive with many hand-optimized implementations from the literature.

READ FULL TEXT
research
05/23/2022

SparseLNR: Accelerating Sparse Tensor Computations Using Loop Nest Restructuring

Sparse tensor algebra computations have become important in many real-wo...
research
09/09/2023

Compiling Recurrences over Dense and Sparse Arrays

Recurrence equations lie at the heart of many computational paradigms in...
research
04/23/2018

Unified Sparse Formats for Tensor Algebra Compilers

This paper shows how to build a sparse tensor algebra compiler that is a...
research
09/07/2022

Sgap: Towards Efficient Sparse Tensor Algebra Compilation for GPU

Sparse compiler is a promising solution for sparse tensor algebra optimi...
research
03/21/2021

Graph Transformation and Specialized Code Generation For Sparse Triangular Solve (SpTRSV)

Sparse Triangular Solve (SpTRSV) is an important computational kernel us...
research
08/22/2022

MOM: Matrix Operations in MLIR

Modern research in code generators for dense linear algebra computations...
research
11/24/2021

Composing Loop-carried Dependence with Other Loops

Sparse fusion is a compile-time loop transformation and runtime scheduli...

Please sign up or login with your details

Forgot password? Click here to reset