Performance Modeling and Prediction for Dense Linear Algebra

06/01/2017
by   Elmar Peise, et al.
0

This dissertation introduces measurement-based performance modeling and prediction techniques for dense linear algebra algorithms. As a core principle, these techniques avoid executions of such algorithms entirely, and instead predict their performance through runtime estimates for the underlying compute kernels. For a variety of operations, these predictions allow to quickly select the fastest algorithm configurations from available alternatives. We consider two scenarios that cover a wide range of computations: To predict the performance of blocked algorithms, we design algorithm-independent performance models for kernel operations that are generated automatically once per platform. For various matrix operations, instantaneous predictions based on such models both accurately identify the fastest algorithm, and select a near-optimal block size. For performance predictions of BLAS-based tensor contractions, we propose cache-aware micro-benchmarks that take advantage of the highly regular structure inherent to contraction algorithms. At merely a fraction of a contraction's runtime, predictions based on such micro-benchmarks identify the fastest combination of tensor traversal and compute kernel.

READ FULL TEXT

page 17

page 24

page 38

page 40

research
11/30/2015

Sparse Tensor Algebra as a Parallel Programming Model

Dense and sparse tensors allow the representation of most bulk data stru...
research
04/27/2023

Co-Design of the Dense Linear AlgebravSoftware Stack for Multicore Processors

This paper advocates for an intertwined design of the dense linear algeb...
research
07/10/2020

Distributed-Memory DMRG via Sparse and Dense Parallel Tensor Contractions

The Density Matrix Renormalization Group (DMRG) algorithm is a powerful ...
research
07/27/2022

Correct Compilation of Semiring Contractions

We introduce a formal operational semantics that describes the fused exe...
research
07/01/2016

High-Performance Tensor Contraction without Transposition

Tensor computations--in particular tensor contraction (TC)--are importan...
research
07/05/2022

FLOPs as a Discriminant for Dense Linear Algebra Algorithms

Expressions that involve matrices and vectors, known as linear algebra e...
research
01/15/2020

Algorithms for Tensor Network Contraction Ordering

Contracting tensor networks is often computationally demanding. Well-des...

Please sign up or login with your details

Forgot password? Click here to reset