Log In Sign Up

Parallel Algorithms for Tensor Train Arithmetic

by   Hussam Al Daas, et al.

We present efficient and scalable parallel algorithms for performing mathematical operations for low-rank tensors represented in the tensor train (TT) format. We consider algorithms for addition, elementwise multiplication, computing norms and inner products, orthogonalization, and rounding (rank truncation). These are the kernel operations for applications such as iterative Krylov solvers that exploit the TT structure. The parallel algorithms are designed for distributed-memory computation, and we use a data distribution and strategy that parallelizes computations for individual cores within the TT format. We analyze the computation and communication costs of the proposed algorithms to show their scalability, and we present numerical experiments that demonstrate their efficiency on both shared-memory and distributed-memory parallel systems. For example, we observe better single-core performance than the existing MATLAB TT-Toolbox in rounding a 2GB TT tensor, and our implementation achieves a 34× speedup using all 40 cores of a single node. We also show nearly linear parallel scaling on larger TT tensors up to over 10,000 cores for all mathematical operations.


Randomized algorithms for rounding in the Tensor-Train format

The Tensor-Train (TT) format is a highly compact low-rank representation...

Parallel Nonnegative CP Decomposition of Dense Tensors

The CP tensor decomposition is a low-rank approximation of a tensor. We ...

Constructive TT-representation of the tensors given as index interaction functions with applications

This paper presents a method to build explicit tensor-train (TT) represe...

A Fresh Approach to Evaluate Performance in Distributed Parallel Genetic Algorithms

This work proposes a novel approach to evaluate and analyze the behavior...

PLANC: Parallel Low Rank Approximation with Non-negativity Constraints

We consider the problem of low-rank approximation of massive dense non-n...

GBLA -- Gröbner Basis Linear Algebra Package

This is a system paper about a new GPLv2 open source C library GBLA impl...

Enhanced AGCM3D: A Highly Scalable Dynamical Core of Atmospheric General Circulation Model Based on Leap-Format

The finite-difference dynamical core based on the equal-interval latitud...