Communication Lower Bounds of Bilinear Algorithms for Symmetric Tensor Contractions
share
Accurate numerical calculations of electronic structure are often dominated in cost by tensor contractions. These tensors are typically symmetric under interchange of modes, enabling reduced-size representations as well as a reduced computation cost. Direct evaluation algorithms for such contractions use matrix and vector unfoldings of the tensors, computing and accumulating products of input elements. Symmetry preserving algorithms reduce the number of products by multiplying linear combinations of input elements. The two schemes can be encoded via sparse matrices as bilinear algorithms. We formulate a general notion of expansion for bilinear algorithms in terms of the rank of submatrices of the sparse matrix encoding. This expansion bounds the number of products that can be computed provided a bounded amount of data. Consequently, we derive communication lower bounds for any sequential or parallel schedule of a bilinear algorithm with a given expansion. After deriving such expansion bounds for the tensor contraction algorithms, we obtain new results that demonstrate asymptotic communication overheads associated with exploiting symmetries. Computing a nonsymmetric tensor contraction requires less communication than either method for symmetric contractions when either (1) computing a symmetrized tensor-product of tensors of different orders or (2) the tensor unfolding of the contraction corresponds to a matrix-vector product with a nonsquare matrix. Further, when the unfolding is a product of two non-square matrices, asymptotically more communication is needed by the symmetry preserving algorithm than the traditional algorithm, despite its lower computation cost.
READ FULL TEXT