Multiplying Matrices Without Multiplying

06/21/2021
by   Davis Blalock, et al.
72

Multiplying matrices is among the most fundamental and compute-intensive operations in machine learning. Consequently, there has been significant work on efficiently approximating matrix multiplies. We introduce a learning-based algorithm for this task that greatly outperforms existing methods. Experiments using hundreds of matrices from diverse domains show that it often runs 100× faster than exact matrix products and 10× faster than current approximate methods. In the common case that one matrix is known ahead of time, our method also has the interesting property that it requires zero multiply-adds. These results suggest that a mixture of hashing, averaging, and byte shuffling-the core operations of our method-could be a more promising building block for machine learning than the sparsified, factorized, and/or scalar quantized matrix products that have recently been the focus of substantial research and hardware investment.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/01/2020

Density of diagonalizable matrices in sets of structured matrices defined from indefinite scalar products

For an (indefinite) scalar product [x,y]_B = x^HBy for B= ± B^H ∈ Gl_n(ℂ...
03/12/2018

An algorithm for hiding and recovering data using matrices

We present an algorithm for the recovery of a matrix M (non-singular ∈ ...
11/09/2020

Fast and Accurate Pseudoinverse with Sparse Matrix Reordering and Incremental Approach

How can we compute the pseudoinverse of a sparse feature matrix efficien...
05/04/2021

Hadamard matrices in {0,1} presentation and an algorithm for generating them

Hadamard matrices are square n× n matrices whose entries are ones and mi...
08/31/2017

Algorithmic patterns for H-matrices on many-core processors

In this work, we consider the reformulation of hierarchical (H) matrix a...
05/21/2018

Faster Neural Network Training with Approximate Tensor Operations

We propose a novel technique for faster Neural Network (NN) training by ...
05/31/2018

A faster hafnian formula for complex matrices and its benchmarking on the Titan supercomputer

We introduce new and simple algorithms for the calculation of the number...

Code Repositories

bolt

10-100x faster matrix and vector operations.


view repo