
An extracomponents method for evaluating fast matrixvector multiplication with special functions
In calculating integral or discrete transforms, fast algorithms for mult...
read it

Learning Multiplicationfree Linear Transformations
In this paper, we propose several dictionary learning algorithms for spa...
read it

Sparse Linear Networks with a Fixed Butterfly Structure: Theory and Practice
Fast Fourier transform, Wavelets, and other wellknown transforms in sig...
read it

Approximate Eigenvalue Decompositions of Linear Transformations with a Few Householder Reflectors
The ability to decompose a signal in an orthonormal basis (a set of orth...
read it

A Tutorial and Open Source Software for the Efficient Evaluation of Gravity and Magnetic Kernels
Fast computation of threedimensional gravity and magnetic forward model...
read it

Faster JohnsonLindenstrauss Transforms via Kronecker Products
The Kronecker product is an important matrix operation with a wide range...
read it

Deep Learning Models for Global Coordinate Transformations that Linearize PDEs
We develop a deep autoencoder architecture that can be used to find a co...
read it
Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations
Fast linear transforms are ubiquitous in machine learning, including the discrete Fourier transform, discrete cosine transform, and other structured transformations such as convolutions. All of these transforms can be represented by dense matrixvector multiplication, yet each has a specialized and highly efficient (subquadratic) algorithm. We ask to what extent handcrafting these algorithms and implementations is necessary, what structural priors they encode, and how much knowledge is required to automatically learn a fast algorithm for a provided structured transform. Motivated by a characterization of fast matrixvector multiplication as products of sparse matrices, we introduce a parameterization of divideandconquer methods that is capable of representing a large class of transforms. This generic formulation can automatically learn an efficient algorithm for many important transforms; for example, it recovers the O(N N) CooleyTukey FFT algorithm to machine precision, for dimensions N up to 1024. Furthermore, our method can be incorporated as a lightweight replacement of generic matrices in machine learning pipelines to learn efficient and compressible transformations. On a standard task of compressing a single hiddenlayer network, our method exceeds the classification accuracy of unconstrained matrices on CIFAR10 by 3.9 pointsthe first time a structured approach has done sowith 4X faster inference speed and 40X fewer parameters.
READ FULL TEXT
Comments
There are no comments yet.