DeepAI AI Chat
Log In Sign Up

Asymmetric Multiresolution Matrix Factorization

Multiresolution Matrix Factorization (MMF) was recently introduced as an alternative to the dominant low-rank paradigm in order to capture structure in matrices at multiple different scales. Using ideas from multiresolution analysis (MRA), MMF teased out hierarchical structure in symmetric matrices by constructing a sequence of wavelet bases. While effective for such matrices, there is plenty of data that is more naturally represented as nonsymmetric matrices (e.g. directed graphs), but nevertheless has similar hierarchical structure. In this paper, we explore techniques for extending MMF to any square matrix. We validate our approach on numerous matrix compression tasks, demonstrating its efficacy compared to low-rank methods. Moreover, we also show that a combined low-rank and MMF approach, which amounts to removing a small global-scale component of the matrix and then extracting hierarchical structure from the residual, is even more effective than each of the two complementary methods for matrix compression.


page 1

page 2

page 3

page 4


Low-Rank Prune-And-Factorize for Language Model Compression

The components underpinning PLMs – large weight matrices – were shown to...

Learning Multiresolution Matrix Factorization and its Wavelet Networks on Graphs

Multiresolution Matrix Factorization (MMF) is unusual amongst fast matri...

The Incremental Multiresolution Matrix Factorization Algorithm

Multiresolution analysis and matrix factorization are foundational tools...

From graphs to DAGs: a low-complexity model and a scalable algorithm

Learning directed acyclic graphs (DAGs) is long known a critical challen...

ACCAMS: Additive Co-Clustering to Approximate Matrices Succinctly

Matrix completion and approximation are popular tools to capture a user'...

Group-sparse Embeddings in Collective Matrix Factorization

CMF is a technique for simultaneously learning low-rank representations ...

Hierarchical Low-rank Structure of Parameterized Distributions

This note shows that the matrix forms of several one-parameter distribut...