Asymmetric Multiresolution Matrix Factorization

Multiresolution Matrix Factorization (MMF) was recently introduced as an alternative to the dominant low-rank paradigm in order to capture structure in matrices at multiple different scales. Using ideas from multiresolution analysis (MRA), MMF teased out hierarchical structure in symmetric matrices by constructing a sequence of wavelet bases. While effective for such matrices, there is plenty of data that is more naturally represented as nonsymmetric matrices (e.g. directed graphs), but nevertheless has similar hierarchical structure. In this paper, we explore techniques for extending MMF to any square matrix. We validate our approach on numerous matrix compression tasks, demonstrating its efficacy compared to low-rank methods. Moreover, we also show that a combined low-rank and MMF approach, which amounts to removing a small global-scale component of the matrix and then extracting hierarchical structure from the residual, is even more effective than each of the two complementary methods for matrix compression.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/25/2023

Low-Rank Prune-And-Factorize for Language Model Compression

The components underpinning PLMs – large weight matrices – were shown to...
research
11/02/2021

Learning Multiresolution Matrix Factorization and its Wavelet Networks on Graphs

Multiresolution Matrix Factorization (MMF) is unusual amongst fast matri...
research
05/16/2017

The Incremental Multiresolution Matrix Factorization Algorithm

Multiresolution analysis and matrix factorization are foundational tools...
research
04/10/2022

From graphs to DAGs: a low-complexity model and a scalable algorithm

Learning directed acyclic graphs (DAGs) is long known a critical challen...
research
12/31/2014

ACCAMS: Additive Co-Clustering to Approximate Matrices Succinctly

Matrix completion and approximation are popular tools to capture a user'...
research
12/20/2013

Group-sparse Embeddings in Collective Matrix Factorization

CMF is a technique for simultaneously learning low-rank representations ...
research
11/29/2019

Hierarchical Low-rank Structure of Parameterized Distributions

This note shows that the matrix forms of several one-parameter distribut...

Please sign up or login with your details

Forgot password? Click here to reset