Self-Expressive Decompositions for Matrix Approximation and Clustering

05/04/2015
by   Eva L. Dyer, et al.
0

Data-aware methods for dimensionality reduction and matrix decomposition aim to find low-dimensional structure in a collection of data. Classical approaches discover such structure by learning a basis that can efficiently express the collection. Recently, "self expression", the idea of using a small subset of data vectors to represent the full collection, has been developed as an alternative to learning. Here, we introduce a scalable method for computing sparse SElf-Expressive Decompositions (SEED). SEED is a greedy method that constructs a basis by sequentially selecting incoherent vectors from the dataset. After forming a basis from a subset of vectors in the dataset, SEED then computes a sparse representation of the dataset with respect to this basis. We develop sufficient conditions under which SEED exactly represents low rank matrices and vectors sampled from a unions of independent subspaces. We show how SEED can be used in applications ranging from matrix approximation and denoising to clustering, and apply it to numerous real-world datasets. Our results demonstrate that SEED is an attractive low-complexity alternative to other sparse matrix factorization approaches such as sparse PCA and self-expressive methods for clustering.

READ FULL TEXT

page 4

page 9

research
07/01/2020

A New Basis for Sparse PCA

The statistical and computational performance of sparse principal compon...
research
11/24/2021

PMSSC: Parallelizable Multi-Subset based Self-Expressive Model for Subspace Clustering

Subspace clustering methods embrace a self-expressive model that represe...
research
08/06/2016

Randomized Matrix Decompositions using R

Matrix decompositions are fundamental tools in the area of applied mathe...
research
05/04/2020

Robust Non-Linear Matrix Factorization for Dictionary Learning, Denoising, and Clustering

Low dimensional nonlinear structure abounds in datasets across computer ...
research
02/05/2021

Matrix Decomposition on Graphs: A Functional View

We propose a functional view of matrix decomposition problems on graphs ...
research
04/22/2022

Paramixer: Parameterizing Mixing Links in Sparse Factors Works Better than Dot-Product Self-Attention

Self-Attention is a widely used building block in neural modeling to mix...

Please sign up or login with your details

Forgot password? Click here to reset