Nonlinear Matrix Approximation with Radial Basis Function Components

06/03/2021
by   Elizaveta Rebrova, et al.
21

We introduce and investigate matrix approximation by decomposition into a sum of radial basis function (RBF) components. An RBF component is a generalization of the outer product between a pair of vectors, where an RBF function replaces the scalar multiplication between individual vector elements. Even though the RBF functions are positive definite, the summation across components is not restricted to convex combinations and allows us to compute the decomposition for any real matrix that is not necessarily symmetric or positive definite. We formulate the problem of seeking such a decomposition as an optimization problem with a nonlinear and non-convex loss function. Several modern versions of the gradient descent method, including their scalable stochastic counterparts, are used to solve this problem. We provide extensive empirical evidence of the effectiveness of the RBF decomposition and that of the gradient-based fitting algorithm. While being conceptually motivated by singular value decomposition (SVD), our proposed nonlinear counterpart outperforms SVD by drastically reducing the memory required to approximate a data matrix with the same L_2-error for a wide range of matrix types. For example, it leads to 2 to 10 times memory save for Gaussian noise, graph adjacency matrices, and kernel matrices. Moreover, this proximity-based decomposition can offer additional interpretability in applications that involve, e.g., capturing the inner low-dimensional structure of the data, retaining graph connectivity structure, and preserving the acutance of images.

READ FULL TEXT

page 2

page 6

page 7

page 8

page 9

research
04/18/2022

Fast optimization of common basis for matrix set through Common Singular Value Decomposition

SVD (singular value decomposition) is one of the basic tools of machine ...
research
06/07/2017

Fast Eigen Decomposition for Low-Rank Matrix Approximation

In this paper we present an efficient algorithm to compute the eigen dec...
research
02/01/2022

Approximation of Images via Generalized Higher Order Singular Value Decomposition over Finite-dimensional Commutative Semisimple Algebra

Low-rank approximation of images via singular value decomposition is wel...
research
04/24/2023

Preconditioner Design via the Bregman Divergence

We study a preconditioner for a Hermitian positive definite linear syste...
research
11/26/2019

Matrix Decompositions and Sparse Graph Regularity

We introduce and study a matrix decomposition that is a common generaliz...
research
01/22/2019

An Exact Reformulation of Feature-Vector-based Radial-Basis-Function Networks for Graph-based Observations

Radial-basis-function networks are traditionally defined for sets of vec...
research
07/19/2019

The LAPW method with eigendecomposition based on the Hari–Zimmermann generalized hyperbolic SVD

In this paper we propose an accurate, highly parallel algorithm for the ...

Please sign up or login with your details

Forgot password? Click here to reset