MahNMF: Manhattan Non-negative Matrix Factorization

07/14/2012
by   Naiyang Guan, et al.
0

Non-negative matrix factorization (NMF) approximates a non-negative matrix X by a product of two non-negative low-rank factor matrices W and H. NMF and its extensions minimize either the Kullback-Leibler divergence or the Euclidean distance between X and W^T H to model the Poisson noise or the Gaussian noise. In practice, when the noise distribution is heavy tailed, they cannot perform well. This paper presents Manhattan NMF (MahNMF) which minimizes the Manhattan distance between X and W^T H for modeling the heavy tailed Laplacian noise. Similar to sparse and low-rank matrix decompositions, MahNMF robustly estimates the low-rank part and the sparse part of a non-negative matrix and thus performs effectively when data are contaminated by outliers. We extend MahNMF for various practical applications by developing box-constrained MahNMF, manifold regularized MahNMF, group sparse MahNMF, elastic net inducing MahNMF, and symmetric MahNMF. The major contribution of this paper lies in two fast optimization algorithms for MahNMF and its extensions: the rank-one residual iteration (RRI) method and Nesterov's smoothing method. In particular, by approximating the residual matrix by the outer product of one row of W and one row of H in MahNMF, we develop an RRI method to iteratively update each variable of W and H in a closed form solution. Although RRI is efficient for small scale MahNMF and some of its extensions, it is neither scalable to large scale matrices nor flexible enough to optimize all MahNMF extensions. Since the objective functions of MahNMF and its extensions are neither convex nor smooth, we apply Nesterov's smoothing method to recursively optimize one factor matrix with another matrix fixed. By setting the smoothing parameter inversely proportional to the iteration number, we improve the approximation accuracy iteratively for both MahNMF and its extensions.

READ FULL TEXT

page 26

page 28

page 32

page 33

page 34

page 35

page 36

page 37

research
10/25/2021

Fast Rank-1 NMF for Missing Data with KL Divergence

We propose a fast non-gradient based method of rank-1 non-negative matri...
research
07/12/2019

A Quantum-inspired Classical Algorithm for Separable Non-negative Matrix Factorization

Non-negative Matrix Factorization (NMF) asks to decompose a (entry-wise)...
research
10/14/2020

Low-rank Convex/Sparse Thermal Matrix Approximation for Infrared-based Diagnostic System

Active and passive thermography are two efficient techniques extensively...
research
05/07/2018

Semi-Orthogonal Non-Negative Matrix Factorization

Non-negative Matrix Factorization (NMF) is a popular clustering and dime...
research
06/02/2019

Truncated Cauchy Non-negative Matrix Factorization

Non-negative matrix factorization (NMF) minimizes the Euclidean distance...
research
11/02/2022

Rank Selection for Non-negative Matrix Factorization

Non-Negative Matrix Factorization (NMF) is a widely used dimension reduc...
research
03/05/2018

Relative Pairwise Relationship Constrained Non-negative Matrix Factorisation

Non-negative Matrix Factorisation (NMF) has been extensively used in mac...

Please sign up or login with your details

Forgot password? Click here to reset