Convergence to Second-Order Stationarity for Non-negative Matrix Factorization: Provably and Concurrently

02/26/2020
by   Ioannis Panageas, et al.
0

Non-negative matrix factorization (NMF) is a fundamental non-convex optimization problem with numerous applications in Machine Learning (music analysis, document clustering, speech-source separation etc). Despite having received extensive study, it is poorly understood whether or not there exist natural algorithms that can provably converge to a local minimum. Part of the reason is because the objective is heavily symmetric and its gradient is not Lipschitz. In this paper we define a multiplicative weight update type dynamics (modification of the seminal Lee-Seung algorithm) that runs concurrently and provably avoids saddle points (first order stationary points that are not second order). Our techniques combine tools from dynamical systems such as stability and exploit the geometry of the NMF objective by reducing the standard NMF formulation over the non-negative orthant to a new formulation over (a scaled) simplex. An important advantage of our method is the use of concurrent updates, which permits implementations in parallel computing environments.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/08/2015

Intersecting Faces: Non-negative Matrix Factorization With New Guarantees

Non-negative matrix factorization (NMF) is a natural model of admixture ...
research
04/07/2016

A Unified Framework for Sparse Non-Negative Least Squares using Multiplicative Updates and the Non-Negative Matrix Factorization Problem

We study the sparse non-negative least squares (S-NNLS) problem. S-NNLS ...
research
12/30/2016

Non-Negative Matrix Factorization Test Cases

Non-negative matrix factorization (NMF) is a prob- lem with many applica...
research
05/08/2017

Non-negative Matrix Factorization via Archetypal Analysis

Given a collection of data points, non-negative matrix factorization (NM...
research
05/27/2021

Non-negative matrix factorization algorithms greatly improve topic model fits

We report on the potential for using algorithms for non-negative matrix ...
research
11/11/2016

Recovery Guarantee of Non-negative Matrix Factorization via Alternating Updates

Non-negative matrix factorization is a popular tool for decomposing data...
research
07/01/2016

On Mixed Memberships and Symmetric Nonnegative Matrix Factorizations

The problem of finding overlapping communities in networks has gained mu...

Please sign up or login with your details

Forgot password? Click here to reset