No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis

04/03/2017
by   Rong Ge, et al.
0

In this paper we develop a new framework that captures the common landscape underlying the common non-convex low-rank matrix problems including matrix sensing, matrix completion and robust PCA. In particular, we show for all above problems (including asymmetric cases): 1) all local minima are also globally optimal; 2) no high-order saddle points exists. These results explain why simple algorithms such as stochastic gradient descent have global converge, and efficiently optimize these non-convex objective functions in practice. Our framework connects and simplifies the existing analyses on optimization landscapes for matrix sensing and symmetric matrix completion. The framework naturally leads to new results for asymmetric matrix completion and robust PCA.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/27/2021

Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization

We study the asymmetric low-rank factorization problem: min_𝐔∈ℝ^m ×...
research
05/24/2016

Matrix Completion has No Spurious Local Minimum

Matrix completion is a basic machine learning problem that has wide appl...
research
11/06/2017

Memory-efficient Kernel PCA via Partial Matrix Sampling and Nonconvex Optimization: a Model-free Analysis of Local Minima

Kernel PCA is a widely used nonlinear dimension reduction technique in m...
research
09/10/2015

Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees

Optimization problems with rank constraints arise in many applications, ...
research
08/21/2016

A Non-convex One-Pass Framework for Generalized Factorization Machine and Rank-One Matrix Sensing

We develop an efficient alternating framework for learning a generalized...
research
03/28/2018

Non-Convex Matrix Completion Against a Semi-Random Adversary

Matrix completion is a well-studied problem with many machine learning a...
research
09/09/2022

Optimal (0,1)-Matrix Completion with Majorization Ordered Objectives (To the memory of Pravin Varaiya)

We propose and examine two optimal (0,1)-matrix completion problems with...

Please sign up or login with your details

Forgot password? Click here to reset