Analysis of Asymptotic Escape of Strict Saddle Sets in Manifold Optimization

11/28/2019
by   Thomas Y. Hou, et al.
0

In this paper, we provide some analysis on the asymptotic escape of strict saddles in manifold optimization using the projected gradient descent (PGD) algorithm. One of our main contributions is that we extend the current analysis to include non-isolated and possibly continuous saddle sets with complicated geometry. We prove that the PGD is able to escape strict critical submanifolds under certain conditions on the geometry and the distribution of the saddle point sets. We also show that the PGD may fail to escape strict saddles under weaker assumptions even if the saddle point set has zero measure and there is a uniform escape direction. We provide a counterexample to illustrate this important point. We apply this saddle analysis to the phase retrieval problem on the low-rank matrix manifold, prove that there are only a finite number of saddles, and they are strict saddles with high probability. We also show the potential application of our analysis for a broader range of manifold optimization problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/20/2021

Asymptotic Escape of Spurious Critical Points on the Low-rank Matrix Manifold

We show that the Riemannian gradient descent algorithm on the low-rank m...
research
12/31/2020

Fast Global Convergence for Low-rank Matrix Recovery via Riemannian Gradient Descent with Random Initialization

In this paper, we propose a new global analysis framework for a class of...
research
06/23/2022

Low-Rank Mirror-Prox for Nonsmooth and Low-Rank Matrix Optimization Problems

Low-rank and nonsmooth matrix optimization problems capture many fundame...
research
01/07/2021

Boundary Conditions for Linear Exit Time Gradient Trajectories Around Saddle Points: Analysis and Algorithm

Gradient-related first-order methods have become the workhorse of large-...
research
11/27/2020

A Grassmann Manifold Handbook: Basic Geometry and Computational Aspects

The Grassmann manifold of linear subspaces is important for the mathemat...
research
06/01/2020

Exit Time Analysis for Approximations of Gradient Descent Trajectories Around Saddle Points

This paper considers the problem of understanding the exit time for traj...
research
02/27/2020

The Landscape of Matrix Factorization Revisited

We revisit the landscape of the simple matrix factorization problem. For...

Please sign up or login with your details

Forgot password? Click here to reset