Finding stationary points on bounded-rank matrices: A geometric hurdle and a smooth remedy

by   Eitan Levin, et al.

We consider the problem of provably finding a stationary point of a smooth function to be minimized on the variety of bounded-rank matrices. While optimization on low-rank matrices has been extensively studied, existing algorithms do not provide such a basic guarantee. We trace this back to a geometric obstacle: On a nonsmooth set, there may be sequences of points along which standard measures of stationarity tend to zero, but whose limit point is not stationary. We name such events apocalypses, as they can cause optimization algorithms to converge to non-stationary points. We illustrate this on an existing algorithm using an explicit apocalypse on the bounded-rank matrix variety. To provably find stationary points, we modify a trust-region method on a standard smooth parameterization of the variety. The method relies on the known fact that second-order stationary points on the parameter space map to stationary points on the variety. Our geometric observations and proposed algorithm generalize beyond bounded-rank matrices. We give a geometric characterization of apocalypses on general constraint sets, which implies that Clarke-regular sets do not admit apocalypses. Such sets include smooth manifolds, manifolds with boundaries, and convex sets. Our trust-region method supports parameterization by any complete Riemannian manifold.


page 1

page 2

page 3

page 4


An apocalypse-free first-order low-rank optimization algorithm

We consider the problem of minimizing a differentiable function with loc...

Reexamining Low Rank Matrix Factorization for Trace Norm Regularization

Trace norm regularization is a widely used approach for learning low ran...

First-order optimization on stratified sets

We consider the problem of minimizing a differentiable function with loc...

On Geometric Connections of Embedded and Quotient Geometries in Riemannian Fixed-rank Matrix Optimization

In this paper, we propose a general procedure for establishing the lands...

Nonconvex Factorization and Manifold Formulations are Almost Equivalent in Low-rank Matrix Optimization

In this paper, we consider the geometric landscape connection of the wid...

Blind Demixing for Low-Latency Communication

In the next generation wireless networks, lowlatency communication is cr...

Please sign up or login with your details

Forgot password? Click here to reset