Riemannian optimization using three different metrics for Hermitian PSD fixed-rank constraints: an extended version

by   Shixin Zheng, et al.

We consider smooth optimization problems with a Hermitian positive semi-definite fixed-rank constraint, where a quotient geometry with three Riemannian metrics g^i(·, ·) (i=1,2,3) is used to represent this constraint. By taking the nonlinear conjugate gradient method (CG) as an example, we show that CG on the quotient geometry with metric g^1 is equivalent to CG on the factor-based optimization framework, which is often called the Burer–Monteiro approach. We also show that CG on the quotient geometry with metric g^3 is equivalent to CG on the commonly-used embedded geometry. We call two CG methods equivalent if they produce an identical sequence of iterates {X_k}. In addition, we show that if the limit point of the sequence {X_k} generated by an algorithm has lower rank, that is X_k∈ℂ^n× n, k = 1, 2, … has rank p and the limit point X_* has rank r < p, then the condition number of the Riemannian Hessian with metric g^1 can be unbounded, but those of the other two metrics stay bounded. Numerical experiments show that the Burer–Monteiro CG method has slower local convergence rate if the limit point has a reduced rank, compared to CG on the quotient geometry under the other two metrics. This slower convergence rate can thus be attributed to the large condition number of the Hessian near a minimizer.


page 1

page 2

page 3

page 4


Operator-valued formulas for Riemannian Gradient and Hessian and families of tractable metrics in optimization and machine learning

We provide an explicit formula for the Levi-Civita connection and Rieman...

On Geometric Connections of Embedded and Quotient Geometries in Riemannian Fixed-rank Matrix Optimization

In this paper, we propose a general procedure for establishing the lands...

Principal subbundles for dimension reduction

In this paper we demonstrate how sub-Riemannian geometry can be used for...

Solving Trust Region Subproblems Using Riemannian Optimization

The Trust Region Subproblem is a fundamental optimization problem that t...

Asymptotic behaviour of learning rates in Armijo's condition

Fix a constant 0<α <1. For a C^1 function f:ℝ^k→ℝ, a point x and a posit...

Riemannian Langevin Monte Carlo schemes for sampling PSD matrices with fixed rank

This paper introduces two explicit schemes to sample matrices from Gibbs...

Computing Brascamp-Lieb Constants through the lens of Thompson Geometry

This paper studies algorithms for efficiently computing Brascamp-Lieb co...

Please sign up or login with your details

Forgot password? Click here to reset