Riemannian optimization using three different metrics for Hermitian PSD fixed-rank constraints: an extended version

04/16/2022
by   Shixin Zheng, et al.
0

We consider smooth optimization problems with a Hermitian positive semi-definite fixed-rank constraint, where a quotient geometry with three Riemannian metrics g^i(·, ·) (i=1,2,3) is used to represent this constraint. By taking the nonlinear conjugate gradient method (CG) as an example, we show that CG on the quotient geometry with metric g^1 is equivalent to CG on the factor-based optimization framework, which is often called the Burer–Monteiro approach. We also show that CG on the quotient geometry with metric g^3 is equivalent to CG on the commonly-used embedded geometry. We call two CG methods equivalent if they produce an identical sequence of iterates {X_k}. In addition, we show that if the limit point of the sequence {X_k} generated by an algorithm has lower rank, that is X_k∈ℂ^n× n, k = 1, 2, … has rank p and the limit point X_* has rank r < p, then the condition number of the Riemannian Hessian with metric g^1 can be unbounded, but those of the other two metrics stay bounded. Numerical experiments show that the Burer–Monteiro CG method has slower local convergence rate if the limit point has a reduced rank, compared to CG on the quotient geometry under the other two metrics. This slower convergence rate can thus be attributed to the large condition number of the Hessian near a minimizer.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/21/2020

Operator-valued formulas for Riemannian Gradient and Hessian and families of tractable metrics in optimization and machine learning

We provide an explicit formula for the Levi-Civita connection and Rieman...
research
10/23/2021

On Geometric Connections of Embedded and Quotient Geometries in Riemannian Fixed-rank Matrix Optimization

In this paper, we propose a general procedure for establishing the lands...
research
07/06/2023

Principal subbundles for dimension reduction

In this paper we demonstrate how sub-Riemannian geometry can be used for...
research
10/15/2020

Solving Trust Region Subproblems Using Riemannian Optimization

The Trust Region Subproblem is a fundamental optimization problem that t...
research
07/07/2020

Asymptotic behaviour of learning rates in Armijo's condition

Fix a constant 0<α <1. For a C^1 function f:ℝ^k→ℝ, a point x and a posit...
research
09/08/2023

Riemannian Langevin Monte Carlo schemes for sampling PSD matrices with fixed rank

This paper introduces two explicit schemes to sample matrices from Gibbs...
research
08/09/2022

Computing Brascamp-Lieb Constants through the lens of Thompson Geometry

This paper studies algorithms for efficiently computing Brascamp-Lieb co...

Please sign up or login with your details

Forgot password? Click here to reset