On the analysis of optimization with fixed-rank matrices: a quotient geometric view

by   Shuyu Dong, et al.

We study a type of Riemannian gradient descent (RGD) algorithm, designed through Riemannian preconditioning, for optimization on ℳ_k^m× n – the set of m× n real matrices with a fixed rank k. Our analysis is based on a quotient geometric view of ℳ_k^m× n: by identifying this set with the quotient manifold of a two-term product space ℝ_*^m× k×ℝ_*^n× k of matrices with full column rank via matrix factorization, we find an explicit form for the update rule of the RGD algorithm, which leads to a novel approach to analysing their convergence behavior in rank-constrained optimization. We then deduce some interesting properties that reflect how RGD distinguishes from other matrix factorization algorithms such as those based on the Euclidean geometry. In particular, we show that the RGD algorithm are not only faster than Euclidean gradient descent but also do not rely on the balancing technique to ensure its efficiency while the latter does. Starting from the novel results, we further show that this RGD algorithm is guaranteed to solve matrix sensing and matrix completion problems with linear convergence rate, under mild conditions related to the restricted positive definiteness property. Numerical experiments on matrix sensing and completion are provided to demonstrate these properties.


page 1

page 2

page 3

page 4


Exact Linear Convergence Rate Analysis for Low-Rank Symmetric Matrix Completion via Gradient Descent

Factorization-based gradient descent is a scalable and efficient algorit...

Implicit regularization and solution uniqueness in over-parameterized matrix sensing

We consider whether algorithmic choices in over-parameterized linear mat...

Splitting numerical integration for matrix completion

Low rank matrix approximation is a popular topic in machine learning. In...

Recursive Importance Sketching for Rank Constrained Least Squares: Algorithms and High-order Convergence

In this paper, we propose a new ecursive mportance ketching algorithm fo...

Riemannian Perspective on Matrix Factorization

We study the non-convex matrix factorization approach to matrix completi...

Positive Semidefinite Matrix Factorization: A Connection with Phase Retrieval and Affine Rank Minimization

Positive semidefinite matrix factorization (PSDMF) expresses each entry ...

A General Algorithm for Solving Rank-one Matrix Sensing

Matrix sensing has many real-world applications in science and engineeri...

Please sign up or login with your details

Forgot password? Click here to reset