On the Local Linear Rate of Consensus on the Stiefel Manifold

01/22/2021
by   Shixiang Chen, et al.
0

We study the convergence properties of Riemannian gradient method for solving the consensus problem (for an undirected connected graph) over the Stiefel manifold. The Stiefel manifold is a non-convex set and the standard notion of averaging in the Euclidean space does not work for this problem. We propose Distributed Riemannian Consensus on Stiefel Manifold (DRCS) and prove that it enjoys a local linear convergence rate to global consensus. More importantly, this local rate asymptotically scales with the second largest singular value of the communication matrix, which is on par with the well-known rate in the Euclidean space. To the best of our knowledge, this is the first work showing the equality of the two rates. The main technical challenges include (i) developing a Riemannian restricted secant inequality for convergence analysis, and (ii) to identify the conditions (e.g., suitable step-size and initialization) under which the algorithm always stays in the local region.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/14/2021

Decentralized Riemannian Gradient Descent on the Stiefel Manifold

We consider a distributed non-convex optimization where a network of age...
02/18/2017

Riemannian stochastic variance reduced gradient

Stochastic variance reduction algorithms have recently become popular fo...
03/30/2020

Learning Theory for Estimation of Animal Motion Submanifolds

This paper describes the formulation and experimental testing of a novel...
06/16/2021

Averaging on the Bures-Wasserstein manifold: dimension-free convergence of gradient descent

We study first-order optimization algorithms for computing the barycente...
07/15/2022

Riemannian Natural Gradient Methods

This paper studies large-scale optimization problems on Riemannian manif...
05/17/2018

Fast, asymptotically efficient, recursive estimation in a Riemannian manifold

Stochastic optimisation in Riemannian manifolds, especially the Riemanni...
12/30/2011

On The Convergence of Gradient Descent for Finding the Riemannian Center of Mass

We study the problem of finding the global Riemannian center of mass of ...