
Trivializations for GradientBased Optimization on Manifolds
We introduce a framework to study the transformation of problems with ma...
read it

CurvatureDependant Global Convergence Rates for Optimization on Manifolds of Bounded Geometry
We give curvaturedependant convergence rates for the optimization of we...
read it

Adaptive Canonical Correlation Analysis Based On Matrix Manifolds
In this paper, we formulate the Canonical Correlation Analysis (CCA) pro...
read it

Continuous Flattening of All Polyhedral Manifolds using Countably Infinite Creases
We prove that any finite polyhedral manifold in 3D can be continuously f...
read it

Curvatures of Stiefel manifolds with deformation metrics
We compute curvatures of a family of tractable metrics on Stiefel manifo...
read it

A toolkit to describe and interactively display threemanifolds embedded in fourspace
A data structure and toolkit are presented here that allow for the descr...
read it

Local criteria for triangulation of manifolds
We present criteria for establishing a triangulation of a manifold. Give...
read it
Adaptive and Momentum Methods on Manifolds Through Trivializations
Adaptive methods do not have a direct generalization to manifolds as the adaptive term is not invariant. Momentum methods on manifolds suffer from efficiency problems stemming from the curvature of the manifold. We introduce a framework to generalize adaptive and momentum methods to arbitrary manifolds by noting that for every differentiable manifold, there exists a radially convex open set that covers almost all the manifold. Being radially convex, this set is diffeomorphic to ℝ^n. This gives a natural generalization of any adaptive and momentumbased algorithm to a set that covers almost all the manifold in an arbitrary manifolds. We also show how to extend these methods to the context of gradient descent methods with a retraction. For its implementation, we bring an approximation to the exponential of matrices that needs just of 5 matrix multiplications, making it particularly efficient on GPUs. In practice, we see that this family of algorithms closes the numerical gap created by an incorrect use of momentum and adaptive methods on manifolds. At the same time, we see that the most efficient algorithm of this family is given by simply pulling back the problem to the tangent space at the initial point via the exponential map.
READ FULL TEXT
Comments
There are no comments yet.