Accelerated Algorithms for Convex and Non-Convex Optimization on Manifolds

10/18/2020
by   Lizhen Lin, et al.
22

We propose a general scheme for solving convex and non-convex optimization problems on manifolds. The central idea is that, by adding a multiple of the squared retraction distance to the objective function in question, we "convexify" the objective function and solve a series of convex sub-problems in the optimization procedure. One of the key challenges for optimization on manifolds is the difficulty of verifying the complexity of the objective function, e.g., whether the objective function is convex or non-convex, and the degree of non-convexity. Our proposed algorithm adapts to the level of complexity in the objective function. We show that when the objective function is convex, the algorithm provably converges to the optimum and leads to accelerated convergence. When the objective function is non-convex, the algorithm will converge to a stationary point. Our proposed method unifies insights from Nesterov's original idea for accelerating gradient descent algorithms with recent developments in optimization algorithms in Euclidean space. We demonstrate the utility of our algorithms on several manifold optimization tasks such as estimating intrinsic and extrinsic Fréchet means on spheres and low-rank matrix factorization with Grassmann manifolds applied to the Netflix rating data set.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/30/2019

Inexact Proximal-Point Penalty Methods for Non-Convex Optimization with Non-Convex Constraints

Non-convex optimization problems arise from various areas in science and...
research
08/17/2016

Mollifying Networks

The optimization of deep neural networks can be more challenging than tr...
research
01/23/2020

Replica Exchange for Non-Convex Optimization

Gradient descent (GD) is known to converge quickly for convex objective ...
research
10/14/2020

Alternating Minimization Based First-Order Method for the Wireless Sensor Network Localization Problem

We propose an algorithm for the Wireless Sensor Network localization pro...
research
08/16/2021

A diffusion-map-based algorithm for gradient computation on manifolds and applications

We recover the gradient of a given function defined on interior points o...
research
11/14/2019

Solving Inverse Problems by Joint Posterior Maximization with a VAE Prior

In this paper we address the problem of solving ill-posed inverse proble...
research
10/26/2018

Communication Efficient Parallel Algorithms for Optimization on Manifolds

The last decade has witnessed an explosion in the development of models,...

Please sign up or login with your details

Forgot password? Click here to reset