Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds

03/27/2021
by   Alexander Novikov, et al.
24

In scientific computing and machine learning applications, matrices and more general multidimensional arrays (tensors) can often be approximated with the help of low-rank decompositions. Since matrices and tensors of fixed rank form smooth Riemannian manifolds, one of the popular tools for finding low-rank approximations is to use Riemannian optimization. Nevertheless, efficient implementation of Riemannian gradients and Hessians, required in Riemannian optimization algorithms, can be a nontrivial task in practice. Moreover, in some cases, analytic formulas are not even available. In this paper, we build upon automatic differentiation and propose a method that, given an implementation of the function to be minimized, efficiently computes Riemannian gradients and matrix-by-vector products between an approximate Riemannian Hessian and a given vector.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/14/2020

Multilevel Riemannian optimization for low-rank problems

Large-scale optimization problems arising from the discretization of pro...
research
08/23/2013

Manopt, a Matlab toolbox for optimization on manifolds

Optimization on manifolds is a rapidly developing branch of nonlinear op...
research
05/13/2023

Structured Low-Rank Tensor Learning

We consider the problem of learning low-rank tensors from partial observ...
research
08/02/2021

Tensor completion using geodesics on Segre manifolds

We propose a Riemannian conjugate gradient (CG) optimization method for ...
research
12/07/2015

Fast Optimization Algorithm on Riemannian Manifolds and Its Application in Low-Rank Representation

The paper addresses the problem of optimizing a class of composite funct...
research
04/26/2017

Riemannian Optimization for Skip-Gram Negative Sampling

Skip-Gram Negative Sampling (SGNS) word embedding model, well known by i...
research
09/23/2018

Generalized Low-Rank Optimization for Topological Cooperation in Ultra-Dense Networks

Network densification is a natural way to support dense mobile applicati...

Please sign up or login with your details

Forgot password? Click here to reset