Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds

03/27/2021
by   Alexander Novikov, et al.
Skoltech
24

In scientific computing and machine learning applications, matrices and more general multidimensional arrays (tensors) can often be approximated with the help of low-rank decompositions. Since matrices and tensors of fixed rank form smooth Riemannian manifolds, one of the popular tools for finding low-rank approximations is to use Riemannian optimization. Nevertheless, efficient implementation of Riemannian gradients and Hessians, required in Riemannian optimization algorithms, can be a nontrivial task in practice. Moreover, in some cases, analytic formulas are not even available. In this paper, we build upon automatic differentiation and propose a method that, given an implementation of the function to be minimized, efficiently computes Riemannian gradients and matrix-by-vector products between an approximate Riemannian Hessian and a given vector.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/14/2020

Multilevel Riemannian optimization for low-rank problems

Large-scale optimization problems arising from the discretization of pro...
08/23/2013

Manopt, a Matlab toolbox for optimization on manifolds

Optimization on manifolds is a rapidly developing branch of nonlinear op...
05/13/2023

Structured Low-Rank Tensor Learning

We consider the problem of learning low-rank tensors from partial observ...
08/02/2021

Tensor completion using geodesics on Segre manifolds

We propose a Riemannian conjugate gradient (CG) optimization method for ...
12/07/2015

Fast Optimization Algorithm on Riemannian Manifolds and Its Application in Low-Rank Representation

The paper addresses the problem of optimizing a class of composite funct...
04/26/2017

Riemannian Optimization for Skip-Gram Negative Sampling

Skip-Gram Negative Sampling (SGNS) word embedding model, well known by i...
09/23/2018

Generalized Low-Rank Optimization for Topological Cooperation in Ultra-Dense Networks

Network densification is a natural way to support dense mobile applicati...

Please sign up or login with your details

Forgot password? Click here to reset