Run Procrustes, Run! On the convergence of accelerated Procrustes Flow

06/01/2018
by   Anastasios Kyrillidis, et al.
0

In this work, we present theoretical results on the convergence of non-convex accelerated gradient descent in matrix factorization models. The technique is applied to matrix sensing problems with squared loss, for the estimation of a rank r optimal solution X^∈R^n × n. We show that the acceleration leads to linear convergence rate, even under non-convex settings where the variable X is represented as U U^ for U ∈R^n × r. Our result has the same dependence on the condition number of the objective --and the optimal solution-- as that of the recent results on non-accelerated algorithms. However, acceleration is observed in practice, both in synthetic examples and in two real applications: neuronal multi-unit activities recovery from single electrode recordings, and quantum state tomography on quantum computing simulators.

READ FULL TEXT
research
04/14/2021

Fast quantum state reconstruction via accelerated non-convex programming

We propose a new quantum state reconstruction method that combines ideas...
research
05/31/2021

Generalized AdaGrad (G-AdaGrad) and Adam: A State-Space Perspective

Accelerated gradient-based methods are being extensively used for solvin...
research
06/04/2016

Provable Burer-Monteiro factorization for a class of norm-constrained matrix problems

We study the projected gradient descent method on low-rank matrix proble...
research
09/14/2023

Acceleration by Stepsize Hedging I: Multi-Step Descent and the Silver Stepsize Schedule

Can we accelerate convergence of gradient descent without changing the a...
research
02/15/2023

Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimization

Quasar convexity is a condition that allows some first-order methods to ...
research
01/31/2020

Learning Unitaries by Gradient Descent

We study the hardness of learning unitary transformations by performing ...
research
07/04/2015

Convex Factorization Machine for Regression

We propose the convex factorization machine (CFM), which is a convex var...

Please sign up or login with your details

Forgot password? Click here to reset