Provable Efficient Online Matrix Completion via Non-convex Stochastic Gradient Descent

05/26/2016
by   Chi Jin, et al.
0

Matrix completion, where we wish to recover a low rank matrix by observing a few entries from it, is a widely studied problem in both theory and practice with wide applications. Most of the provable algorithms so far on this problem have been restricted to the offline setting where they provide an estimate of the unknown matrix using all observations simultaneously. However, in many applications, the online version, where we observe one entry at a time and dynamically update our estimate, is more appealing. While existing algorithms are efficient for the offline setting, they could be highly inefficient for the online setting. In this paper, we propose the first provable, efficient online algorithm for matrix completion. Our algorithm starts from an initial estimate of the matrix and then performs non-convex stochastic gradient descent (SGD). After every observation, it performs a fast update involving only one row of two tall matrices, giving near linear total runtime. Our algorithm can be naturally used in the offline setting as well, where it gives competitive sample complexity and runtime to state of the art algorithms. Our proofs introduce a general framework to show that SGD updates tend to stay away from saddle surfaces and could be of broader interests for other non-convex problems to prove tight rates.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/05/2014

Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problems

Stochastic gradient descent (SGD) on a low-rank factorization is commonl...
research
11/04/2014

Fast Exact Matrix Completion with Finite Samples

Matrix completion is the problem of recovering a low rank matrix by obse...
research
05/25/2016

Fast Algorithms for Robust PCA via Gradient Descent

We consider the problem of Robust PCA in the fully and partially observe...
research
08/01/2018

Matrix completion and extrapolation via kernel regression

Matrix completion and extrapolation (MCEX) are dealt with here over repr...
research
03/28/2018

Non-Convex Matrix Completion Against a Semi-Random Adversary

Matrix completion is a well-studied problem with many machine learning a...
research
05/24/2016

Matrix Completion has No Spurious Local Minimum

Matrix completion is a basic machine learning problem that has wide appl...
research
08/24/2022

Accelerating SGD for Highly Ill-Conditioned Huge-Scale Online Matrix Completion

The matrix completion problem seeks to recover a d× d ground truth matri...

Please sign up or login with your details

Forgot password? Click here to reset