Large-Scale Low-Rank Matrix Learning with Nonconvex Regularizers

08/01/2017
by   Quanming Yao, et al.
0

Low-rank modeling has many important applications in computer vision and machine learning. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better empirical performance. However, the resulting optimization problem is much more challenging. Recent state-of-the-art requires an expensive full SVD in each iteration. In this paper, we show that for many commonly-used nonconvex low-rank regularizers, a cutoff can be derived to automatically threshold the singular values obtained from the proximal operator. This allows such operator being efficiently approximated by power method. Based on it, we develop a proximal gradient algorithm (and its accelerated variant) with inexact proximal splitting and prove that a convergence rate of O(1/T) where T is the number of iterations is guaranteed. Furthermore, we show the proposed algorithm can be well parallelized, which achieves nearly linear speedup w.r.t the number of threads. Extensive experiments are performed on matrix completion and robust principal component analysis, which shows a significant speedup over the state-of-the-art. Moreover, the matrix solution obtained is more accurate and has a lower rank than that of the nuclear norm regularizer.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/03/2015

Fast Low-Rank Matrix Learning with Nonconvex Regularization

Low-rank modeling has a lot of important applications in machine learnin...
research
03/16/2017

Accelerated and Inexact Soft-Impute for Large-Scale Matrix and Tensor Completion

Matrix and tensor completion aim to recover a low-rank matrix / tensor f...
research
05/06/2022

Low-rank Tensor Learning with Nonconvex Overlapped Nuclear Norm Regularization

Nonconvex regularization has been popularly used in low-rank matrix lear...
research
10/27/2020

Learning Low-Rank Document Embeddings with Weighted Nuclear Norm Regularization

Recently, neural embeddings of documents have shown success in various l...
research
11/17/2015

Robust PCA via Nonconvex Rank Approximation

Numerous applications in data mining and machine learning require recove...
research
07/23/2018

Scalable Tensor Completion with Nonconvex Regularization

Low-rank tensor completion problem aims to recover a tensor from limited...
research
03/29/2014

Scalable Robust Matrix Recovery: Frank-Wolfe Meets Proximal Methods

Recovering matrices from compressive and grossly corrupted observations ...

Please sign up or login with your details

Forgot password? Click here to reset