DeepAI AI Chat
Log In Sign Up

A Novel M-Estimator for Robust PCA

by   Teng Zhang, et al.
University of Minnesota

We study the basic problem of robust subspace recovery. That is, we assume a data set that some of its points are sampled around a fixed subspace and the rest of them are spread in the whole ambient space, and we aim to recover the fixed underlying subspace. We first estimate "robust inverse sample covariance" by solving a convex minimization procedure; we then recover the subspace by the bottom eigenvectors of this matrix (their number correspond to the number of eigenvalues close to 0). We guarantee exact subspace recovery under some conditions on the underlying data. Furthermore, we propose a fast iterative algorithm, which linearly converges to the matrix minimizing the convex problem. We also quantify the effect of noise and regularization and discuss many other practical and theoretical issues for improving the subspace recovery in various settings. When replacing the sum of terms in the convex energy function (that we minimize) with the sum of squares of terms, we obtain that the new minimizer is a scaled version of the inverse sample covariance (when exists). We thus interpret our minimizer and its subspace (spanned by its bottom eigenvectors) as robust versions of the empirical inverse covariance and the PCA subspace respectively. We compare our method with many other algorithms for robust PCA on synthetic and real data sets and demonstrate state-of-the-art speed and accuracy.


page 31

page 33


Distributed Robust Subspace Recovery

We study Robust Subspace Recovery (RSR) in distributed settings. We cons...

Fast, Robust and Non-convex Subspace Recovery

This work presents a fast and non-convex algorithm for robust subspace r...

A Well-Tempered Landscape for Non-convex Robust Subspace Recovery

We present a mathematical analysis of a non-convex energy landscape for ...

On the Subspace of Image Gradient Orientations

We introduce the notion of Principal Component Analysis (PCA) of image g...

lp-Recovery of the Most Significant Subspace among Multiple Subspaces with Outliers

We assume data sampled from a mixture of d-dimensional linear subspaces ...

New algorithms for computing the least trimmed squares estimator

Instead of minimizing the sum of all n squared residuals as the classica...

List-Decodable Subspace Recovery via Sum-of-Squares

We give the first efficient algorithm for the problem of list-decodable ...