DeepAI AI Chat
Log In Sign Up

Implicit Bias of Projected Subgradient Method Gives Provable Robust Recovery of Subspaces of Unknown Codimension

by   Paris V. Giampouras, et al.
Johns Hopkins University

Robust subspace recovery (RSR) is a fundamental problem in robust representation learning. Here we focus on a recently proposed RSR method termed Dual Principal Component Pursuit (DPCP) approach, which aims to recover a basis of the orthogonal complement of the subspace and is amenable to handling subspaces of high relative dimension. Prior work has shown that DPCP can provably recover the correct subspace in the presence of outliers, as long as the true dimension of the subspace is known. We show that DPCP can provably solve RSR problems in the unknown subspace dimension regime, as long as orthogonality constraints – adopted in previous DPCP formulations – are relaxed and random initialization is used instead of spectral one. Namely, we propose a very simple algorithm based on running multiple instances of a projected sub-gradient descent method (PSGM), with each problem instance seeking to find one vector in the null space of the subspace. We theoretically prove that under mild conditions this approach will succeed with high probability. In particular, we show that 1) all of the problem instances will converge to a vector in the nullspace of the subspace and 2) the ensemble of problem instance solutions will be sufficiently diverse to fully span the nullspace of the subspace thus also revealing its true unknown codimension. We provide empirical results that corroborate our theoretical results and showcase the remarkable implicit rank regularization behavior of PSGM algorithm that allows us to perform RSR without being aware of the subspace dimension.


Dual Principal Component Pursuit: Probability Analysis and Efficient Algorithms

Recent methods for learning a linear subspace from data corrupted by out...

Dual Principal Component Pursuit

We consider the problem of outlier rejection in single subspace learning...

Robust Recovery of Subspace Structures by Low-Rank Representation

In this work we address the subspace recovery problem. Given a set of da...

List Decodable Subspace Recovery

Learning from data in the presence of outliers is a fundamental problem ...

Exact Subspace Segmentation and Outlier Detection by Low-Rank Representation

In this work, we address the following matrix recovery problem: suppose ...

A Well-Tempered Landscape for Non-convex Robust Subspace Recovery

We present a mathematical analysis of a non-convex energy landscape for ...

Subspace Tracking from Missing and Outlier Corrupted Data

We study the related problems of subspace tracking in the presence of mi...