Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picture

11/11/2015
by   Ju Sun, et al.
0

We consider the problem of recovering a complete (i.e., square and invertible) matrix A_0, from Y ∈R^n × p with Y = A_0 X_0, provided X_0 is sufficiently sparse. This recovery problem is central to theoretical understanding of dictionary learning, which seeks a sparse representation for a collection of input signals and finds numerous applications in modern signal processing and machine learning. We give the first efficient algorithm that provably recovers A_0 when X_0 has O(n) nonzeros per column, under suitable probability model for X_0. In contrast, prior results based on efficient algorithms either only guarantee recovery when X_0 has O(√(n)) zeros per column, or require multiple rounds of SDP relaxation to work when X_0 has O(n^1-δ) nonzeros per column (for any constant δ∈ (0, 1)). Our algorithmic pipeline centers around solving a certain nonconvex optimization problem with a spherical constraint. In this paper, we provide a geometric characterization of the objective landscape. In particular, we show that the problem is highly structured: with high probability, (1) there are no "spurious" local minimizers; and (2) around all saddle points the objective has a negative directional curvature. This distinctive structure makes the problem amenable to efficient optimization algorithms. In a companion paper (arXiv:1511.04777), we design a second-order trust-region algorithm over the sphere that provably converges to a local minimizer from arbitrary initializations, despite the presence of saddle points.

READ FULL TEXT

page 3

page 10

research
04/26/2015

Complete Dictionary Recovery over the Sphere

We consider the problem of recovering a complete (i.e., square and inver...
research
11/15/2015

Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Method

We consider the problem of recovering a complete (i.e., square and inver...
research
01/20/2020

Finding the Sparsest Vectors in a Subspace: Theory, Algorithms, and Applications

The problem of finding the sparsest vector (direction) in a low dimensio...
research
10/21/2015

When Are Nonconvex Problems Not Scary?

In this note, we focus on smooth nonconvex optimization problems that ob...
research
02/24/2020

Complete Dictionary Learning via ℓ_p-norm Maximization

Dictionary learning is a classic representation learning method that has...
research
05/05/2020

Manifold Proximal Point Algorithms for Dual Principal Component Pursuit and Orthogonal Dictionary Learning

We consider the problem of maximizing the ℓ_1 norm of a linear map over ...
research
11/16/2021

Robust recovery for stochastic block models

We develop an efficient algorithm for weak recovery in a robust version ...

Please sign up or login with your details

Forgot password? Click here to reset