Alternating minimization for dictionary learning with random initialization

11/09/2017
by   Niladri S. Chatterji, et al.
0

We present theoretical guarantees for an alternating minimization algorithm for the dictionary learning/sparse coding problem. The dictionary learning problem is to factorize vector samples y^1,y^2,..., y^n into an appropriate basis (dictionary) A^* and sparse vectors x^1*,...,x^n*. Our algorithm is a simple alternating minimization procedure that switches between ℓ_1 minimization and gradient descent in alternate steps. Dictionary learning and specifically alternating minimization algorithms for dictionary learning are well studied both theoretically and empirically. However, in contrast to previous theoretical analyses for this problem, we replace the condition on the operator norm (that is, the largest magnitude singular value) of the true underlying dictionary A^* with a condition on the matrix infinity norm (that is, the largest magnitude term). This not only allows us to get convergence rates for the error of the estimated dictionary measured in the matrix infinity norm, but also ensures that a random initialization will provably converge to the global optimum. Our guarantees are under a reasonable generative model that allows for dictionaries with growing operator norms, and can handle an arbitrary level of overcompleteness, while having sparsity that is information theoretically optimal. We also establish upper bounds on the sample complexity of our algorithm.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/23/2022

Simple Alternating Minimization Provably Solves Complete Dictionary Learning

This paper focuses on complete dictionary learning problem, where the go...
research
04/17/2016

Identifying global optimality for dictionary learning

Learning new representations of input observations in machine learning i...
research
10/25/2018

Subgradient Descent Learns Orthogonal Dictionaries

This paper concerns dictionary learning, i.e., sparse coding, a fundamen...
research
02/01/2018

Analysis of Fast Alternating Minimization for Structured Dictionary Learning

Methods exploiting sparsity have been popular in imaging and signal proc...
research
05/31/2018

Analysis of Fast Structured Dictionary Learning

Sparsity-based models and techniques have been exploited in many signal ...
research
06/06/2019

Complete Dictionary Learning via ℓ^4-Norm Maximization over the Orthogonal Group

This paper considers the fundamental problem of learning a complete (ort...
research
02/22/2019

Unique Sharp Local Minimum in ℓ_1-minimization Complete Dictionary Learning

We study the problem of globally recovering a dictionary from a set of s...

Please sign up or login with your details

Forgot password? Click here to reset