Provably Accurate Double-Sparse Coding

11/09/2017
by   Thanh V. Nguyen, et al.
0

Sparse coding is a crucial subroutine in algorithms for various signal processing, deep learning, and other machine learning applications. The central goal is to learn an overcomplete dictionary that can sparsely represent a given input dataset. However, a key challenge is that storage, transmission, and processing of the learned dictionary can be untenably high if the data dimension is high. In this paper, we consider the double-sparsity model introduced by Rubinstein et al. (2010b) where the dictionary itself is the product of a fixed, known basis and a data-adaptive sparse component. First, we introduce a simple algorithm for double-sparse coding that can be amenable to efficient implementation via neural architectures. Second, we theoretically analyze its performance and demonstrate asymptotic sample complexity and running time benefits over existing (provable) approaches for sparse coding. To our knowledge, our work introduces the first computationally efficient algorithm for double-sparse coding that enjoys rigorous statistical guarantees. Finally, we support our analysis via several numerical experiments on simulated data, confirming that our method can indeed be useful in problem sizes encountered in practical applications.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

08/28/2013

New Algorithms for Learning Incoherent and Overcomplete Dictionaries

In sparse recovery we are given a matrix A (the dictionary) and a vector...
03/02/2015

Simple, Efficient, and Neural Algorithms for Sparse Coding

Sparse coding is a basic task in many fields including signal processing...
07/05/2018

Deeply-Sparse Signal rePresentations (DS^2P)

The solution to the regularized least-squares problem min_x∈R^p+1/2y-Ax_...
01/03/2014

More Algorithms for Provable Dictionary Learning

In dictionary learning, also known as sparse coding, the algorithm is gi...
11/24/2020

The Interpretable Dictionary in Sparse Coding

Artificial neural networks (ANNs), specifically deep learning networks, ...
02/18/2012

On the Sample Complexity of Predictive Sparse Coding

The goal of predictive sparse coding is to learn a representation of exa...
05/09/2012

Convex Coding

Inspired by recent work on convex formulations of clustering (Lashkari &...

Code Repositories

double-sparse-coding

Matlab code for learning doubly sparse dictionary on synthetic data. Details can be found in the paper "A Provable Approach for Double-Sparse Coding".


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.