DeepAI
Log In Sign Up

Dense and Sparse Coding: Theory and Architectures

06/16/2020
by   Abiy Tasissa, et al.
0

The sparse representation model has been successfully utilized in a number of signal and image processing tasks; however, recent research has highlighted its limitations in certain deep-learning architectures. This paper proposes a novel dense and sparse coding model that considers the problem of recovering a dense vector 𝐱 and a sparse vector 𝐮 given linear measurements of the form 𝐲 = 𝐀𝐱+𝐁𝐮. Our first theoretical result proposes a new natural geometric condition based on the minimal angle between subspaces corresponding to the measurement matrices 𝐀 and 𝐁 to establish the uniqueness of solutions to the linear system. The second analysis shows that, under mild assumptions and sufficient linear measurements, a convex program recovers the dense and sparse components with high probability. The standard RIPless analysis cannot be directly applied to this setup. Our proof is a non-trivial adaptation of techniques from anisotropic compressive sensing theory and is based on an analysis of a matrix derived from the measurement matrices 𝐀 and 𝐁. We begin by demonstrating the effectiveness of the proposed model on simulated data. Then, to address its use in a dictionary learning setting, we propose a dense and sparse auto-encoder (DenSaE) that is tailored to it. We demonstrate that a) DenSaE denoises natural images better than architectures derived from the sparse coding model (𝐁𝐮), b) training the biases in the latter amounts to implicitly learning the 𝐀𝐱 + 𝐁𝐮 model, and c) 𝐀 and 𝐁 capture low- and high-frequency contents, respectively.

READ FULL TEXT

page 5

page 6

page 13

page 14

page 15

05/14/2014

Group-based Sparse Representation for Image Restoration

Traditional patch-based sparse representation modeling of natural images...
07/10/2015

Riemannian Dictionary Learning and Sparse Coding for Positive Definite Matrices

Data encoded as symmetric positive definite (SPD) matrices frequently ar...
07/13/2022

A Unified Recovery of Structured Signals Using Atomic Norm

In many applications we seek to recover signals from linear measurements...
11/20/2012

Forest Sparsity for Multi-channel Compressive Sensing

In this paper, we investigate a new compressive sensing model for multi-...
07/05/2018

Deeply-Sparse Signal rePresentations (DS^2P)

The solution to the regularized least-squares problem min_x∈R^p+1/2y-Ax_...
07/23/2019

Convolutional Dictionary Learning in Hierarchical Networks

Filter banks are a popular tool for the analysis of piecewise smooth sig...
03/27/2017

Graph Regularized Tensor Sparse Coding for Image Representation

Sparse coding (SC) is an unsupervised learning scheme that has received ...