Efficient Dictionary Learning with Sparseness-Enforcing Projections

by   Markus Thom, et al.

Learning dictionaries suitable for sparse coding instead of using engineered bases has proven effective in a variety of image processing tasks. This paper studies the optimization of dictionaries on image data where the representation is enforced to be explicitly sparse with respect to a smooth, normalized sparseness measure. This involves the computation of Euclidean projections onto level sets of the sparseness measure. While previous algorithms for this optimization problem had at least quasi-linear time complexity, here the first algorithm with linear time complexity and constant space complexity is proposed. The key for this is the mathematically rigorous derivation of a characterization of the projection's result based on a soft-shrinkage function. This theory is applied in an original algorithm called Easy Dictionary Learning (EZDL), which learns dictionaries with a simple and fast-to-compute Hebbian-like learning rule. The new algorithm is efficient, expressive and particularly simple to implement. It is demonstrated that despite its simplicity, the proposed learning algorithm is able to generate a rich variety of dictionaries, in particular a topographic organization of atoms or separable atoms. Further, the dictionaries are as expressive as those of benchmark learning algorithms in terms of the reproduction quality on entire images, and result in an equivalent denoising performance. EZDL learns approximately 30 faster than the already very efficient Online Dictionary Learning algorithm, and is therefore eligible for rapid data set analysis and problems with vast quantities of learning samples.


page 5

page 12

page 13


Efficient and Parallel Separable Dictionary Learning

Separable, or Kronecker product, dictionaries provide natural decomposit...

Astronomical Image Denoising Using Dictionary Learning

Astronomical images suffer a constant presence of multiple defects that ...

Learning Simple Thresholded Features with Sparse Support Recovery

The thresholded feature has recently emerged as an extremely efficient, ...

Learning Deep Analysis Dictionaries – Part II: Convolutional Dictionaries

In this paper, we introduce a Deep Convolutional Analysis Dictionary Mod...

A Split-and-Merge Dictionary Learning Algorithm for Sparse Representation

In big data image/video analytics, we encounter the problem of learning ...

Learning Multiplication-free Linear Transformations

In this paper, we propose several dictionary learning algorithms for spa...

Analyzing sparse dictionaries for online learning with kernels

Many signal processing and machine learning methods share essentially th...

Please sign up or login with your details

Forgot password? Click here to reset