Online Orthogonal Dictionary Learning Based on Frank-Wolfe Method

by   Ye Xue, et al.

Dictionary learning is a widely used unsupervised learning method in signal processing and machine learning. Most existing works of dictionary learning are in an offline manner. There are mainly two offline ways for dictionary learning. One is to do an alternative optimization of both the dictionary and the sparse code; the other way is to optimize the dictionary by restricting it over the orthogonal group. The latter one is called orthogonal dictionary learning which has a lower complexity implementation, hence, it is more favorable for lowcost devices. However, existing schemes on orthogonal dictionary learning only work with batch data and can not be implemented online, which is not applicable for real-time applications. This paper proposes a novel online orthogonal dictionary scheme to dynamically learn the dictionary from streaming data without storing the historical data. The proposed scheme includes a novel problem formulation and an efficient online algorithm design with convergence analysis. In the problem formulation, we relax the orthogonal constraint to enable an efficient online algorithm. In the algorithm design, we propose a new Frank-Wolfe-based online algorithm with a convergence rate of O(ln t/t^(1/4)). The convergence rate in terms of key system parameters is also derived. Experiments with synthetic data and real-world sensor readings demonstrate the effectiveness and efficiency of the proposed online orthogonal dictionary learning scheme.



There are no comments yet.


page 1

page 8


Online Convex Dictionary Learning

Dictionary learning is a dimensionality reduction technique widely used ...

Manifold Proximal Point Algorithms for Dual Principal Component Pursuit and Orthogonal Dictionary Learning

We consider the problem of maximizing the ℓ_1 norm of a linear map over ...

Neurogenesis-Inspired Dictionary Learning: Online Model Adaption in a Changing World

In this paper, we focus on online representation learning in non-station...

Complete Dictionary Learning via ℓ_p-norm Maximization

Dictionary learning is a classic representation learning method that has...

Fault Handling in Large Water Networks with Online Dictionary Learning

Fault detection and isolation in water distribution networks is an activ...

A Theoretically Novel Trade-off for Sparse Secret-key Generation

We in this paper theoretically go over a rate-distortion based sparse di...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.