DeepAI AI Chat
Log In Sign Up

Online dictionary learning for kernel LMS. Analysis and forward-backward splitting algorithm

by   Wei Gao, et al.
Université Nice Sophia Antipolis

Adaptive filtering algorithms operating in reproducing kernel Hilbert spaces have demonstrated superiority over their linear counterpart for nonlinear system identification. Unfortunately, an undesirable characteristic of these methods is that the order of the filters grows linearly with the number of input data. This dramatically increases the computational burden and memory requirement. A variety of strategies based on dictionary learning have been proposed to overcome this severe drawback. Few, if any, of these works analyze the problem of updating the dictionary in a time-varying environment. In this paper, we present an analytical study of the convergence behavior of the Gaussian least-mean-square algorithm in the case where the statistics of the dictionary elements only partially match the statistics of the input data. This allows us to emphasize the need for updating the dictionary in an online way, by discarding the obsolete elements and adding appropriate ones. We introduce a kernel least-mean-square algorithm with L1-norm regularization to automatically perform this task. The stability in the mean of this method is analyzed, and its performance is tested with experiments.


page 1

page 2

page 3

page 4


Improving Sparsity in Kernel Adaptive Filters Using a Unit-Norm Dictionary

Kernel adaptive filters, a class of adaptive nonlinear time-series model...

Kernel Least Mean Square with Adaptive Kernel Size

Kernel adaptive filters (KAF) are a class of powerful nonlinear filters ...

Random Inverse Problems Over Graphs: Decentralized Online Learning

We establish a framework of random inverse problems with real-time obser...

Study of Set-Membership Adaptive Kernel Algorithms

In the last decade, a considerable research effort has been devoted to d...

Online Graph Dictionary Learning

Dictionary learning is a key tool for representation learning, that expl...

Study of Frequency domain exponential functional link network filters

The exponential functional link network (EFLN) filter has attracted trem...