DeepAI AI Chat
Log In Sign Up

Contraction Principle based Robust Iterative Algorithms for Machine Learning

by   Rangeet Mitra, et al.

Iterative algorithms are ubiquitous in the field of data mining. Widely known examples of such algorithms are the least mean square algorithm, backpropagation algorithm of neural networks. Our contribution in this paper is an improvement upon this iterative algorithms in terms of their respective performance metrics and robustness. This improvement is achieved by a new scaling factor which is multiplied to the error term. Our analysis shows that in essence, we are minimizing the corresponding LASSO cost function, which is the reason of its increased robustness. We also give closed form expressions for the number of iterations for convergence and the MSE floor of the original cost function for a minimum targeted value of the L1 norm. As a concluding theme based on the stochastic subgradient algorithm, we give a comparison between the well known Dantzig selector and our algorithm based on contraction principle. By these simulations we attempt to show the optimality of our approach for any widely used parent iterative optimization problem.


page 1

page 2

page 3

page 4


l1-norm Penalized Orthogonal Forward Regression

A l1-norm penalized orthogonal forward regression (l1-POFR) algorithm is...

Non-convex optimization via strongly convex majoirziation-minimization

In this paper, we introduce a class of nonsmooth nonconvex least square ...

On a fixed-point continuation method for a convex optimization problem

We consider a variation of the classical proximal-gradient algorithm for...

Efficient ML Direction of Arrival Estimation assuming Unknown Sensor Noise Powers

This paper presents an efficient method for computing maximum likelihood...

Study of Robust Distributed Diffusion RLS Algorithms with Side Information for Adaptive Networks

This work develops robust diffusion recursive least squares algorithms t...

Sparse LMS via Online Linearized Bregman Iteration

We propose a version of least-mean-square (LMS) algorithm for sparse sys...

On Many-to-Many Mapping Between Concordance Correlation Coefficient and Mean Square Error

The concordance correlation coefficient (CCC) is one of the most widely ...