Iteratively reweighted least squares for robust regression via SVM and ELM

03/27/2019
by   Hongwei Dong, et al.
0

The measure of most robust machine learning methods is reweighted. To overcome the optimization difficulty of the implicitly reweighted robust methods (including modifying loss functions and objectives), we try to use a more direct method: explicitly iteratively reweighted method to handle noise (even heavy-tailed noise and outlier) robustness. In this paper, an explicitly iterative reweighted framework based on two kinds of kernel based regression algorithm (LS-SVR and ELM) is established, and a novel weight selection strategy is proposed at the same time. Combining the proposed weight function with the iteratively reweighted framework, we propose two models iteratively reweighted least squares support vector machine (IRLS-SVR) and iteratively reweighted extreme learning machine (IRLS-ELM) to implement robust regression. Different from the traditional explicitly reweighted robust methods, we carry out multiple reweighted operations in our work to further improve robustness. The convergence and approximability of the proposed algorithms are proved theoretically. Moreover, the robustness of the algorithm is analyzed in detail from many angles. Experiments on both artificial data and benchmark datasets confirm the validity of the proposed methods.

READ FULL TEXT
research
03/22/2017

Randomized Kernel Methods for Least-Squares Support Vector Machines

The least-squares support vector machine is a frequently used kernel met...
research
09/26/2022

Convex Support Vector Regression

Nonparametric regression subject to convexity or concavity constraints i...
research
04/27/2021

Robust Classification via Support Vector Machines

The loss function choice for any Support Vector Machine classifier has r...
research
08/21/2022

High-Dimensional Composite Quantile Regression: Optimal Statistical Guarantees and Fast Algorithms

The composite quantile regression (CQR) was introduced by Zou and Yuan [...
research
03/18/2019

Robust Inference via Multiplier Bootstrap

This paper investigates the theoretical underpinnings of two fundamental...
research
12/24/2022

A Bayesian Robust Regression Method for Corrupted Data Reconstruction

Because of the widespread existence of noise and data corruption, recove...
research
09/29/2020

A Framework of Learning Through Empirical Gain Maximization

We develop in this paper a framework of empirical gain maximization (EGM...

Please sign up or login with your details

Forgot password? Click here to reset