Kernel-based L_2-Boosting with Structure Constraints

09/16/2020
by   Yao Wang, et al.
0

Developing efficient kernel methods for regression is very popular in the past decade. In this paper, utilizing boosting on kernel-based weaker learners, we propose a novel kernel-based learning algorithm called kernel-based re-scaled boosting with truncation, dubbed as KReBooT. The proposed KReBooT benefits in controlling the structure of estimators and producing sparse estimate, and is near overfitting resistant. We conduct both theoretical analysis and numerical simulations to illustrate the power of KReBooT. Theoretically, we prove that KReBooT can achieve the almost optimal numerical convergence rate for nonlinear approximation. Furthermore, using the recently developed integral operator approach and a variant of Talagrand's concentration inequality, we provide fast learning rates for KReBooT, which is a new record of boosting-type algorithms. Numerically, we carry out a series of simulations to show the promising performance of KReBooT in terms of its good generalization, near over-fitting resistance and structure constraints.

READ FULL TEXT
research
05/06/2015

Re-scale boosting for regression and classification

Boosting is a learning scheme that combines weak prediction rules to pro...
research
09/15/2022

A new Kernel Regression approach for Robustified L_2 Boosting

We investigate L_2 boosting in the context of kernel regression. Kernel ...
research
10/09/2015

Functional Frank-Wolfe Boosting for General Loss Functions

Boosting is a generic learning method for classification and regression....
research
02/11/2019

KTBoost: Combined Kernel and Tree Boosting

In this article, we introduce a novel boosting algorithm called `KTBoost...
research
04/01/2020

Fully-Corrective Gradient Boosting with Squared Hinge: Fast Learning Rates and Early Stopping

Boosting is a well-known method for improving the accuracy of weak learn...
research
06/14/2019

Learning Landmark-Based Ensembles with Random Fourier Features and Gradient Boosting

We propose a Gradient Boosting algorithm for learning an ensemble of ker...
research
11/24/2019

Fast Polynomial Kernel Classification for Massive Data

In the era of big data, it is highly desired to develop efficient machin...

Please sign up or login with your details

Forgot password? Click here to reset