DeepAI AI Chat
Log In Sign Up

Kernel-based L_2-Boosting with Structure Constraints

by   Yao Wang, et al.

Developing efficient kernel methods for regression is very popular in the past decade. In this paper, utilizing boosting on kernel-based weaker learners, we propose a novel kernel-based learning algorithm called kernel-based re-scaled boosting with truncation, dubbed as KReBooT. The proposed KReBooT benefits in controlling the structure of estimators and producing sparse estimate, and is near overfitting resistant. We conduct both theoretical analysis and numerical simulations to illustrate the power of KReBooT. Theoretically, we prove that KReBooT can achieve the almost optimal numerical convergence rate for nonlinear approximation. Furthermore, using the recently developed integral operator approach and a variant of Talagrand's concentration inequality, we provide fast learning rates for KReBooT, which is a new record of boosting-type algorithms. Numerically, we carry out a series of simulations to show the promising performance of KReBooT in terms of its good generalization, near over-fitting resistance and structure constraints.


Re-scale boosting for regression and classification

Boosting is a learning scheme that combines weak prediction rules to pro...

A new Kernel Regression approach for Robustified L_2 Boosting

We investigate L_2 boosting in the context of kernel regression. Kernel ...

Functional Frank-Wolfe Boosting for General Loss Functions

Boosting is a generic learning method for classification and regression....

KTBoost: Combined Kernel and Tree Boosting

In this article, we introduce a novel boosting algorithm called `KTBoost...

Boosting as a kernel-based method

Boosting combines weak (biased) learners to obtain effective learning al...

Fully-Corrective Gradient Boosting with Squared Hinge: Fast Learning Rates and Early Stopping

Boosting is a well-known method for improving the accuracy of weak learn...

Learning Landmark-Based Ensembles with Random Fourier Features and Gradient Boosting

We propose a Gradient Boosting algorithm for learning an ensemble of ker...