Gradient Boosted Binary Histogram Ensemble for Large-scale Regression

06/03/2021
by   Hanyuan Hang, et al.
0

In this paper, we propose a gradient boosting algorithm for large-scale regression problems called Gradient Boosted Binary Histogram Ensemble (GBBHE) based on binary histogram partition and ensemble learning. From the theoretical perspective, by assuming the Hölder continuity of the target function, we establish the statistical convergence rate of GBBHE in the space C^0,α and C^1,0, where a lower bound of the convergence rate for the base learner demonstrates the advantage of boosting. Moreover, in the space C^1,0, we prove that the number of iterations to achieve the fast convergence rate can be reduced by using ensemble regressor as the base learner, which improves the computational efficiency. In the experiments, compared with other state-of-the-art algorithms such as gradient boosted regression tree (GBRT), Breiman's forest, and kernel-based methods, our GBBHE algorithm shows promising performance with less running time on large-scale datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/05/2021

Local Adaptivity of Gradient Boosting in Histogram Transform Ensemble Learning

In this paper, we propose a gradient boosting algorithm called adaptive ...
research
12/08/2019

Histogram Transform Ensembles for Large-scale Regression

We propose a novel algorithm for large-scale regression problems named h...
research
06/10/2021

GBHT: Gradient Boosting Histogram Transform for Density Estimation

In this paper, we propose a density estimation algorithm called Gradient...
research
05/06/2015

Re-scale boosting for regression and classification

Boosting is a learning scheme that combines weak prediction rules to pro...
research
07/02/2017

Stochastic Configuration Networks Ensemble for Large-Scale Data Analytics

This paper presents a fast decorrelated neuro-ensemble with heterogeneou...
research
09/09/2021

Almost sure convergence of the accelerated weight histogram algorithm

The accelerated weight histogram (AWH) algorithm is an iterative extende...

Please sign up or login with your details

Forgot password? Click here to reset