Soft Gradient Boosting Machine

06/07/2020
by   Ji Feng, et al.
0

Gradient Boosting Machine has proven to be one successful function approximator and has been widely used in a variety of areas. However, since the training procedure of each base learner has to take the sequential order, it is infeasible to parallelize the training process among base learners for speed-up. In addition, under online or incremental learning settings, GBMs achieved sub-optimal performance due to the fact that the previously trained base learners can not adapt with the environment once trained. In this work, we propose the soft Gradient Boosting Machine (sGBM) by wiring multiple differentiable base learners together, by injecting both local and global objectives inspired from gradient boosting, all base learners can then be jointly optimized with linear speed-up. When using differentiable soft decision trees as base learner, such device can be regarded as an alternative version of the (hard) gradient boosting decision trees with extra benefits. Experimental results showed that, sGBM enjoys much higher time efficiency with better accuracy, given the same base learner in both on-line and off-line settings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/15/2018

Gradient Boosting With Piece-Wise Linear Regression Trees

Gradient boosting using decision trees as base learners, so called Gradi...
research
06/06/2022

Multi-learner risk reduction under endogenous participation dynamics

Prediction systems face exogenous and endogenous distribution shift – th...
research
02/21/2023

Variational Boosted Soft Trees

Gradient boosting machines (GBMs) based on decision trees consistently d...
research
06/17/2020

MixBoost: A Heterogeneous Boosting Machine

Modern gradient boosting software frameworks, such as XGBoost and LightG...
research
09/12/2021

Feature Importance in Gradient Boosting Trees with Cross-Validation Feature Selection

Gradient Boosting Machines (GBM) are among the go-to algorithms on tabul...
research
03/20/2019

Accelerating Gradient Boosting Machine

Gradient Boosting Machine (GBM) is an extremely powerful supervised lear...
research
05/18/2019

Gradient tree boosting with random output projections for multi-label classification and multi-output regression

In many applications of supervised learning, multiple classification or ...

Please sign up or login with your details

Forgot password? Click here to reset