GBDT-MO: Gradient Boosted Decision Trees for Multiple Outputs

09/10/2019
by   Zhendong Zhang, et al.
0

Gradient boosted decision trees (GBDTs) are widely used in machine learning, and the output of current GBDT implementations is a single variable. When there are multiple outputs, GBDT constructs multiple trees corresponding to the output variables. In this case, the correlations between variables are ignored by such a strategy causing redundancy of the learned tree structures. In this paper, we propose a general method to learn GBDT for multiple outputs, called GBDT-MO. Each leaf of GBDT-MO constructs predictions of all variables or a subset of automatically selected variables. This is achieved by considering the summation of objective gains over all output variables. Moreover, we extend histogram approximation into multiple output case and speed up the training process by the extended one. Various experiments on synthetic and real-world datasets verify that the learning mechanism of GBDT-MO plays a role in indirect regularization. Our code is available online.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/01/2021

Tree in Tree: from Decision Trees to Decision Graphs

Decision trees have been widely used as classifiers in many machine lear...
research
10/20/2022

Improving Data Quality with Training Dynamics of Gradient Boosting Decision Trees

Real world datasets contain incorrectly labeled instances that hamper th...
research
11/23/2022

SketchBoost: Fast Gradient Boosted Decision Tree for Multioutput Problems

Gradient Boosted Decision Tree (GBDT) is a widely-used machine learning ...
research
03/27/2013

Multiple decision trees

This paper describes experiments, on two domains, to investigate the eff...
research
12/03/2020

Optimal Policy Trees

We propose an approach for learning optimal tree-based prescription poli...
research
05/18/2019

Gradient tree boosting with random output projections for multi-label classification and multi-output regression

In many applications of supervised learning, multiple classification or ...
research
07/03/2020

On Symbolically Encoding the Behavior of Random Forests

Recent work has shown that the input-output behavior of some machine lea...

Please sign up or login with your details

Forgot password? Click here to reset