Gradient Boosted Decision Tree Neural Network

10/17/2019
by   Mohammad Saberian, et al.
0

In this paper we propose a method to build a neural network that is similar to an ensemble of decision trees. We first illustrate how to convert a learned ensemble of decision trees to a single neural network with one hidden layer and an input transformation. We then relax some properties of this network such as thresholds and activation functions to train an approximately equivalent decision tree ensemble. The final model, Hammock, is surprisingly simple: a fully connected two layers neural network where the input is quantized and one-hot encoded. Experiments on large and small datasets show this simple method can achieve performance similar to that of Gradient Boosted Decision Trees.

READ FULL TEXT

page 1

page 2

page 3

research
10/11/2022

Neural Networks are Decision Trees

In this manuscript, we show that any neural network having piece-wise li...
research
06/20/2020

Model family selection for classification using Neural Decision Trees

Model selection consists in comparing several candidate models according...
research
04/28/2021

Deep Neural Network as an alternative to Boosted Decision Trees for PID

In this paper we recreate, and improve, the binary classification method...
research
11/07/2018

YASENN: Explaining Neural Networks via Partitioning Activation Sequences

We introduce a novel approach to feed-forward neural network interpretat...
research
01/26/2019

Efficient Toxicity Prediction via Simple Features Using Shallow Neural Networks and Decision Trees

Toxicity prediction of chemical compounds is a grand challenge. Lately, ...
research
04/26/2023

Enhancing Robustness of Gradient-Boosted Decision Trees through One-Hot Encoding and Regularization

Gradient-boosted decision trees (GBDT) are widely used and highly effect...
research
04/26/2019

Formal Verification of Decision-Tree Ensemble Model and Detection of its Violating-input-value Ranges

As one type of machine-learning model, a "decision-tree ensemble model" ...

Please sign up or login with your details

Forgot password? Click here to reset