Random boosting and random^2 forests – A random tree depth injection approach

09/13/2020
by   Tobias Markus Krabel, et al.
0

The induction of additional randomness in parallel and sequential ensemble methods has proven to be worthwhile in many aspects. In this manuscript, we propose and examine a novel random tree depth injection approach suitable for sequential and parallel tree-based approaches including Boosting and Random Forests. The resulting methods are called Random Boost and Random^2 Forest. Both approaches serve as valuable extensions to the existing literature on the gradient boosting framework and random forests. A Monte Carlo simulation, in which tree-shaped data sets with different numbers of final partitions are built, suggests that there are several scenarios where Random Boost and Random^2 Forest can improve the prediction performance of conventional hierarchical boosting and random forest approaches. The new algorithms appear to be especially successful in cases where there are merely a few high-order interactions in the generated data. In addition, our simulations suggest that our random tree depth injection approach can improve computation time by up to 40 terms of prediction accuracy turn out to be minor or even negligible in most cases.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/21/2018

Boosting Random Forests to Reduce Bias; One-Step Boosted Forest and its Variance Estimate

In this paper we propose using the principle of boosting to reduce the b...
research
04/28/2015

Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers

There is a large literature explaining why AdaBoost is a successful clas...
research
02/24/2021

Generalised Boosted Forests

This paper extends recent work on boosting random forests to model non-G...
research
09/04/2023

Hidden variables unseen by Random Forests

Random Forests are widely claimed to capture interactions well. However,...
research
05/24/2018

Prediction of Autism Treatment Response from Baseline fMRI using Random Forests and Tree Bagging

Treating children with autism spectrum disorders (ASD) with behavioral i...
research
05/09/2019

Two-stage Best-scored Random Forest for Large-scale Regression

We propose a novel method designed for large-scale regression problems, ...
research
09/30/2022

Sparse tree-based initialization for neural networks

Dedicated neural network (NN) architectures have been designed to handle...

Please sign up or login with your details

Forgot password? Click here to reset