Analyzing the tree-layer structure of Deep Forests

10/29/2020
by   Ludovic Arnould, et al.
12

Random forests on the one hand, and neural networks on the other hand, have met great success in the machine learning community for their predictive performance. Combinations of both have been proposed in the literature, notably leading to the so-called deep forests (DF) [25]. In this paper, we investigate the mechanisms at work in DF and outline that DF architecture can generally be simplified into more simple and computationally efficient shallow forests networks. Despite some instability, the latter may outperform standard predictive tree-based methods. In order to precisely quantify the improvement achieved by these light network configurations over standard tree learners, we theoretically study the performance of a shallow tree network made of two layers, each one composed of a single centered tree. We provide tight theoretical lower and upper bounds on its excess risk. These theoretical results show the interest of tree-network architectures for well-structured data provided that the first layer, acting as a data encoder, is rich enough.

READ FULL TEXT

page 5

page 21

page 22

page 23

research
06/26/2019

A Debiased MDI Feature Importance Measure for Random Forests

Tree ensembles such as Random Forests have achieved impressive empirical...
research
04/11/2022

Random Similarity Forests

The wealth of data being gathered about humans and their surroundings dr...
research
05/25/2019

Asymptotic Distributions and Rates of Convergence for Random Forests and other Resampled Ensemble Learners

Random forests remain among the most popular off-the-shelf supervised le...
research
02/10/2023

Conceptual Views on Tree Ensemble Classifiers

Random Forests and related tree-based methods are popular for supervised...
research
04/07/2023

HyperTab: Hypernetwork Approach for Deep Learning on Small Tabular Datasets

Deep learning has achieved impressive performance in many domains, such ...
research
02/23/2021

Provable Boolean Interaction Recovery from Tree Ensemble obtained via Random Forests

Random Forests (RF) are at the cutting edge of supervised machine learni...
research
12/03/2014

Deep Distributed Random Samplings for Supervised Learning: An Alternative to Random Forests?

In (zhang2014nonlinear,zhang2014nonlinear2), we have viewed machine lear...

Please sign up or login with your details

Forgot password? Click here to reset