A Novel Splitting Criterion Inspired by Geometric Mean Metric Learning for Decision Tree

04/23/2022
by   Dan Li, et al.
0

Decision tree (DT) attracts persistent research attention due to its impressive empirical performance and interpretability in numerous applications. However, the growth of traditional yet widely-used univariate decision trees (UDTs) is quite time-consuming as they need to traverse all the features to find the splitting value with the maximal reduction of the impurity at each internal node. In this paper, we newly design a splitting criterion to speed up the growth. The criterion is induced from Geometric Mean Metric Learning (GMML) and then optimized under its diagonalized metric matrix constraint, consequently, a closed-form rank of feature discriminant abilities can at once be obtained and the top 1 feature at each node used to grow an intent DT (called as dGMML-DT, where d is an abbreviation for diagonalization). We evaluated the performance of the proposed methods and their corresponding ensembles on benchmark datasets. The experiment shows that dGMML-DT achieves comparable or better classification results more efficiently than the UDTs with 10x average speedup. Furthermore, dGMML-DT can straightforwardly be extended to its multivariable counterpart (dGMML-MDT) without needing laborious operations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/25/2020

The Max-Cut Decision Tree: Improving on the Accuracy and Running Time of Decision Trees

Decision trees are a widely used method for classification, both by them...
research
04/13/2023

Heterogeneous Oblique Double Random Forest

The decision tree ensembles use a single data feature at each node for s...
research
05/11/2022

Subspace Learning Machine (SLM): Methodology and Performance

Inspired by the feedforward multilayer perceptron (FF-MLP), decision tre...
research
07/18/2016

Geometric Mean Metric Learning

We revisit the task of learning a Euclidean metric from data. We approac...
research
09/18/2023

Des-q: a quantum algorithm to construct and efficiently retrain decision trees for regression and binary classification

Decision trees are widely used in machine learning due to their simplici...
research
02/27/2019

Neural Packet Classification

Packet classification is a fundamental problem in computer networking. T...

Please sign up or login with your details

Forgot password? Click here to reset