Optimal Decision Trees for Nonlinear Metrics

09/15/2020
by   Emir Demirović, et al.
0

Nonlinear metrics, such as the F1-score, Matthews correlation coefficient, and Fowlkes-Mallows index, are often used to evaluate the performance of machine learning models, in particular, when facing imbalanced datasets that contain more samples of one class than the other. Recent optimal decision tree algorithms have shown remarkable progress in producing trees that are optimal with respect to linear criteria, such as accuracy, but unfortunately nonlinear metrics remain a challenge. To address this gap, we propose a novel algorithm based on bi-objective optimisation, which treats misclassifications of each binary class as a separate objective. We show that, for a large class of metrics, the optimal tree lies on the Pareto frontier. Consequently, we obtain the optimal tree by using our method to generate the set of all nondominated trees. To the best of our knowledge, this is the first method to compute provably optimal decision trees for nonlinear metrics. Our approach leads to a trade-off when compared to optimising linear metrics: the resulting trees may be more desirable according to the given nonlinear metric at the expense of higher runtimes. Nevertheless, the experiments illustrate that runtimes are reasonable for majority of the tested datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/14/2023

Scalable Optimal Multiway-Split Decision Trees with Constraints

There has been a surge of interest in learning optimal decision trees us...
research
10/13/2020

Succinct Explanations With Cascading Decision Trees

Classic decision tree learning is a binary classification algorithm that...
research
09/16/2022

Exploring the Whole Rashomon Set of Sparse Decision Trees

In any given machine learning problem, there may be many models that cou...
research
07/24/2020

MurTree: Optimal Classification Trees via Dynamic Programming and Search

Decision tree learning is a widely used approach in machine learning, fa...
research
05/16/2022

The Influence of Dimensions on the Complexity of Computing Decision Trees

A decision tree recursively splits a feature space ℝ^d and then assigns ...
research
06/16/2016

ACDC: α-Carving Decision Chain for Risk Stratification

In many healthcare settings, intuitive decision rules for risk stratific...

Please sign up or login with your details

Forgot password? Click here to reset