Sparsity in Optimal Randomized Classification Trees

02/21/2020
by   Rafael Blanquero, et al.
0

Decision trees are popular Classification and Regression tools and, when small-sized, easy to interpret. Traditionally, a greedy approach has been used to build the trees, yielding a very fast training process; however, controlling sparsity (a proxy for interpretability) is challenging. In recent studies, optimal decision trees, where all decisions are optimized simultaneously, have shown a better learning performance, especially when oblique cuts are implemented. In this paper, we propose a continuous optimization approach to build sparse optimal classification trees, based on oblique cuts, with the aim of using fewer predictor variables in the cuts as well as along the whole tree. Both types of sparsity, namely local and global, are modeled by means of regularizations with polyhedral norms. The computational experience reported supports the usefulness of our methodology. In all our data sets, local and global sparsity can be improved without harming classification accuracy. Unlike greedy approaches, our ability to easily trade in some of our classification accuracy for a gain in global sparsity is shown.

READ FULL TEXT

page 26

page 27

research
10/19/2021

Optimal randomized classification trees

Classification and Regression Trees (CARTs) are off-the-shelf techniques...
research
12/09/2021

On multivariate randomized classification trees: l_0-based sparsity, VC dimension and decomposition methods

Decision trees are widely-used classification and regression models beca...
research
04/28/2021

Universal Consistency of Decision Trees in High Dimensions

This paper shows that decision trees constructed with Classification and...
research
03/16/2022

Greedy Algorithms for Decision Trees with Hypotheses

We investigate at decision trees that incorporate both traditional queri...
research
10/26/2021

Learning Optimal Decision Trees Using MaxSAT

We present a Combinatorial Optimization approach based on Maximum Satisf...
research
05/05/2023

Learning Decision Trees with Gradient Descent

Decision Trees (DTs) are commonly used for many machine learning tasks d...
research
05/13/2023

A Novel Memetic Strategy for Optimized Learning of Classification Trees

Given the increasing interest in interpretable machine learning, classif...

Please sign up or login with your details

Forgot password? Click here to reset