DeepAI AI Chat
Log In Sign Up

MurTree: Optimal Classification Trees via Dynamic Programming and Search

07/24/2020
by   Emir Demirović, et al.
LAAS-CNRS
Institute of Science and Technology Austria
Monash University
The University of Melbourne
Delft University of Technology
RMIT University
0

Decision tree learning is a widely used approach in machine learning, favoured in applications that require concise and interpretable models. Heuristic methods are traditionally used to quickly produce models with reasonably high accuracy. A commonly criticised point, however, is that the resulting trees may not necessarily be the best representation of the data in terms of accuracy, size, and other considerations such as fairness. In recent years, this motivated the development of optimal classification tree algorithms that globally optimise the decision tree in contrast to heuristic methods that perform a sequence of locally optimal decisions. We follow this line of work and provide a novel algorithm for learning optimal classification trees based on dynamic programming and search. Our algorithm supports constraints on the depth of the tree and number of nodes and we argue it can be extended with other requirements. The success of our approach is attributed to a series of specialised techniques that exploit properties unique to classification trees. Whereas algorithms for optimal classification trees have traditionally been plagued by high runtimes and limited scalability, we show in a detailed experimental study that our approach uses only a fraction of the time required by the state-of-the-art and can handle datasets with tens of thousands of instances, providing several orders of magnitude improvements and notably contributing towards the practical realisation of optimal decision trees.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/29/2019

Optimal Sparse Decision Trees

Decision tree algorithms have been among the most popular algorithms for...
07/11/2019

On the Optimality of Trees Generated by ID3

Since its inception in the 1980s, ID3 has become one of the most success...
12/01/2021

How Smart Guessing Strategies Can Yield Massive Scalability Improvements for Sparse Decision Tree Optimization

Sparse decision tree optimization has been one of the most fundamental p...
04/13/2021

Deep imagination is a close to optimal policy for planning in large decision trees under limited resources

Many decisions involve choosing an uncertain course of actions in deep a...
09/15/2020

Optimal Decision Trees for Nonlinear Metrics

Nonlinear metrics, such as the F1-score, Matthews correlation coefficien...
10/19/2021

Optimal randomized classification trees

Classification and Regression Trees (CARTs) are off-the-shelf techniques...
02/21/2020

Sparsity in Optimal Randomized Classification Trees

Decision trees are popular Classification and Regression tools and, when...