DeepAI AI Chat
Log In Sign Up

Mixed integer linear optimization formulations for learning optimal binary classification trees

by   Brandon Alston, et al.

Decision trees are powerful tools for classification and regression that attract many researchers working in the burgeoning area of machine learning. One advantage of decision trees over other methods is their interpretability, which is often preferred over other higher accuracy methods that are relatively uninterpretable. A binary classification tree has two types of vertices: (i) branching vertices which have exactly two children and where datapoints are assessed on a set of discrete features; and (ii) leaf vertices at which datapoints are given a discrete prediction. An optimal binary classification tree can be obtained by solving a biobjective optimization problem that seeks to (i) maximize the number of correctly classified datapoints and (ii) minimize the number of branching vertices. In this paper, we propose four mixed integer linear optimization (MILO) formulations for designing optimal binary classification trees: two flow-based formulations and two-cut based formulations. We provide theoretical comparisons between our proposed formulations and the strongest flow-based MILO formulation of Aghaei et al. (2021). We conduct experiments on 13 publicly available datasets to show the models' ability to scale and the strength of a biobjective approach using Pareto frontiers. Our code and data are available on GitHub.


page 1

page 2

page 3

page 4


Strong Optimal Classification Trees

Decision trees are among the most popular machine learning models and ar...

Learning Optimal Classification Trees: Strong Max-Flow Formulations

We consider the problem of learning optimal binary classification trees....

Learning Binary Trees via Sparse Relaxation

One of the most classical problems in machine learning is how to learn b...

Margin Optimal Classification Trees

In recent years there has been growing attention to interpretable machin...

Tightness of prescriptive tree-based mixed-integer optimization formulations

We focus on modeling the relationship between an input feature vector an...

On Using Linear Diophantine Equations to Tune the extent of Look Ahead while Hiding Decision Tree Rules

This paper focuses on preserving the privacy of sensitive pat-terns when...

Robust Optimal Classification Trees under Noisy Labels

In this paper we propose a novel methodology to construct Optimal Classi...

Code Repositories