Model family selection for classification using Neural Decision Trees

Model selection consists in comparing several candidate models according to a metric to be optimized. The process often involves a grid search, or such, and cross-validation, which can be time consuming, as well as not providing much information about the dataset itself. In this paper we propose a method to reduce the scope of exploration needed for the task. The idea is to quantify how much it would be necessary to depart from trained instances of a given family, reference models (RMs) carrying `rigid' decision boundaries (e.g. decision trees), so as to obtain an equivalent or better model. In our approach, this is realized by progressively relaxing the decision boundaries of the initial decision trees (the RMs) as long as this is beneficial in terms of performance measured on an analyzed dataset. More specifically, this relaxation is performed by making use of a neural decision tree, which is a neural network built from DTs. The final model produced by our method carries non-linear decision boundaries. Measuring the performance of the final model, and its agreement to its seeding RM can help the user to figure out on which family of models he should focus on.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/17/2019

Gradient Boosted Decision Tree Neural Network

In this paper we propose a method to build a neural network that is simi...
research
10/07/2022

To tree or not to tree? Assessing the impact of smoothing the decision boundaries

When analyzing a dataset, it can be useful to assess how smooth the deci...
research
01/14/2019

Towards Using Context-Dependent Symbols in CTC Without State-Tying Decision Trees

Deep neural acoustic models benefit from context dependent modeling of o...
research
10/26/2020

An Approach to Evaluating Learning Algorithms for Decision Trees

Learning algorithms produce software models for realising critical class...
research
06/05/2023

Permutation Decision Trees

Decision Tree is a well understood Machine Learning model that is based ...
research
01/10/2022

A Study on Mitigating Hard Boundaries of Decision-Tree-based Uncertainty Estimates for AI Models

Outcomes of data-driven AI models cannot be assumed to be always correct...
research
01/13/2017

What Can I Do Now? Guiding Users in a World of Automated Decisions

More and more processes governing our lives use in some part an automati...

Please sign up or login with your details

Forgot password? Click here to reset