NBDT: Neural-Backed Decision Trees

04/01/2020
by   Alvin Wan, et al.
0

Deep learning is being adopted in settings where accurate and justifiable predictions are required, ranging from finance to medical imaging. While there has been recent work providing post-hoc explanations for model predictions, there has been relatively little work exploring more directly interpretable models that can match state-of-the-art accuracy. Historically, decision trees have been the gold standard in balancing interpretability and accuracy. However, recent attempts to combine decision trees with deep learning have resulted in models that (1) achieve accuracies far lower than that of modern neural networks (e.g. ResNet) even on small datasets (e.g. MNIST), and (2) require significantly different architectures, forcing practitioners pick between accuracy and interpretability. We forgo this dilemma by creating Neural-Backed Decision Trees (NBDTs) that (1) achieve neural network accuracy and (2) require no architectural changes to a neural network. NBDTs achieve accuracy within 1 TinyImageNet, using recently state-of-the-art WideResNet; and within 2 EfficientNet on ImageNet. This yields state-of-the-art explainable models on ImageNet, with NBDTs improving the baseline by  14 Furthermore, we show interpretability of our model's decisions both qualitatively and quantitatively via a semi-automatic process. Code and pretrained NBDTs can be found at https://github.com/alvinwan/neural-backed-decision-trees.

READ FULL TEXT
research
06/11/2020

SegNBDT: Visual Decision Rules for Segmentation

The black-box nature of neural networks limits model decision interpreta...
research
09/07/2022

A Survey of Neural Trees

Neural networks (NNs) and decision trees (DTs) are both popular models o...
research
02/14/2021

Connecting Interpretability and Robustness in Decision Trees through Separation

Recent research has recognized interpretability and robustness as essent...
research
10/26/2021

Learning Optimal Decision Trees Using MaxSAT

We present a Combinatorial Optimization approach based on Maximum Satisf...
research
08/06/2021

Simple Modifications to Improve Tabular Neural Networks

There is growing interest in neural network architectures for tabular da...
research
04/30/2022

Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees

Influence estimation analyzes how changes to the training data can lead ...
research
06/11/2020

How Interpretable and Trustworthy are GAMs?

Generalized additive models (GAMs) have become a leading model class for...

Please sign up or login with your details

Forgot password? Click here to reset