Explaining random forest prediction through diverse rulesets

03/29/2022
by   Klest Dedja, et al.
6

Tree-ensemble algorithms, such as random forest, are effective machine learning methods popular for their flexibility, high performance, and robustness to overfitting. However, since multiple learners are combined,they are not as interpretable as a single decision tree. In this work we propose a methodology, called Local Tree eXtractor (LTreeX) which is able to explain the forest prediction for a given test instance with a few diverse rules. Starting from the decision trees generated by a random forest, our method 1) pre-selects a subset of them, 2) creates a vector representation, and 3) eventually clusters such a representation. Each cluster prototype results in a rule that explains the test instance prediction. We test the effectiveness of LTreeX on 71 real-world datasets and we demonstrate the validity of our approach for binary classification, regression, multi-label classification and time-to-event tasks. In all set-ups, we show that our extracted surrogate model manages to approximate the performance of the corresponding ensemble model, while selecting only few trees from the whole forest.We also show that our proposed approach substantially outperforms other explainable methods in terms of predictive performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/13/2020

Rule Covering for Interpretation and Boosting

We propose two algorithms for interpretation and boosting of tree-based ...
research
07/05/2022

Local Multi-Label Explanations for Random Forest

Multi-label classification is a challenging task, particularly in domain...
research
07/13/2022

Contextual Decision Trees

Focusing on Random Forests, we propose a multi-armed contextual bandit r...
research
04/13/2021

Conclusive Local Interpretation Rules for Random Forests

In critical situations involving discrimination, gender inequality, econ...
research
09/25/2018

RAFP-Pred: Robust Prediction of Antifreeze Proteins using Localized Analysis of n-Peptide Compositions

In extreme cold weather, living organisms produce Antifreeze Proteins (A...
research
02/16/2017

Tree Ensembles with Rule Structured Horseshoe Regularization

We propose a new Bayesian model for flexible nonlinear regression and cl...
research
09/16/2017

Relevant Ensemble of Trees

Tree ensembles are flexible predictive models that can capture relevant ...

Please sign up or login with your details

Forgot password? Click here to reset