Approximation Trees: Statistical Stability in Model Distillation

08/22/2018
by   Yichen Zhou, et al.
2

This paper examines the stability of learned explanations for black-box predictions via model distillation with decision trees. One approach to intelligibility in machine learning is to use an understandable `student' model to mimic the output of an accurate `teacher'. Here, we consider the use of regression trees as a student model, in which nodes of the tree can be used as `explanations' for particular predictions, and the whole structure of the tree can be used as a global representation of the resulting function. However, individual trees are sensitive to the particular data sets used to train them, and an interpretation of a student model may be suspect if small changes in the training data have a large effect on it. In this context, access to outcomes from a teacher helps to stabilize the greedy splitting strategy by generating a much larger corpus of training examples than was originally available. We develop tests to ensure that enough examples are generated at each split so that the same splitting rule would be chosen with high probability were the tree to be re trained. Further, we develop a stopping rule to indicate how deep the tree should be built based on recent results on the variability of Random Forests when these are used as the teacher. We provide concrete examples of these procedures on the CAD-MDD and COMPAS data sets.

READ FULL TEXT

page 12

page 27

page 28

research
11/22/2022

A Generic Approach for Reproducible Model Distillation

Model distillation has been a popular method for producing interpretable...
research
10/30/2019

Distilling Black-Box Travel Mode Choice Model for Behavioral Interpretation

Machine learning has proved to be very successful for making predictions...
research
01/26/2018

Transparent Model Distillation

Model distillation was originally designed to distill knowledge from a l...
research
06/15/2023

Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Theory of Mind

Large Language Models (LLMs) perform complex reasoning by generating exp...
research
08/21/2020

Rectified Decision Trees: Exploring the Landscape of Interpretable and Effective Machine Learning

Interpretability and effectiveness are two essential and indispensable r...
research
02/13/2021

Distilling Double Descent

Distillation is the technique of training a "student" model based on exa...
research
06/11/2020

How Interpretable and Trustworthy are GAMs?

Generalized additive models (GAMs) have become a leading model class for...

Please sign up or login with your details

Forgot password? Click here to reset