The Mythos of Model Interpretability

06/10/2016
by   Zachary C Lipton, et al.
0

Supervised machine learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world? We want models to be not only good, but interpretable. And yet the task of interpretation appears underspecified. Papers provide diverse and sometimes non-overlapping motivations for interpretability, and offer myriad notions of what attributes render models interpretable. Despite this ambiguity, many papers proclaim interpretability axiomatically, absent further explanation. In this paper, we seek to refine the discourse on interpretability. First, we examine the motivations underlying interest in interpretability, finding them to be diverse and occasionally discordant. Then, we address model properties and techniques thought to confer interpretability, identifying transparency to humans and post-hoc explanations as competing notions. Throughout, we discuss the feasibility and desirability of different notions, and question the oft-made assertions that linear models are interpretable and that deep neural networks are not.

READ FULL TEXT
04/08/2019

Quantifying Interpretability of Arbitrary Machine Learning Models Through Functional Decomposition

To obtain interpretable machine learning models, either interpretable mo...
10/23/2020

Model Interpretability through the Lens of Computational Complexity

In spite of several claims stating that some models are more interpretab...
04/30/2021

Interpretability of Epidemiological Models : The Curse of Non-Identifiability

Interpretability of epidemiological models is a key consideration, espec...
05/29/2018

Human-in-the-Loop Interpretability Prior

We often desire our models to be interpretable as well as accurate. Prio...
11/22/2020

A Bayesian Account of Measures of Interpretability in Human-AI Interaction

Existing approaches for the design of interpretable agent behavior consi...
07/08/2019

Optimal Explanations of Linear Models

When predictive models are used to support complex and important decisio...
04/30/2020

Hide-and-Seek: A Template for Explainable AI

Lack of transparency has been the Achilles heal of Neural Networks and t...