DeepAI AI Chat
Log In Sign Up

The Intriguing Properties of Model Explanations

by   Maruan Al-Shedivat, et al.
Carnegie Mellon University

Linear approximations to the decision boundary of a complex model have become one of the most popular tools for interpreting predictions. In this paper, we study such linear explanations produced either post-hoc by a few recent methods or generated along with predictions with contextual explanation networks (CENs). We focus on two questions: (i) whether linear explanations are always consistent or can be misleading, and (ii) when integrated into the prediction process, whether and how explanations affect the performance of the model. Our analysis sheds more light on certain properties of explanations produced by different methods and suggests that learning models that explain and predict jointly is often advantageous.


page 1

page 2

page 3

page 4


Contextual Explanation Networks

We introduce contextual explanation networks (CENs)---a class of models ...

What does LIME really see in images?

The performance of modern algorithms on certain computer vision tasks su...

Bandits for Learning to Explain from Explanations

We introduce Explearn, an online algorithm that learns to jointly output...

Faithfully Explaining Rankings in a News Recommender System

There is an increasing demand for algorithms to explain their outcomes. ...

The Unreliability of Explanations in Few-Shot In-Context Learning

How can prompting a large language model like GPT-3 with explanations im...

Accurate and Intuitive Contextual Explanations using Linear Model Trees

With the ever-increasing use of complex machine learning models in criti...

Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations

In attempts to "explain" predictions of machine learning models, researc...