
An Evaluation of the HumanInterpretability of Explanation
Recent years have seen a boom in interest in machine learning systems that can provide a humanunderstandable rationale for their predictions or decisions. However, exactly what kinds of explanation are truly humaninterpretable remains poorly understood. This work advances our understanding of what makes explanations interpretable under three specific tasks that users may perform with machine learning systems: simulation of the response, verification of a suggested response, and determining whether the correctness of a suggested response changes under a change to the inputs. Through carefully controlled humansubject experiments, we identify regularizers that can be used to optimize for the interpretability of machine learning systems. Our results show that the type of complexity matters: cognitive chunks (newly defined concepts) affect performance more than variable repetitions, and these trends are consistent across tasks and domains. This suggests that there may exist some common design principles for explanation systems.
01/31/2019 ∙ by Isaac Lage, et al. ∙ 16 ∙ shareread it

Quality of Uncertainty Quantification for Bayesian Neural Network Inference
Bayesian Neural Networks (BNNs) place priors over the parameters in a neural network. Inference in BNNs, however, is difficult; all inference methods for BNNs are approximate. In this work, we empirically compare the quality of predictive uncertainty estimates for 10 common inference methods on both regression and classification tasks. Our experiments demonstrate that commonly used metrics (e.g. test loglikelihood) can be misleading. Our experiments also indicate that inference innovations designed to capture structure in the posterior do not necessarily produce high quality posterior approximations.
06/24/2019 ∙ by Jiayu Yao, et al. ∙ 8 ∙ shareread it

Regional Tree Regularization for Interpretability in Black Box Models
The lack of interpretability remains a barrier to the adoption of deep neural networks. Recently, tree regularization has been proposed to encourage deep neural networks to resemble compact, axisaligned decision trees without significant compromises in accuracy. However, it may be unreasonable to expect that a single tree can predict well across all possible inputs. In this work, we propose regional tree regularization, which encourages a deep model to be wellapproximated by several separate decision trees specific to predefined regions of the input space. Practitioners can define regions based on domain knowledge of contexts where different decisionmaking logic is needed. Across many datasets, our approach delivers more accurate predictions than simply training separate decision trees for each region, while producing simpler explanations than other neural net regularization schemes without sacrificing predictive power. Two healthcare case studies in critical care and HIV demonstrate how experts can improve understanding of deep models via our approach.
08/13/2019 ∙ by Mike Wu, et al. ∙ 5 ∙ shareread it

Evaluating Reinforcement Learning Algorithms in Observational Health Settings
Much attention has been devoted recently to the development of machine learning algorithms with the goal of improving treatment policies in healthcare. Reinforcement learning (RL) is a subfield within machine learning that is concerned with learning how to make sequences of decisions so as to optimize longterm effects. Already, RL algorithms have been proposed to identify decisionmaking strategies for mechanical ventilation, sepsis management and treatment of schizophrenia. However, before implementing treatment policies learned by blackbox algorithms in highstakes clinical decision problems, special care must be taken in the evaluation of these policies. In this document, our goal is to expose some of the subtleties associated with evaluating RL algorithms in healthcare. We aim to provide a conceptual starting point for clinical and computational researchers to ask the right questions when designing and evaluating algorithms for new ways of treating patients. In the following, we describe how choices about how to summarize a history, variance of statistical estimators, and confounders in more adhoc measures can result in unreliable, even misleading estimates of the quality of a treatment policy. We also provide suggestions for mitigating these effectsfor while there is much promise for mining observational health data to uncover better treatment policies, evaluation must be performed thoughtfully.
05/31/2018 ∙ by Omer Gottesman, et al. ∙ 4 ∙ shareread it

Defining Admissible Rewards for High Confidence Policy Evaluation
A key impediment to reinforcement learning (RL) in real applications with limited, batch data is defining a reward function that reflects what we implicitly know about reasonable behaviour for a task and allows for robust offpolicy evaluation. In this work, we develop a method to identify an admissible set of reward functions for policies that (a) do not diverge too far from past behaviour, and (b) can be evaluated with high confidence, given only a collection of past trajectories. Together, these ensure that we propose policies that we trust to be implemented in highrisk settings. We demonstrate our approach to reward design on synthetic domains as well as in a critical care context, for a reward that consolidates clinical objectives to learn a policy for weaning patients from mechanical ventilation.
05/30/2019 ∙ by Niranjani Prasad, et al. ∙ 1 ∙ shareread it

Beyond Sparsity: Tree Regularization of Deep Models for Interpretability
The lack of interpretability remains a key barrier to the adoption of deep models in many applications. In this work, we explicitly regularize deep models so human users might step through the process behind their predictions in little time. Specifically, we train deep timeseries models so their classprobability predictions have high accuracy while being closely modeled by decision trees with few nodes. Using intuitive toy examples as well as medical tasks for treating sepsis and HIV, we demonstrate that this new tree regularization yields models that are easier for humans to simulate than simpler L1 or L2 penalties without sacrificing predictive power.
11/16/2017 ∙ by Mike Wu, et al. ∙ 0 ∙ shareread it

Accountability of AI Under the Law: The Role of Explanation
The ubiquity of systems using artificial intelligence or "AI" has brought increasing attention to how those systems should be regulated. The choice of how to regulate AI systems will require care. AI systems have the potential to synthesize large amounts of data, allowing for greater levels of personalization and precision than ever beforeapplications range from clinical decision support to autonomous driving and predictive policing. That said, there exist legitimate concerns about the intentional and unintentional negative consequences of AI systems. There are many ways to hold AI systems accountable. In this work, we focus on one: explanation. Questions about a legal right to explanation from AI systems was recently debated in the EU General Data Protection Regulation, and thus thinking carefully about when and how explanation from AI systems might improve accountability is timely. In this work, we review contexts in which explanation is currently required under the law, and then list the technical considerations that must be considered if we desired AI systems that could provide kinds of explanations that are currently required of humans.
11/03/2017 ∙ by Finale DoshiVelez, et al. ∙ 0 ∙ shareread it

Decomposition of Uncertainty for Active Learning and Reliable Reinforcement Learning in Stochastic Systems
Bayesian neural networks (BNNs) with latent variables are probabilistic models which can automatically identify complex stochastic patterns in the data. We study in these models a decomposition of predictive uncertainty into its epistemic and aleatoric components. We show how such a decomposition arises naturally in a Bayesian active learning scenario and develop a new objective for reliable reinforcement learning (RL) with an epistemic and aleatoric risk element. Our experiments illustrate the usefulness of the resulting decomposition in active learning and reliable RL.
10/19/2017 ∙ by Stefan Depeweg, et al. ∙ 0 ∙ shareread it

Weighted Tensor Decomposition for Learning Latent Variables with Partial Data
Tensor decomposition methods are popular tools for learning latent variables given only lowerorder moments of the data. However, the standard assumption is that we have sufficient data to estimate these moments to high accuracy. In this work, we consider the case in which certain dimensions of the data are not always observedcommon in applied settings, where not all measurements may be taken for all observationsresulting in moment estimates of varying quality. We derive a weighted tensor decomposition approach that is computationally as efficient as the nonweighted approach, and demonstrate that it outperforms methods that do not appropriately leverage these lessobserved dimensions.
10/18/2017 ∙ by Omer Gottesman, et al. ∙ 0 ∙ shareread it

Rollback Hamiltonian Monte Carlo
We propose a new framework for Hamiltonian Monte Carlo (HMC) on truncated probability distributions with smooth underlying density functions. Traditional HMC requires computing the gradient of potential function associated with the target distribution, and therefore does not perform its full power on truncated distributions due to lack of continuity and differentiability. In our framework, we introduce a sharp sigmoid factor in the density function to approximate the probability drop at the truncation boundary. The target potential function is approximated by a new potential which smoothly extends to the entire sample space. HMC is then performed on the approximate potential. While our method is easy to implement and applies to a wide range of problems, it also achieves comparable computational efficiency on various sampling tasks compared to other baseline methods. RBHMC also gives rise to a new approach for Bayesian inference on constrained spaces.
09/08/2017 ∙ by Kexin Yi, et al. ∙ 0 ∙ shareread it

PredictionConstrained Training for SemiSupervised Mixture and Topic Models
Supervisory signals have the potential to make lowdimensional data representations, like those learned by mixture and topic models, more interpretable and useful. We propose a framework for training latent variable models that explicitly balances two goals: recovery of faithful generative explanations of highdimensional data, and accurate prediction of associated semantic labels. Existing approaches fail to achieve these goals due to an incomplete treatment of a fundamental asymmetry: the intended application is always predicting labels from data, not data from labels. Our predictionconstrained objective for training generative models coherently integrates lossbased supervisory signals while enabling effective semisupervised learning from partially labeled data. We derive learning algorithms for semisupervised mixture and topic models using stochastic gradient descent with automatic differentiation. We demonstrate improved prediction quality compared to several previous supervised topic models, achieving predictions competitive with highdimensional logistic regression on text sentiment analysis and electronic health records tasks while simultaneously learning interpretable topics.
07/23/2017 ∙ by Michael C. Hughes, et al. ∙ 0 ∙ shareread it
Finale DoshiVelez
is this you? claim profile
Assistant Professor of Computer Science at Harvard Paulson School of Engineering and Applied Sciences, Harvard University, MSc from the University of Cambridge as a Marshall Scholar, her PhD from MIT, and her postdoc at Harvard Medical School.