Solving the Black Box Problem: A General-Purpose Recipe for Explainable Artificial Intelligence
share
Many of the computing systems developed using machine learning are opaque: it is difficult to explain why they do what they do, or how they work. The Explainable AI research program aims to develop analytic techniques for rendering such systems transparent, but lacks a general understanding of what it actually takes to do so. The aim of this discussion is to provide a general-purpose recipe for Explainable AI: A series of steps that should be taken to render an opaque computing system transparent. After analyzing the dual notions of 'opacity' and 'transparency', this recipe invokes David Marr's influential levels of analysis framework to characterize the different questions that should be asked about an opaque computing system, as well as the different ways in which these questions should be answered by different agents. By applying this recipe to recent techniques such as input heatmapping, feature-detector identification, and diagnostic classification, it will be possible to determine the extent to which Explainable AI can already solve the so-called Black Box Problem, as well as the extent to which more sophisticated techniques will be needed.
READ FULL TEXT