DeepAI
Log In Sign Up

Comprehensible Counterfactual Interpretation on Kolmogorov-Smirnov Test

The Kolmogorov-Smirnov (KS) test is popularly used in many applications, such as anomaly detection, astronomy, database security and AI systems. One challenge remained untouched is how we can obtain an interpretation on why a test set fails the KS test. In this paper, we tackle the problem of producing counterfactual interpretations for test data failing the KS test. Concept-wise, we propose the notion of most comprehensible counterfactual interpretations, which accommodates both the KS test data and the user domain knowledge in producing interpretations. Computation-wise, we develop an efficient algorithm MOCHI that avoids enumerating and checking an exponential number of subsets of the test set failing the KS test. MOCHI not only guarantees to produce the most comprehensible counterfactual interpretations, but also is orders of magnitudes faster than the baselines. Experiment-wise, we present a systematic empirical study on a series of benchmark real datasets to verify the effectiveness, efficiency and scalability of most comprehensible counterfactual interpretations and MOCHI.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/17/2020

Impact of Accuracy on Model Interpretations

Model interpretations are often used in practice to extract real world i...
06/16/2021

Model-Based Counterfactual Synthesizer for Interpretation

Counterfactuals, serving as one of the emerging type of model interpreta...
03/21/2022

Diverse Counterfactual Explanations for Anomaly Detection in Time Series

Data-driven methods that detect anomalies in times series data are ubiqu...
12/22/2021

CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models

We introduce the CRASS (counterfactual reasoning assessment) data set an...
06/30/2022

Tuple Interpretations and Applications to Higher-Order Runtime Complexity

Tuple interpretations are a class of algebraic interpretation that subsu...
09/10/2021

Counterfactual Adversarial Learning with Representation Interpolation

Deep learning models exhibit a preference for statistical fitting over l...
05/29/2019

Model Similarity Mitigates Test Set Overuse

Excessive reuse of test data has become commonplace in today's machine l...