Post-discovery Analysis of Anomalous Subsets

by   Isaiah Onando Mulang', et al.

Analyzing the behaviour of a population in response to disease and interventions is critical to unearth variability in healthcare as well as understand sub-populations that require specialized attention, but also to assist in designing future interventions. Two aspects become very essential in such analysis namely: i) Discovery of differentiating patterns exhibited by sub-populations, and ii) Characterization of the identified subpopulations. For the discovery phase, an array of approaches in the anomalous pattern detection literature have been employed to reveal differentiating patterns, especially to identify anomalous subgroups. However, these techniques are limited to describing the anomalous subgroups and offer little in form of insightful characterization, thereby limiting interpretability and understanding of these data-driven techniques in clinical practices. In this work, we propose an analysis of differentiated output (rather than discovery) and quantify anomalousness similarly to the counter-factual setting. To this end we design an approach to perform post-discovery analysis of anomalous subsets, in which we initially identify the most important features on the anomalousness of the subsets, then by perturbation, the approach seeks to identify the least number of changes necessary to lose anomalousness. Our approach is presented and the evaluation results on the 2019 MarketScan Commercial Claims and Medicare data, show that extra insights can be obtained by extrapolated examination of the identified subgroups.



page 1

page 2

page 3

page 4


Sparsity-based Feature Selection for Anomalous Subgroup Discovery

Anomalous pattern detection aims to identify instances where deviation f...

Discovery of Complex Anomalous Patterns of Sexual Violence in El Salvador

When sexual violence is a product of organized crime or social imaginary...

Preservation of Anomalous Subgroups On Machine Learning Transformed Data

In this paper, we investigate the effect of machine learning based anony...

ATD: Anomalous Topic Discovery in High Dimensional Discrete Data

We propose an algorithm for detecting patterns exhibited by anomalous cl...

Outlier Detection from Network Data with Subnetwork Interpretation

Detecting a small number of outliers from a set of data observations is ...

Semantic-based Anomalous Pattern Discovery in Moving Object Trajectories

In this work, we investigate a novel semantic approach for pattern disco...

An Alternative Metric for Detecting Anomalous Ship Behavior Using a Variation of the DBSCAN Clustering Algorithm

There is a growing need to quickly and accurately identify anomalous beh...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Healthcare is known for its variations in the population, treatment, and outcome domains  Appleby et al. (2011); Krumholz (2013). Examples of such variations include diversity in the population (e.g., a particular subgroup being vulnerable or at high risk), heterogeneous effects of a treatment regime across a given population, or different interventions across population segments in response to a specific disease.

To improve clinical practice and the quality of interventions, there is a need to trace and reconstruct such variations. However, large-scale observational health data and longitudinal clinical records are characterized by complex interactions of patient characteristics and interventions, making challenging to analyze and interpret non-random variations of patient populations, care delivery, and outcomes. Manual approaches heavily rely on domain expertise that requires pre-assumption of features responsible for variations and lack scalability for large and complex interactions expected in clinical data. In addition, manual approaches cannot readily unearth complex, non-obvious variations in healthcare.

Fortunately, researchers have sought to develop automatic and data-driven approaches for the scalable discovery of differentiated patterns of care. Particularly, machine learning techniques in the anomalous pattern detection literature could be employed to encode variations in healthcare. Subset scanning  

Neill (2012); Neill et al. (2013) is one of these approaches that aim to identify anomalous subgroups in a population with divergent characteristics compared to the expected baseline in an efficient manner. In subset scanning, anomalousness is quantified by a scoring function, typically a log-likelihood ratio statistic, that is maximized over the subsets in a multidimensional array to identify the subset with the highest score  Neill (2012). This scoring function must satisfy the Linear-Time Subset Scanning (LTSS) so that the search is efficiently conducted in linear time Neill (2012).

The typical input to a subset scanning algorithm is a set of discrete/discretized features e.g. Gender: Male/Female, Race: Black/Brown/White, Smoking: Yes/No, and Weight: Low/Mid/High; and target variable of interest e.g. Mortality. The typical output of subset scanning consists of 3 main items: 1) the highest scoring (most anomalous) subset characterized by specific feature values (e.g. Gender: Female, Race: Black/White, Smoking: Yes, and Weight: High), (2) the score of the highest-scoring subset, and (3) the empirical p-value evaluating the significance of the identified subset. This output is thus merely descriptive and there is still a significant concern related to the characterization of the findings into practices, particularly in clinical care. The characterization includes understanding the dynamics associated with anomalousness of a particular sub-population in response to variation in patient features. In other words, there is an implicit next step necessary to assist better utilization of these methods and inform actions.

In this paper, we propose the post-discovery analysis of anomalous subsets aimed at enhancing the clinical usefulness of subset scanning results and triggering a paradigm shift from description to actionable insights. To this end, we tackle two major research questions as our contributions:

  1. Which of the features (and their values) describing the identified subset have the greatest impact on the anomalous score?

  2. What is the smallest change in feature values that causes a loss in anomalousness?

This paper describes our initial step towards a larger agenda of post-discovery analysis of anomalous subsets. Our proposed solution involves a two-step approach that first identifies the features and feature values that contribute most significantly to the anomalous behavior of the identified subset, then performs cross-substitution of these feature values to attain a significant drop in anomalousness. The rest of this paper is structured as follows: Section 2 offers a summary of related work followed by the details of the proposed method in Section 3. The data and experiments are described in Section 4, finally, Section 5 concludes and offers future directions.

2 Related Work

Although our work is an initial step towards post-discovery analysis, being the first proposal in this direction, it lies in the intersection between two fields namely: Subgroup analysis and counterfactual explanations. We view our work as a special form of counterfactual analysis that operates on unsupervised scenarios. Hereafter, we discuss existing work in these two areas in literature.

2.1 Subset Scanning

Subset Scanning algorithms Neill (2012); Somanchi and Neill (2017); McFowland et al. (2013) identify the subset of records with the most significant difference in the distribution of the observed outcome compared to the expected outcome and have been applied across a variety of use cases. Neil et al. applied subset scanning in disease surveillance to detect geolocations and data streams suggestive of emerging outbreaks  Neill (2012); Neill et al. (2013)

. Zheng et al. developed a subset scanning approach for detecting predictive bias in binary classifiers  

Zhang and Neill (2016). McFowland et al. proposed a subset scanning algorithm for detecting heterogeneous treatment effects in clinical trial data  McFowland III et al. (2018). Somanchi et al. developed the anomalous patterns of care scan that combines propensity score modeling with subset scanning to enable anomalous pattern detection in anomalous pattern detection in observational health data Somanchi and Neill (2017). Each of these methods optimizes a scoring function, such as a log-likelihood ratio statistic, over all subsets in a dataset and selects the highest scoring subset. The scoring function is constrained to a mathematical property referred to as Linear-Time Subset Scanning (LTSS) that enables the search to be conducted in linear as opposed to exponential time  Neill (2012).

2.2 Counterfactual Analysis

Traditionally, counterfactual explanations have been employed to help understand the limitations of machine learning methods. To this end, methods for counterfactual explanations try to find an example that is as close as possible to the data points in the training set of a model. Counterfactual explanation methods can either be model-specific or model-agnostic. Model-specific methods utilize the internal training characteristic (e.g., the hyperparameters) of the underlying machine learning model  

Looveren and Klaise (2019); Mothilal et al. (2020); Russell (2019). Model-agnostic methods separate the explanations from the machine learning model. They encompass functionality that apply to any arbitrary modelsDhurandhar et al. (2019); Karimi et al. (2020); Sharma et al. (2020); White and d’Avila Garcez (2020), generally by only accessing the prediction function of a fitted model  Dandl et al. (2020). Schölkopf et. al.  Schölkopf et al. (2021) discuss causal representation learning as an avenue for counterfactual explanation. These methods operate on supervised machine learning models and attempt to find examples close to one of the data points.

In our study, the discovery methods that identify the subgroups (e.g. anomalous subsets) are inherently model-agnostic search algorithms. We work with a similar research question: What features describing the individual would need to change to achieve the desired output?  Karimi et al. (2020) However, we aim to identify the exact changes that can be made to the characteristics of a subgroup of individual to achieve a desired output (in our case, a significant change in the anomalous score). Our objective is therefore slightly different from traditional counterfactual explanation, in two ways: i) Given a subgroup that has been identified as anomalous (e.g, having a significantly higher rate of an outcome than a global average), we seek to find the least number of perturbations to the subgroup’s features and feature values of that would lead to loss of anomalousness; and ii) As opposed to finding the counterfactual example, we find the feature values from the complement subset.

3 Methods

Figure 1 illustrates our proposed pipeline for the Post-Discovery Analysis of Anomalous Subsets. The approach takes in as input, an anomalous subset together with the original dataset and aims to identify feature substitutions that result in loss of anomalousness. It consists of two major subtasks namely: i) Feature relevance ranking: takes an anomalous subset and identifies the most significant features and feature values contributing to anomalousness ; ii) Cross-substitution: Identifies the possible permutations of feature values within the anomalous subsets by scoring minimally altered subsets obtained through replacing values of the anomalous features with alternatives from the nonanomalous set . In this section, we describe these steps in detail.

Figure 1: Pipeline for Post-Discovery Analysis of Anomalous Subsets

3.1 Problem formulation

The post-discovery analysis task assumes that there is an output from prior discovery search through a given dataset that produced a descriptive anomalous subset, and aims to identify the extent to which the feature values can be perturbed to achieve a loss of anomalousness. Formally, if we let denote a dataset of length , where each data-point is characterized by a set of features such that the whole feature space set in is given by and represents the outcome label, there exists, by definition, a set s.t. and , called the anomalous subset, and characterized by the anomalous feature values , and the complete anomalous features description is , where represents the value of the feature in that contributes to the anomalousness of , e.g., Smoking and Yes. The tuple is the output of subset scanning function , where is a scoring function that obtains the anomalous score , and

is a set containing the measures of effect such as the odds ratio and the p-value. The post-discovery analysis is denoted as

where is a cross-substitution function, that performs substitutions of the form , where and . The function

is a feature selection routine, that identifies a set of features that most contribute to anomalousness. The output

is a minimalistic set of feature values that substitute in the anomalous subset to cause a loss in anomalousness.

3.2 Multi-Dimensional Subset Scanning(MDSS)

To corroborate our post-discovery analysis approach to insightful analysis of anomalous subgroups, we apply the Multi-Dimensional Subset Scanning(MDSS) Neill and Kumar (2013) from the anomalous pattern detection literature. Specifically, we use Automatic stratification (AutoStrat), a version of MDSS where the deviation between average outcomes in () and each sample () is evaluated by maximizing a Bernoulli likelihood ratio scoring statistic,

. The null hypothesis assumes that the likelihood of the outcome in each sample

similar to expected (), i.e., ; while the alternative hypothesis assumes a constant multiplicative increase in the outcome odds for some given subgroup, where . The scoring function, for a subset, , which contains samples, is computed as:

Consequently, subsets in which the average of () is greater than will have higher scores. Subset selection is iterated until convergence to a local maximum is found, and the global maximum is subsequently optimized using multiple random restarts. Once the differentiated subset of samples is identified using MDSS, empirical p-value (via randomization testing) is computed to evaluate the significance of the differentiation. Moreover, subset characterization is conducted to provide interpretation of these anomalous features and subset .

3.3 Quantifying Feature Relevance to Anomalousness of a Subset

The anomalous subsets produced by the subset scanning approaches described above are characterized by features and feature values contributing to the anomalousness. The intuition behind feature selection is that not all feature values contribute equally to the anomalousness of a subset, measured by the anomalous score. The feature selection, therefore, strives to identify the set of features and feature values in the anomalous subset that induce the greatest impact on the anomalous score. Such features are prime candidates for analysis as a change in their values has a higher likelihood of resulting in a significant drop in the anomalous score. Taking the Output tuple from a discovery method, and the dataset . The feature selection routine proceeds by first obtaining the marginal expected values of the results conditioned on each feature value. Therefore, for each anomalous feature value, , we calculate its expected value as . We then calculate the deviations of from the mean expected value of the dataset , the deviation . In the next step, we rank the feature values based on their calculated deviations. if . A threshold value () is used to select a set of most deviating feature values, to form the selected features .

Input: ; anomalous subset; dataset; selected feature set
Output: ()
while  and  do
        Q.dequeue for each  do
               for each  do
                      // score from a Bernoulli likelihood scoring function
return ()
Algorithm 1 Cross-Substitution

3.4 Cross Substitution

After identifying the features and feature values that are most relevant to the anomalousness of a subset, we then perform the cross substitution process. The aim is to find minimal changes in the anomalous subset that results in a significant drop in the anomalous score. This is achieved by drawing values from the compliment set to substitute feature values in the anomalous subset. Assume, for example, an anomalous subset described in Table2 and its complement described in Table2.

Feature Value(s) E.g. E.g.
Feature Values
Gender Female
Race Black, White
Smoking Yes
Weight High
Table 2: Complement Subset
Value(s) E.g. E.g.
Feature Values
Gender Male
Race Brown
Smoking No
, Weight Low, Mid
Table 1: Anomalous Subset

The possible substitutions include: {, , , , , , }. A single substitution results in a new subset that is different from the anomalous subset. These perturbations are recursively carried out until we exhaust the space of possible substitutions. If this space is vastly complex, the results of the feature selection phase become invariably important.The cross-substitution therefore begins alteration in order of the feature ranking obtained from the feature selection. Each of the generated subsets are then scored with same scoring function used by the discovery method (e.g. Bernoulli likelihood ratio scoring statistic). This scoring aims to generate new measures of effect . Statistical significance test is used to determine whether the new subset loses the anomalousness (typically a p-value statistic or odds ratio is used, obtained i similar procedure as described in section 2). The stopping criterion is attained when the statistical significance of a subset surpasses a given threshold. Algorithm 1 elaborates the cross-substitution analysis.

4 Experiments and Results

To validate our proposed methodology, we conducted a study whose objectives were twofold. First, we aimed to identify the subpopulation of osteoarthritis of the knee (OA Knee) with the most evidence of having higher-than-expected rates of major joint replacement (MJR) surgeries. Second, we aimed to identify the smallest perturbations in the characteristics of the identified anomalous subset that resulted in a significant loss of anomalousness.

4.1 Dataset and Experimental Setup

We analyzed the 2019 MarketScan Commercial Claims and Medicare data in the IBM MarketScan Research Databases ( The database is fully deidentified, hence this study did not require Institutional Review Board approval. We defined our target cohort as newly diagnosed OA Knee patients receiving outpatient services in 2019; and our outcome cohort as outpatient OA Knee patients who underwent any MJR surgery in 2019. The final analytic dataset consisted of OA Knee patients, of whom () had MJR surgeries.

We extracted 5 discrete features with different cardinalities as follows: age group (18 to 34, 35 to 44, 45 to 54, 55 to 64, 65 and Older), gender (Female, Male), region (North Central, Northeast, South, Unknown, West), metropolitan statistical area (Rural, Urban), and employment status (Active Full Time, Active Part-Time or Seasonal, COBRA Continuee, Early Retiree, Long Term Disability, Medicare Eligible Retiree, Other/Unknown, Retiree (Status Unknown), Surviving Spouse/Dependendant). We defined the observed outcome as a binary indicator variable such that for OA Knee patients who underwent an MJR surgery, and otherwise. We defined the expected outcome as a simple mean of the observed outcome. Accordingly, our final analysis dataset consisted of 5 features, 1 observed outcome, and one expected outcome.

4.2 Identification and characteristics of the highest scoring (most anomalous) subset

We used our automatic stratification approach to identify the highest-scoring subset with the most evidence of having higher rates of MJR surgeries than the global average in our dataset. To correct for multiple hypothesis testing and estimate the statistical significance of the identified anomalous subset, we used parametric bootstrapping to compute the empirical p-value of the subset

McFowland et al. (2013).

Using this approach, we discovered that OA Knee patients who are 55 to 64 years old, reside in the West, North Central or South regions of the United States, and have an employment status as Full-time, Medicare retiree, Early Retiree, COBRA Continue, or Long Term Disability, were most likely to undergo MJR surgeries. Among this subpopulation consisting of OA Knee patients, the rate of MJR surgeries was significantly higher ( in the subpopulation compared to in the complement subpopulation; odds ratio 2.09, confidence interval, 2.02 to 2.17, p-value <0.001).

4.3 Post-discovery characteristics of the highest scoring (most anomalous) subset

To perform post-discovery analysis, we first determined the relevance of features to the anomalous score. For the MJR dataset, the expected value of the output in was . The expected value of the anomalous subset was , denoting that all records with the combination of features in the anomalous subset underwent MJR surgery. We proceeded by calculating the expected output for each feature-value in the anomalous subset. For example in the anomalous subset, one of the feature values for the feature Employment Status (EESTATU) was "Medicare Retiree". We found that the expected value of the this feature value in the dataset is .

Consequently, we calculate two deviation statistics: i) A subset deviation: deviation of the feature value from the anomalous subset . ii) Global deviation: deviation of the feature value from the expected value of the dataset . The deviation ratio of the two deviations is then used to score and rank each feature value against the other feature values in the anomalous subset.

Table 3 show the results of the feature relevance and the cross-substitution steps respectively. In the feature relevance stage, we observe that the feature values with a negative deviation ratio are more relevant than a feature with positive deviation ratios. Overall the feature values with larger negative ratios are less important than those with lower negative ratio scores. For our MJR use case, the ranking of feature relevance is therefore as follows: 1. Employment Medicare Retiree, 2. Early Retiree, 2. Early Retiree.

Feature Value D.R
EESTATU Medical Retiree -52.21
EESTATU Early Retiree -62.27
EESTATU L.T Disability -78.06
EESTATU COBRA Continue -91.79
REGION West -91.96
AGEGRP 55-64 -92.20
REGION N. Cent -92.95
EESTATU Full Time 15293.85
REGION South 798.75
Table 3: Feature Relevance Ranking

In the Cross-Substitution stage, we perturbed the anomalous subset by changing feature values. Of note is that when we cross-substitute a feature value from the complement for an anomalous feature value, we effectively create a new subset. Consequently, we calculated a new anomalous score , the empirical p-value of the score, and the odds ratio of the outcome in the new subset compared to its complement. For example, in the MJR dataset, the anomalous subset exhibited a score of , empirical p-value , and an odds ratio of . By substituting the employment status value "Medicare Retiree" in the anomalous subset with "Other Unknown" in the complement subset, we obtain new subset with anomalous score of , empirical p-value of , and an odds ratio of . These statistics are obtained using the same scoring method used in the discovery stage to allow for consistency in comparison. Table 4 shows the post-discovery scores for single substitutions of the two most significant anomalous feature values. Although the p-values stay constant as seen in figure 2, we observe a considerable degradation of the anomalous score and the Odds Ratio.

Feature Value Substitute O_Score N_Score O_OR N_OR
Medicare Retiree Other/Unknown 500.22 465.92 2.09 2.20
Medicare Retiree Part Time 500.22 477.93 2.09 2.05
Medicare Retiree Retiree (Status Unk) 500.22 482.87 2.09 2.05
Medicare Retiree Spouse/Dependent 500.22 483.34 2.09 2.06
Early Retiree Other/Unknown 500.22 352.94 2.09 1.86
Early Retiree Part Time 500.22 360.09 2.09 1.81
Early Retiree Retiree (Status Unk) 500.22 364.57 2.09 1.82
Early Retiree Spouse/Dependent 500.22 365.37 2.09 1.82
Table 4: Observing Changes in Anomalous Scores for Substituting two Feature Values

Figure 2 portrays the effects of making substitutions to single feature values. Two measures of effect are plotted namely: the statistical p-value of the anomalous score, and the odds ratio. For the single substitutions, we observe that the p-value statistic remains relatively the same for most feature value perturbations. However there is a few single substitutions that cause a statistically significant change in the anomalous score namely, the substitutions on the feature: AGEGRP: 55-64 -> 18-34, 55-64 -> 35-44, 55-64 -> 45-54, 55-64 -> . From this result we can conclude thus: "if the sub-population that belong to the anomalous subgroup had been changed form the specific age group 55-64 to the ages -> 18-34, 45-54, , there would not have been a major joint replacement surgery. Note that single value perturbations to the most relevant feature: Medicare Retiree did not cause any statistically significant change to the anomalous score. This can be attributed to the fact that the feature employment status (EESTATU) is multi-valued in the anomalous subset, a single change in one of the feature values has little effect especially since there are 3 more feature values considered. We expect that higher combination substitutions will reveal better insights into this phenomenon. The odds ratio, on the other hand, indicates a variation of the score from the substitutions. For example the odds ratio for 55-64 -> is relatively higher than 55-64 -> 18-34 meaning that the former is more likely to occur in this dataset as opposed to the latter. Similarly, the substitution: Medical Retiree -> Other Unknown, has a very high odds ratio, attesting to the possibility of such a change in this dataset, albeit with no change in the statistical significant measure (Already explained).

Figure 2: Plotting Cross-Substitution of Single Feature Values in the Anomalous Subset against the two Measures of Effect : P-Value and Odds Ratio

5 Conclusion

In this paper, we discuss a new view to interpreting population responses to disease interventions, that has the potential to improve clinical practice, As well as assist in understanding the output of subgroup analysis methods like the Autostrat. The post-discovery analysis seeks to unearth insights presented in the otherwise descriptive discovery methods outputs, by performing sufficient perturbations to the feature values. The aim thereof is to obtain the smallest changes that can lead to a loss of anomalousness. As avenues for extension, we view our method for determining feature relevance that uses expected value weighting as a pointer characterization technique that can be extended by investigating other statistical measures related to the features in the identified subset. In addition, our current experiments on cross-substitution perform only single substitutions of the feature values. This leaves an avenue for improvement, by considering the space of all possible combinations of feature values and substitutions. Such an approach would require consideration of the cost and complexity of cross-substitution. Akin to the work in the Subset Scanning literature, we envisage the need for an approach that reduces the search in this space to Linear Time. This work lays a foundation for the discussion concerning the design of interventions, such post-discovery analysis has great potential to inspire the better design of interventions beyond clinical Care and Public Health.


  • [1] J. Appleby, V. Raleigh, F. Frosini, G. Bevan, H. Gao, and T. Lyscom (2011) Variations in health care. The good, the bad and the inexplicable. London: The King’s Fund. Cited by: §1.
  • [2] S. Dandl, C. Molnar, M. Binder, and B. Bischl (2020) Multi-objective counterfactual explanations. In International Conference on Parallel Problem Solving from Nature, pp. 448–469. Cited by: §2.2.
  • [3] A. Dhurandhar, T. Pedapati, A. Balakrishnan, P. Chen, K. Shanmugam, and R. Puri (2019) Model agnostic contrastive explanations for structured data. CoRR abs/1906.00117. External Links: Link, 1906.00117 Cited by: §2.2.
  • [4] A. Karimi, G. Barthe, B. Balle, and I. Valera (2020) Model-agnostic counterfactual explanations for consequential decisions. ArXiv abs/1905.11190. Cited by: §2.2, §2.2.
  • [5] H. M. Krumholz (2013) Variations in health care, patient preferences, and high-quality decision making. Journal of the American Medical Association 310 (2), pp. 151–152. Cited by: §1.
  • [6] A. V. Looveren and J. Klaise (2019) Interpretable counterfactual explanations guided by prototypes. CoRR abs/1907.02584. External Links: Link, 1907.02584 Cited by: §2.2.
  • [7] E. McFowland, S. Speakman, and D. B. Neill (2013) Fast generalized subset scan for anomalous pattern detection. The Journal of Machine Learning Research 14 (1), pp. 1533–1561. Cited by: §2.1, §4.2.
  • [8] E. McFowland III, S. Somanchi, and D. B. Neill (2018) Efficient discovery of heterogeneous treatment effects in randomized experiments via anomalous pattern detection. arXiv preprint arXiv:1803.09159. Cited by: §2.1.
  • [9] R. K. Mothilal, A. Sharma, and C. Tan (2020) Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* ’20, New York, NY, USA, pp. 607–617. External Links: ISBN 9781450369367, Link, Document Cited by: §2.2.
  • [10] D. B. Neill, E. McFowland III, and H. Zheng (2013) Fast subset scan for multivariate event detection. Statistics in medicine 32 (13), pp. 2185–2208. Cited by: §1, §2.1.
  • [11] D. B. Neill and T. Kumar (2013-04-04) Fast multidimensional subset scan for outbreak detection and characterization. Online Journal of Public Health Informatics 5 (1), pp. e91. Cited by: §3.2.
  • [12] D. B. Neill (2012) Fast subset scan for spatial pattern detection. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 74 (2), pp. 337–360. Cited by: §1, §2.1.
  • [13] C. Russell (2019) Efficient search for diverse coherent explanations. FAT* ’19, New York, NY, USA. External Links: ISBN 9781450361255, Link, Document Cited by: §2.2.
  • [14] B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio (2021) Toward causal representation learning.

    Proceedings of the IEEE - Advances in Machine Learning and Deep Neural Networks

    109 (5), pp. 612–634.
    External Links: Document, Link Cited by: §2.2.
  • [15] S. Sharma, J. Henderson, and J. Ghosh (2020) CERTIFAI: a common framework to provide explanations and analyse the fairness and robustness of black-box models. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES ’20, New York, NY, USA, pp. 166–172. External Links: ISBN 9781450371100, Link, Document Cited by: §2.2.
  • [16] E. Somanchi and D. B. Neill (2017) Detecting anomalous patterns of care using health insurance claims. Presented at Conference on Information Systems and Technology. Cited by: §2.1.
  • [17] A. White and A. S. d’Avila Garcez (2020) Measurable counterfactual local explanations for any classifier. In ECAI, Cited by: §2.2.
  • [18] Z. Zhang and D. B. Neill (2016) Identifying significant predictive bias in classifiers. arXiv preprint arXiv:1611.08292. Cited by: §2.1.