Towards Ground Truth Explainability on Tabular Data

07/20/2020 ∙ by Brian Barr, et al. ∙ Capital One NYU college 28

In data science, there is a long history of using synthetic data for method development, feature selection and feature engineering. Our current interest in synthetic data comes from recent work in explainability. Today's datasets are typically larger and more complex - requiring less interpretable models. In the setting of post hoc explainability, there is no ground truth for explanations. Inspired by recent work in explaining image classifiers that does provide ground truth, we propose a similar solution for tabular data. Using copulas, a concise specification of the desired statistical properties of a dataset, users can build intuition around explainability using controlled data sets and experimentation. The current capabilities are demonstrated on three use cases: one dimensional logistic regression, impact of correlation from informative features, impact of correlation from redundant variables.



There are no comments yet.


page 2

page 3

page 4

page 5

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The combination of large public datasets and novel machine learning architectures has provided state of the art predictive power in many diverse fields, such as computer vision and machine translation. These models are largely regarded as opaque


models. With their prevalence and increasing adoption, an active field of research is eXplainable Artificial Intelligence (XAI), which has sought to provide explanations for their predictions.

One avenue of research is on building interpretability into the model architecture Kim et al. (2015); Canini et al. (2016); Lee et al. (2019). We focus on the area post hoc explanations – which occurs after model training. Currently there is no one size fits all solution. The method of explanation depends on model type Chen et al. (2018); Krakovna and Doshi-Velez (2016); Grath et al. (2018), desired granularity Ribeiro et al. (2016); Ibrahim et al. (2019); Dhurandhar et al. (2018); Bhatt et al. (2020a); van der Linden et al. (2019), and audience Wachter et al. (2017); Bhatt et al. (2020b).

In support of the methods, there are a growing number of packages that seek to provide an umbrella of methods such as AIX360 Arya et al. (2019), ELI5 TeamHG-Memex (2019), and Alibi Klaise et al. (2020).

Early methods were focused on explaining image classifiers. The use of sensitivities of the output class based on the input image pixels, provides a visual and immediately interpretable explanation for the classifier’s prediction. For tabular data, the intuitive visual nature of sensitivities is not a natural metaphor. Additionally, where as computer vision typically relies on correlations between pixels as features, for tabular data that can be detrimental Aas et al. (2019).

An ongoing challenge in XAI is the lack of ground truth. To add to the landscape of XAI, and move towards ground truth explainability for tabular data, we provide a flexible synthetic data generation method allowing generation of arbitrarily complex data. The current implementation is focused on the task of binary classification.

The paper is structured as follows: previous work is discussed in Section 2, Section 3 presents the method used to construct the synthetic tabular data, and Section 4 shows some results from three use cases: one dimensional logistic regression, impact of correlation from informative features, and impact of correlation from redundant variables.

2 Previous work

Early use cases of synthetic data focused on the tasks of feature and model selection Guyon (2003). This method is available in the scikit-learn Pedregosa et al. (2011) module . An alternative method of generating tabular data for classification is to generate clusters and apply class labels to them.

Another approach is to model joint probability from an actual dataset. This can be helpful in dealing with sensitive data and as an aid in sharing data where there are privacy and regulatory restrictions on the use of actual data Ping et al. (2017); Howe et al. (2017); Gonçalves et al. (2020)

. Techniques used range from probabilistic models, to Bayesian networks, to generative adversarial neural networks (GANS). In finance, it is typical to use copulas. The theory of copulas has been developed considerably in mathematics, statistics, actuarial science, with significant interest in their application to finance

Genest et al. (2009), and their misuse may have led the financial crisis Salmon (2012). However, methods that mimic the statistics of other datasets are incapable of providing ground truth for explanations - since they lack the complete data generation process that imposes a functional dependence between and .

Our research is inspired from recent work in interpreting the image classifiers trained with a carefully crafted dataset that controls the relative feature importance Yang and Kim (2019). In this case, the model can be quantitatively evaluated in the form of known foreground and background images by providing ground truth of local feature importances.

We propose a similar method for tabular data. We use copulas to define the correlation structure and marginal distributions of the independent features. We specify the dependence of the underlying probability field as a symbolic expression. Binary class labels are assigned by setting a threshold probability. This method provides global explanations since we prescribe the coefficients of the terms in the symbolic expression. In some instances, where we build models only from informative features, we can provide ground truth local attributions.

Our contributions are providing a unique and flexible method for synthetic tabular data generation suitable for current model architectures and demonstration of its use in informative experiments highlighting that not all correlation in inputs change local attributions.

Figure 2: Joint probability plot for features and (a) with Gaussian marginals (b) with uniform marginals with no correlation and (c) uniform marginals with positive correlation.

3 Synthetic data generation

The generation of synthetic data seeks a method to provide the joint probability where are the input features, and

is the output variable, and to draw samples from that joint distribution. From those samples, we will fit machine learned models that approximate the conditional probability

(via possibly black box models) and to provide explanations for those models.

We separate the input feature vectors,

, into three categories - informative, redundant, and nuisance features.


Informative features, , used to determine the binary labels, are specified by a copula. A multivariate distribution can be separated into a set of marginal distributions and the correlation structure between them. Copula theory is the mathematical framework of the separation of the correlation structure from the marginal distributions of the feature vectors. See Jaworski et al. (2013); Nelsen (1999) for further details. The current library supports any marginal distribution available in scipy.stats. The results for this paper use a multivariate Gaussian copula.

Redundant features, , are a random linear combination of informative features. Nuisance features, are random uncorrelated vectors drawn from the interval [-1, 1] which are useful as benchmarks to set the lower bound on explainability. Being purely random and not contributing to the specification of the labels, any feature found to have lower importance than a nuisance feature should be subjected to further scrutiny and possibly removed from the model.

The final step in the process is to generate binary labels for the inputs. First, a scalar regression value is created from the informative features via a symbolic expression using the sympy python module.


To generate binary classification probabilities, the regression value is squashed by a sigmoid to the range [0,1]. After setting a threshold probability, class labels are determined.

Additional post processing allows the addition of Gaussian noise to the informative features. A random seed can be specified so that repeated runs of a synthetic dataset yield the same values.

4 Experiments

This section demonstrates some of the capabilities of the synthetic tabular data through the process of modeling and providing local attributions via the SHAP library Lundberg and Lee (2017).

4.1 Logistic regression

The first synthetic data set is for two features, , with no covariance and Gaussian marginal distributions for both. The joint probability plot is shown in Figure 1(a).

The symbolic regression expression is . The probability values and class labels are shown in Figure 0(a). One thousand samples are generated.

The dataset is split 70/30 into a training and test set, with a logistic regression model fit to the training data. The AUC of the model is 99.8% and the coefficients of the model are . To provide local attributions, we fit a SHAP KernelExplainer to the training set with the results shown in Figure 3.

Figure 3: Contours of SHAP values for (left) and (right) for the simple 1-D logistic regression model.

The SHAP values for dominate by two orders of magnitude, roughly in keeping with the relative global importance found in the coefficients. Strong left to right symmetry is broken only in sparsely populated regions. Interesting to note that the SHAP values for also display symmetry from top to bottom - with the opposite sign of the coefficient.

Figure 4: Contours of SHAP values for (top row) and (bottom row) for the baseline data and model (left), correlated model and data (center), and correlated model fit with baseline data (right).

4.2 Presence of correlation in informative features

We investigate the impact of correlation between the input features on model explanations. It is well known that the presence of correlation can alter a model’s explanations, see for example Breiman (2001); Aas et al. (2019).

We again generate datasets for two features, ; the first with uniform marginal distributions, a baseline dataset with no correlation whose joint distribution can be seen in Figure 1(b), and a second dataset, see Figure 1(c) with the covariance specified as:

The symbolic regression expression for this experiment is:

We hold the probability field constant between the datasets. The probability values and class labels are shown in Figure 0(b).

The dataset is split 70/30 into a training and test set, with a dense network with three hidden layers and a total of 255 weights in Tensorflow fit to the training data. The AUC of the resulting model is 100%. To provide local attributions, we fit a SHAP DeepExplainer to the training set with the results shown in Figure 


There is an apparent decrease in SHAP values in the presence of correlation in the input features. Recall that SHAP values are relative with respect to the expectation of the output. With correlation, their density is drawn out of quadrants 2 and 4, and placed in quadrants 1 and 3, leading to a higher expected predicted probability due to the imbalance of the class labels. If we account for that effect by refitting the explainer with the baseline data (which does not suffer the same level of imbalance), the SHAP values look essentially like those from the uncorrelated inputs (as shown in the last column of Figure 4)

In this context, the apparent effect of correlation is an artifact induced from the expectation of the correlated model predictions over the correlated training data – it is not a bias that the model has learned during training.

4.3 Presence of redundant variables

In contrast to the experiment in the previous section, this section considers the effect of correlation with unimportant features – since the redundant features are not explicitly used in the symbolic expression used to derive the binary labels. We reuse the baseline from the previous section. We create a second dataset by augmenting the baseline informative features with two redundant and two nuisance features.

The joint distribution of the informative features, the symbolic expression that maps the informative features to the binary labels all remain unchanged. We keep the same train/test split. The only change to the dense network is to increase the dimension of the input layer to accommodate the two redundant and two nuisance features.

Figure 5: Contours of SHAP values for (top row) and (bottom row) for the baseline data and model(left), redundant feature model for three components of explanation: informative, redundant, and nuisance.

Redundant features have a significant impact on SHAP’s perception of feature importance. Summing the individual components of explanations for the redundant feature model does not recover the values obtained from the baseline model.

5 Conclusions

In this paper, we have described a method of generating synthetic tabular data utilizing copulas and symbolic expressions that provides users the ability to control the complexity of the resulting dataset. We have briefly demonstrated how the origin of correlation amongst inputs (informative vs redundant) influences model explainability. This method is a powerful investigative tool for data scientists, model developers, and explainable AI method developers.

Future work includes: further experimentation with more complex data sets, bridging the gap between global explanations (provided by our symbolic expression) and local explanations, investigation of categorical and ordinal variables, as well as improving local attributions for tabular data.

At the time of publication, we were working on making the repository accessible. Please contact a corresponding author for further details.


  • K. Aas, M. Jullum, and A. Løland (2019) Explaining individual predictions when features are dependent: more accurate approximations to shapley values. CoRR abs/1903.10464. External Links: Link, 1903.10464 Cited by: §1, §4.2.
  • V. Arya, R. K. E. Bellamy, P. Chen, A. Dhurandhar, M. Hind, S. C. Hoffman, S. Houde, Q. V. Liao, R. Luss, A. Mojsilović, S. Mourad, P. Pedemonte, R. Raghavendra, J. Richards, P. Sattigeri, K. Shanmugam, M. Singh, K. R. Varshney, D. Wei, and Y. Zhang (2019) One explanation does not fit all: a toolkit and taxonomy of ai explainability techniques. External Links: Link Cited by: §1.
  • U. Bhatt, A. Weller, and J. M. Moura (2020a) Evaluating and aggregating feature-based model explanations. arXiv preprint arXiv:2005.00631. Cited by: §1.
  • U. Bhatt, A. Xiang, S. Sharma, A. Weller, A. Taly, Y. Jia, J. Ghosh, R. Puri, J. M. Moura, and P. Eckersley (2020b) Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 648–657. Cited by: §1.
  • L. Breiman (2001) Statistical modeling: the two cultures (with comments and a rejoinder by the author). Statist. Sci. 16 (3), pp. 199–231. Cited by: §4.2.
  • K. Canini, A. Cotter, M. R. Gupta, M. M. Fard, and J. Pfeifer (2016) Fast and flexible monotonic functions with ensembles of lattices. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, Red Hook, NY, USA, pp. 2927–2935. External Links: ISBN 9781510838819 Cited by: §1.
  • C. Chen, O. Li, C. Tao, A. J. Barnett, J. Su, and C. Rudin (2018)

    This looks like that: deep learning for interpretable image recognition

    External Links: 1806.10574 Cited by: §1.
  • A. Dhurandhar, P. Chen, R. Luss, C. Tu, P. Ting, K. Shanmugam, and P. Das (2018) Explanations based on the missing: towards contrastive explanations with pertinent negatives. External Links: 1802.07623 Cited by: §1.
  • C. Genest, M. Gendron, and M. Bourdeau-Brien (2009) The advent of copulas in finance. The European journal of finance 15 (7-8), pp. 609–618. Cited by: §2.
  • A. R. Gonçalves, P. Ray, B. Soper, J. L. Stevens, L. Coyle, and A. P. Sales (2020) Generation and evaluation of synthetic patient data. BMC Medical Research Methodology 20, pp. 1471–2288. External Links: Document Cited by: §2.
  • R. M. Grath, L. Costabello, C. L. Van, P. Sweeney, F. Kamiab, Z. Shen, and F. Lecue (2018) Interpretable credit application predictions with counterfactual explanations. External Links: 1811.05245 Cited by: §1.
  • I. Guyon (2003) Design of experiments of the NIPS 2003 variable selection benchmark. Technical report ClopiNet. External Links: Link Cited by: §2.
  • B. Howe, J. Stoyanovich, H. Ping, B. Herman, and M. Gee (2017) Synthetic data for social good. CoRR abs/1710.08874. External Links: Link, 1710.08874 Cited by: §2.
  • M. Ibrahim, M. Louie, C. Modarres, and J. W. Paisley (2019) Global explanations of neural networks: mapping the landscape of predictions. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES 2019, Honolulu, HI, USA, January 27-28, 2019, V. Conitzer, G. K. Hadfield, and S. Vallor (Eds.), pp. 279–287. External Links: Link, Document Cited by: §1.
  • P. Jaworski, F. Durante, and H. Wolfgang (2013) Copulae in mathematical and quantitative finance: proceedings of the workshop held in cracow, 10-11 july 2012. Springer. Cited by: §3.
  • B. Kim, J. A. Shah, and F. Doshi-Velez (2015) Mind the gap: a generative approach to interpretable feature selection and extraction. In Advances in Neural Information Processing Systems, pp. 2260–2268. Cited by: §1.
  • J. Klaise, A. Van Looveren, G. Vacanti, and A. Coca (2020) Alibi: algorithms for monitoring and explaining machine learning models. External Links: Link Cited by: §1.
  • V. Krakovna and F. Doshi-Velez (2016)

    Increasing the interpretability of recurrent neural networks using hidden markov models

    External Links: 1606.05320 Cited by: §1.
  • G. Lee, D. Alvarez-Melis, and T. S. Jaakkola (2019) Towards robust, locally linear deep networks. CoRR abs/1907.03207. External Links: Link, 1907.03207 Cited by: §1.
  • S. M. Lundberg and S. Lee (2017) A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), pp. 4765–4774. External Links: Link Cited by: §4.
  • R. B. Nelsen (1999) An introduction to copulas. Springer. Cited by: §3.
  • F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay (2011) Scikit-learn: machine learning in Python. Journal of Machine Learning Research 12, pp. 2825–2830. Cited by: §2.
  • H. Ping, J. Stoyanovich, and B. Howe (2017) DataSynthesizer: privacy-preserving synthetic datasets. In Proceedings of the 29th International Conference on Scientific and Statistical Database Management, pp. 42:1–42:5. Cited by: §2.
  • M. T. Ribeiro, S. Singh, and C. Guestrin (2016) ”Why should I trust you?”: explaining the predictions of any classifier. CoRR abs/1602.04938. External Links: Link, 1602.04938 Cited by: §1.
  • F. Salmon (2012) The formula that killed wall street. Significance 9 (1), pp. 16–20. Cited by: §2.
  • TeamHG-Memex (2019) Welcome to eli5’s documentation!. TeamHG-Memex (English). External Links: Link Cited by: §1.
  • I. van der Linden, H. Haned, and E. Kanoulas (2019) Global aggregations of local explanations for black box models. External Links: 1907.03039 Cited by: §1.
  • S. Wachter, B. Mittelstadt, and C. Russell (2017) Counterfactual explanations without opening the black box: automated decisions and the gdpr. External Links: 1711.00399 Cited by: §1.
  • M. Yang and B. Kim (2019) Benchmarking Attribution Methods with Relative Feature Importance. CoRR abs/1907.09701. External Links: Link, 1907.09701 Cited by: §2.