Efficient computation of counterfactual explanations of LVQ models

08/02/2019 ∙ by André Artelt, et al. ∙ Bielefeld University 0

With the increasing use of machine learning in practice and because of legal regulations like EU's GDPR, it becomes indispensable to be able to explain the prediction and behavior of machine learning models. An example of easy to understand explanations of AI models are counterfactual explanations. However, for many models it is still an open research problem how to efficiently compute counterfactual explanations. We investigate how to efficiently compute counterfactual explanations of learning vector quantization models. In particular, we propose different types of convex and non-convex programs depending on the used learning vector quantization model.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Due to the recent advances in machine learning (ML), ML models are being more and more used in practice and applied to real-world scenarios [1, 2, 3]. In order to be accepted by the user and the society in general, it becomes important to be able to explain the output and behavior of ML models. Especially because of legal regulations like the EU regulation ”General Data Protection Right” (GDPR) [4]

, that contains a ”right to an explanation”, it is nowadays indispensable to explain the output and behavior of artificial intelligence (AI) in a comprehensible way.

Recently the research community got attracted to explainability and transparency in machine learning [5, 6, 7, 8]. There exist some methods for explaining and understanding ML models [9]. One family of methods are model-agnostic methods [6, 10]. Model-agnostic methods are flexible in the sense that they are not tailored to a particular model or representation. This makes model-agnostic methods (in theory) applicable to many different types of ML models. In particular, ”truly” model-agnostic methods do not need access to the training data or model internals. It is sufficient to have an interface for passing data points to the model and obtaining the output/predictions of the model - the underlying model is viewed as a black-box.

Examples of model-agnostic methods are feature interaction methods [11], feature importance methods [12], partial dependency plots [13]

and local methods that approximates the model locally by an explainable model (e.g. a decision tree

[14, 15]. These methods explain the models by using features as vocabulary.

A different class of model-agnostic explanations are example-based explanations where a prediction or behavior is explained by a (set of) data points [16]. Instances of example-based explanations are prototypes & criticisms [17] and influential instances [18]. Another instance of example-based explanations are counterfactual explanations [19]. A counterfactual explanation is a change of the original input that leads to a different (specific) prediction/behavior of the ML model - what has to be different in order to change the prediction of the model? Such an explanation is considered to be fairly intuitive, human-friendly and useful because it tells people what to do in order to achieve a desired outcome [19, 9].

Model agnostic methods are universal in the sense that they are applicable to all models. However, usually model agnostic methods are computational expensive because they have to make many predictions using the model. Thus, model specific methods are relevant because they can be computed more efficiently. Therefore, it is desirable to have model specific techniques for computing explanations (e.g. contraining the architecture of a neural network 


Learning vector quantization (LVQ) models [21] are prototype based models that represent data by a set of prototypes. LVQ models can be combined with metric learning and thereby increase the effectiveness of the model in case of few prototypes [22, 23]. Furthermore, LVQ models can be used for incremental learning [24] and life-long learning [25].

Therefore, it is highly relevant to be able to efficiently compute counterfactual explanations of LVQ models. In this paper we investigate how to efficiently compute counterfactual explanations of LVQ models by exploiting the special structure of LVQ models. More precisely, our contributions are:

  • We propose model- and regularization-dependent methods for efficiently computing counterfactual explanations of LVQ models.

  • We empirically evaluate our methods and compare them with traditional methods.

The remainder of this paper is structured as follows: First, we briefly review counterfactual explanations (section 2.1) and learning vector quantization models (section 2.2). Then, in section 3 we introduce our convex and non-convex programming framework for efficiently computing counterfactual explanations of different types of LVQ models - note that all derivations can be found in the appendix (section 5). Furthermore, we empirically evaluate the efficiency of our methods. Finally, section 4 summarizes the results of this paper.

2 Review

2.1 Counterfactual explanations

Counterfactual explanations [19] (often just called counterfactuals) are an instance of example-based explanations [16]. Other instances of example-based explanations [9] are influential instances [18] and prototypes & criticisms [17].

A counterfactual states a change to some features/dimensions of a given input such that the resulting data point (called counterfactual) has a different (specified) prediction than the original input. Using a counterfactual instance for explaining the prediction of the original input is considered to be fairly intuitive, human-friendly and useful because it tells people what to do in order to achieve a desired outcome [19, 9].

A classical use case of counterfactual explanations is loan application [3, 9]: Imagine you applied for a credit at a bank. Unfortunately, the bank rejects your application. Now, you would like to know why. In particular, you would like to know what would have to be different so that your application would have been accepted. A possible explanation might be that you would have been accepted if you would earn 500$ more per month and if you would not have a second credit card.

Although counterfactuals constitute very intuitive explanation mechanisms, there do exist a couple of problems.

One problem is that there often exist more than one counterfactual - this is called Rashomon effect [9]. If there are more than one possible explanation (counterfactual), it is not clear which one should be selected.

Another disadvantage of counterfactuals is that there does not exist a general and efficient method for computing counterfactuals of categorical inputs - there are first attempts for bag-of-words models [26] but these methods work on a particular model in a particular domain only. However, recently CERTIFAI [27]

was proposed. In CERTIFAI, a genetic algorithm is used for computing counterfactuals - a benefit of using a genetic algorithm is that we can deal with categorical variables and we can add domain knowledge about feature (interaction) constraints, whereas the downside is time complexity and complex hyperparameter tuning (e.g. size of the population, how to do a cross over, mutation, …).

An alternative - but very similar in the spirit - to counterfactuals [19] is the Growing Spheres method [28]

. However, this method suffers from the curse of dimensionality because it has to draw samples from the input space, which can become difficult if the input space is high-dimensional.

Now, we first formally define the problem of finding counterfactuals in general form: Assume a prediction function is given. Computing a counterfactual of a given input 222We restrict ourself to , but in theory one could use an arbitrary domain . can be interpreted as an optimization problem [19]:



denotes a loss function that penalizes deviation of the prediction

from the requested prediction . denotes a regularization that penalizes deviations from the original input . The hyperparameter denotes the regularization strength.

Two common regularizations are the weighted Manhattan distance and the generalized L2 distance. The weighted Manhattan distance is defined as


where denote the feature wise weights. A popular choice [19] for is the inverse median absolute deviation of the -th feature median in the training data set :


The benefit of this choice is that it takes the (potentially) different variability of the features into account. However, because we need access to the training data set , this regularization is not a truly model-agnostic method - it is not usable if we only have access to a prediction interface of a black-box model.

The generalized L2 distance333also called Mahalanobis distance is defined as


where denotes a symmetric positive semi-definite (s.p.s.d.) matrix. Note that the L2 distance can be recovered by setting . The generalized L2 distance can be interpreted as the Euclidean distance in a linearly distorted space.

Depending on the model and the choice of and , the final optimization problem might be differentiable or not. If it is differentiable, we can use a gradient-based optimization algorithm like conjugate gradients, gradient descent or (L-)BFGS. Otherwise, we have to use a black-box optimization algorithm like Downhill-Simplex (also called Nelder-Mead) or Powell. However, the best choice is to use a model and regularization specific optimizer. Unfortunately, there exit model specific optimizers for few ML models only.

2.2 Learning vector quantization

In learning vector quantization (LVQ) models [21] we compute a set of labeled prototypes from a training data set of labeled real-valued vectors - we refer to the -th prototype as and the corresponding label as .

A new data point is classified according to the label of the nearest prototype:


where denotes a function for computing the distance between a data point and a prototype - usually this is the Euclidean distance444However, other distance functions are possible.:


There exist LVQ models like GMLVQ, LGMLVQ [22], MRSLVQ and LMRSLVQ [23] that learn a custom (class or prototype specific) distance matrix that is used instead of the identity when computing the distance between a data point and a prototype. This gives rise to the generalized L2 distance Eq. 4:


Because must be a s.p.s.d. matrix, instead of learning directly, these LVQ variants learn a matrix and compute the final distance matrix as:


By this, we can guarantee that the matrix is s.p.s.d., whereas the model only has to learn an arbitrary matrix - which is much easier than making some restriction on the definiteness. Training usually takes place by optimizing suitable cost functions as regards prototype locations and metric parameters. For counterfactual reasoning, the specific training method is irrelevant and we refer to the final model only.

3 Counterfactual explanations of LVQ models

We aim for an efficient explicit formulation how to find counterfactuals, given an input and a specific LVQ model.

3.1 General approach

Because a LVQ model assigns the label of the nearest prototype to a given input, we know that the nearest prototype of a counterfactual must be a prototype with . In order to compute a counterfactual of a given input , it is sufficient to solve the following optimization problem for each prototype with and select the counterfactual yielding the smallest value of :


where denotes the set of all prototypes not labeled as . Note that the feasible region of Eq. 9 is always non-empty - the prototype is always a feasible solution.

The pseudocode for computing a counterfactual of a LVQ model is described in Algorithm 1.

Input: Original input , requested prediction of the counterfactual, the LVQ model
Output: Counterfactual

1: Initialize dummy solution
3:for  with  do Try each prototype with the correct label
4:     Solving Eq. 9 yields a counterfactual
5:     if  then Keep this counterfactual if it deviates less from the original input then the currently ”best” counterfactual
8:     end if
9:end for
Algorithm 1 Computing a counterfactual of a LVQ model

Note that the for loop in line 3 of Algorithm 1 can be easily parallelized. Furthermore and in contrast to Eq. 1, we do not have any hyperparameters that need to be chosen.

In the subsequent sections we explore Eq. 9 for different regularizations and LVQ models, and investigate how to solve it efficiently. Note that for the purpose of better readability and due to space constraints, we put all derivations in the appendix (section 5).

3.2 (Generalized matrix) LVQ

When using (generalized matrix) LVQ models - no class or prototype specific distance matrices - the optimization problem Eq. 9

becomes either a linear program (LP) or a convex quadratic program (QP).

When using the weighted Manhattan distance Eq. 2 as a regularization , Eq. 9 can be written as a LP:




We set if the LVQ model uses the Euclidean distance.

When using the Euclidean distance Eq. 4 as a regularization , Eq. 9 can be written as a convex quadratic program with linear constraints:


Note that and are the same as in Eq. 10.

Eq. 10 and Eq. 14 both contain strict inequalities . Unfortunately, strict inequalities are not allowed in linear/quadratic programming because the feasible region would become an open set. However, we could turn the into a by subtract a small number from the right side of the inequality. In practice, when implementing our methods, we found that we can safely replace all by without changing anything else - this might be because of the numerics (like round-off errors) of fixed size floating-point numbers.

Both a linear and a convex quadratic program can be solved efficiently [29]. Another benefit is that we can easily add additional linear constraints like box constraints, restricting the value range of linear interaction of features or freezing some features - these features are not allowed to be different in the counterfactual.

3.3 (Localized generalized matrix) LVQ

In case of LVQ models that learn a class or prototype specific distance matrix , the optimization problem Eq. 9 becomes a quadratically constrained quadratic program (QCQP).

When using the weighted Manhattan distance Eq. 2 as a regularization , the optimization problem Eq. 9 becomes:




When using the Euclidean distance Eq. 4 as a regularization , Eq. 9 can be written as:


Unfortunately, we can not make any statement about the definiteness of . Because is the difference of two s.p.s.d. matrices, all we know is that it is a symmetric matrix. Therefore, Eq. 15 and Eq. 19 are both non-convex QCQPs and solving a non-convex QCQP is known to be NP-hard [30]. However, there exist methods like the Suggest-Improve framework [30] that can approximately solve a non-convex QCQP very efficiently - more details on how to apply this to Eq. 15 and Eq. 19 can be found in the appendix (section 5).

3.4 Experiments

In order to empirically confirm the efficiency of our proposed methods, we conducted some experiments:

We fitted GLVQ, GMLVQ and LGMLVQ models - we always used 3 prototypes per class - to the ”Breast Cancer Wisconsin (Diagnostic) Data Set” [31] and the ”Optical Recognition of Handwritten Digits Data Set” [32]. We used PCA to reduce the dimensionality of the ”Optical Recognition of Handwritten Digits Data Set” to and of the ”Breast Cancer Wisconsin (Diagnostic) Data Set” to .

We implemented our proposed methods (Algorithm 1) for computing counterfactual explanations555Available on GitHub https://github.com/andreArtelt/efficient_computation_counterfactuals_lvq. We used the SCS solver [33, 34] for solving linear/quadratic programs and the Suggest-Improve framework [30] for approximatly solving non-convex QCQPs - we simply picked the target prototype as an initial feasible solution in the Suggest-step and used the penalty convex-concave procedure (CCP)666As a subroutine, we used the MOSEK solver - https://www.mosek.com. We gratefully acknowledge the academic license provided by MOSEK ApS. in the Improve-step [35, 30].
For comparison, we used the optimizer for computing counterfactual explanations of LVQ models as implemented in ceml [36] - the optimization problem Eq. 1, where the loss function is the distance to the nearest prototype with the requested label , is minimized by the Downhill-Simplex algorithm. In all cases (including our methods), we used the Manhattan distance as a regularizer.

For each possible combination of model, data set and method, we did a 4-fold cross validation and recorded for each sample the computation time as well as the Manhattan distance between the counterfactual and the original data point.

The results are listed in Table 1 and Table 2.

Data set Breast cancer Handwritten digits
Model \Method Black-box Ours Black-box Ours
GLVQ 3.26 1.96 6.51 3.99
GMLVQ 2.71 2.46 21.34 4.40
LGMLVQ 2.00 1.57 8.12 7.53
Table 1: Mean Manhattan distance between the counterfactual and the original data point - best (smallest) values are highlighted.
Data set Breast cancer Handwritten digits
Model \Method Black-box Ours Black-box Ours
GLVQ 0.95 0.01 3.15 0.02
GMLVQ 0.25 0.01 2.12 0.02
LGMLVQ 0.91 0.65 3.35 2.09
Table 2: Mean computation time in s - best (smallest) values are highlighted.

In all cases, our method yields counterfactuals that are closer to the original data point and is always (much) faster than optimizing the original cost function Eq. 1 by using the Downhill-Simplex algorithm.

All experiments were implemented in Python 3.6 [37] using the packages cvxpy [38] cvx-qcqp [39], sklearn-lvq [40], numpy [41], scipy [42], scikit-learn [43] and ceml [36].

4 Conclusion

We proposed, and empirically evaluated, model- and regularization-dependent convex and non-convex programs for efficiently computing counterfactual explanations of LVQ models. We found that in many cases we get either a set of linear or convex quadratic programs which both can be solved efficiently. Only in the case of localized LVQ models we have to solve a set of non-convex quadratically constrained quadratic programs - we found that they can be efficiently approximately solved by using the Suggest-Improve framework.

5 Appendix

5.1 Minimizing the Euclidean distance

First, we expand the Euclidean distance (Eq. 4):


Next, we note that that we can drop the constant when optimizing with respect to :


5.2 Minimizing the weighted Manhattan distance

First, we transform the problem of minimizing the weighted Manhattan distance (Eq. 2) into epigraph form:


Next, we separate the dimensions:


After that, we remove the absolute value function:


Finally, we rewrite everything in matrix-vector notation:




5.3 Enforcing a specific prototype as the nearest neighbor

By using the following set of inequalities, we can force the prototype to be the nearest neighbor of the counterfactual - which would cause to be classified as :


We consider a fixed pair of and :




If we only have one global distance matrix , we find that and the inequality Eq. 28 simplifies:




If we do not use a custom distance matrix, we have and Eq. 28 becomes:




5.4 Approximately solving the non-convex QCQP

Recall the non-convex quadratic constraint from Eq. 28:


where the matrix is defined as the difference of two s.p.s.d. matrices:


By making use of Eq. 39, we can rewrite Eq. 38 as:




Under the assumption that our regularization function is a convex function777The weighted Manhattan distance and the Euclidean distance are convex functions!, we can rewrite a generic version of the non-convex QCQP (Eq. 15 and Eq. 19) as follows:


Note that we relaxed the strict inequality. Because and are s.p.s.d. matrices, we know that and are convex functions. Therefore, Eq. 43 is a difference-of-convex program (DCP).

This allows us to use the penalty convex-concave procedure (CCP) [30] for computing an approximate solution of Eq. 43 that is equivalent to the original non-convex QCQPs Eq. 15 and Eq. 19. For using the penalty CCP, we need the first order Taylor approximation of around a current point :