Encoding Concepts in Graph Neural Networks

The opaque reasoning of Graph Neural Networks induces a lack of human trust. Existing graph network explainers attempt to address this issue by providing post-hoc explanations, however, they fail to make the model itself more interpretable. To fill this gap, we introduce the Concept Encoder Module, the first differentiable concept-discovery approach for graph networks. The proposed approach makes graph networks explainable by design by first discovering graph concepts and then using these to solve the task. Our results demonstrate that this approach allows graph networks to: (i) attain model accuracy comparable with their equivalent vanilla versions, (ii) discover meaningful concepts that achieve high concept completeness and purity scores, (iii) provide high-quality concept-based logic explanations for their prediction, and (iv) support effective interventions at test time: these can increase human trust as well as significantly improve model performance.

READ FULL TEXT VIEW PDF

page 6

page 15

page 16

page 19

page 20

page 21

07/25/2021

GCExplainer: Human-in-the-Loop Concept-based Explanations for Graph Neural Networks

While graph neural networks (GNNs) have been shown to perform well on gr...
09/19/2022

Concept Embedding Models

Deploying AI-powered systems requires trustworthy models supporting effe...
06/12/2021

Entropy-based Logic Explanations of Neural Networks

Explainable artificial intelligence has rapidly emerged since lawmakers ...
08/22/2022

Global Concept-Based Interpretability for Graph Neural Networks via Neuron Analysis

Graph neural networks (GNNs) are highly effective on a variety of graph-...
08/08/2021

Human-in-the-loop Extraction of Interpretable Concepts in Deep Learning Models

The interpretation of deep neural networks (DNNs) has become a key topic...
05/01/2021

A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts

Despite substantial progress in applying neural networks (NN) to a wide ...
01/11/2021

Learning Semantically Meaningful Features for Interpretable Classifications

Learning semantically meaningful features is important for Deep Neural N...

1 Introduction

Human trust in machine learning requires high task performance alongside interpretable decision making 

(Shen, 2022). For this reason, the opaqueness of Graph Neural Networks (GNNs, (Scarselli et al., 2008))—despite their state-of-the-art performance (Battaglia et al., 2016; Pal et al., 2020; Stokes et al., 2020; Davies et al., 2021)—raises ethical (Durán and Jongsma, 2021; Lo Piano, 2020) and legal (Wachter et al., 2017; EUGDPR, 2017) concerns. As their practical deployment is now under question, interpreting GNN reasoning has become a major concern in the field (Rudin, 2019; Ying et al., 2019).

Early explainability methods for GNNs produce local, low-level post-hoc explanations (Ying et al., 2019; Luo et al., 2020; Vu and Thai, 2020), which exhibit the same unreliability as analogous methods for convolutional networks (Adebayo et al., 2018; Kindermans et al., 2019; Ghorbani et al., 2019a). In contrast, concept-based explainability overcomes the brittleness of low-level explanations by providing robust global explanations in form of human-understandable concepts (Kim et al., 2018), i.e., interpretable high-level units of information (Ghorbani et al., 2019b; Koh et al., 2020). In relational learning, the Graph Concept Explainer (GCExplainer, (Magister et al., 2021)) pioneered concept-based explainability for GNNs by extracting global subgraph-based concepts, such as a “house-shaped” structure, from the latent space of a trained model. This way users can check whether the extracted concepts are meaningful (Ghorbani et al., 2019b), whether they are coherent across samples (Magister et al., 2021), and whether they contain sufficient information to solve a target task (Yeh et al., 2020). However, as any post-hoc approach, GCExplainer does not encourage the GNN to make interpretable predictions using the extracted concepts (Rudin, 2019). As a result, the opaque reasoning of GNNs ultimately remains an open problem.

To fill this knowledge gap, we propose the Concept Encoder Module (CEM, Figure 1), the first concept-based end-to-end differentiable approach which makes graph networks explainable by design. It achieves this by first discovering a set of concepts and then using these to solve the task at hand. Our module can be introduced in any GNN architecture. We will refer to the resulting family of architectures as “Concept Graph Networks”, or CGNs. We experimentally show that CGNs: (i) attain better or competitive task accuracy w.r.t. their equivalent vanilla GNN, (ii) encode coherent human-understandable concepts and obtain high scores in all the key concept-based explainability metrics, i.e., purity and completeness, (iii) can provide simple and accurate logic explanations based on discovered concepts, (iv) allow effective interventions at concept level: these can increase human trust and significantly improve model performance.

Figure 1:

The proposed concept encoder module makes graph neural networks explainable-by-design by first discovering a set of concepts and then using these to solve the task with an interpretable classifier.

2 Background

Graph Neural Networks

Graph Neural Networks (GNNs, (Scarselli et al., 2008)) are differentiable models designed to process relational data in the form of graphs. A graph can be defined as a tuple which comprises nodes , the entities of a domain, and edges , the relations between pairs of nodes. Nodes (or edges) can be endowed with features , representing characteristics of each entity (or relation), and with ground truth task labels . A typical GNN learns a set of node embeddings with a scheme known as message passing (Gilmer et al., 2017). Specifically, message passing aggregates for each node local information shared by its neighboring nodes :

(1)

A readout function then processes the node embeddings to predict node labels

. GNNs are trained via stochastic gradient descent minimizing the cross entropy loss between predicted

and ground-truth task labels .

Graph Concept Explainer

The Graph Concept Explainer (GCExplainer, (Magister et al., 2021)) is the first (and only) concept-based approach for interpreting GNNs. Following methods successfully applied in vision (Ghorbani et al., 2019b)

, GCExplainer is an unsupervised approach for post-hoc discovery of global concepts. It achieves this by applying k-Means clustering 

(Forgy, 1965) on the node embeddings of a trained GNN. Magister et al. (2021) argued that each of the clusters possibly represents a learnt concept according to human perception, as already suggested by Zhang et al. (2018) and Yeh et al. (2020). Using this clustering, GCExplainer then assigns a concept label to each sample. Finally, it represents each concept using the five samples closest to each cluster centroid, where each sample is visualized as a subgraph with the corresponding node and its -hop neighbors. This visualization technique is effective and aligned with the reasoning of GNNs, as it takes into account how the information flows via message passing. For example, after three layers of message passing, each node can receive at most information from its -hop neighbors.

Trust through concepts and interventions

Predicting tasks as a function of learnt concepts makes the decision process of deep learning models more interpretable 

(Koh et al., 2020; Shen, 2022). In fact, learning intermediate concepts allows models to provide concept-based explanations for their predictions (Ghorbani et al., 2019a) which can take the form of simple logic statements, as shown by Ciravegna et al. (2021). In addition, Koh et al. (2020) showed how learning intermediate concepts allows human experts to rectify mispredicted concepts through effective test-time interventions, thus improving model’s performance and engendering human trust (Shen, 2022).

3 Encoding Concepts in Graph Neural Networks

GCExplainer significantly improves the state of the art in GNN explainability, paving the way for concept-based approaches (Magister et al., 2021). However, all existing methods for graph explainability, including GCExplainer, are post-hoc (Ying et al., 2019; Luo et al., 2020; Vu and Thai, 2020; Bajaj et al., 2021; Lucic et al., 2022). At best, post-hoc techniques can correctly describe what models learn (Rudin, 2019), but they cannot make the GNN itself more interpretable. Therefore, the opaque reasoning of GNNs remains an open problem. To fill this knowledge gap, we propose the Concept Encoder Module (CEM), a differentiable approach which makes GNNs explainable by design. In fact, CEM empowers GNNs through a more interpretable decision making process. It achieves this by first extracting a set of concepts and then using these to solve a classification task. Our approach can be integrated in any GNN architecture. We will refer to the resulting family of architectures as “Concept Graph Networks”, or CGNs. As in GCExplainer, humans can visualize CGN concepts to check whether they are meaningful and coherent. Yet, in contrast to GCExplainer, CGNs allow effective interventions at concept level, allowing human experts to significantly improve model performance. CEM integrates a differentiable concept encoding layer to extract node-level and graph-level concepts with an interpretable task predictor providing logic-based explanations.

Concept encoding

The first CEM step consists in extracting node-level clusters corresponding to concepts. However, in contrast to GCExplainer, in CEM this step is differentiable and integrated in the network architecture, allowing gradients to optimize clusters in GNN embeddings. Specifically, we implemented this differentiable clustering using a normalized softmax activation on the node-level embeddings , associating each node with one cluster/concept. This operation returns for each node a fuzzy encoding :

(2)

CEM then clusters nodes considering the similarity of their fuzzy encodings . Specifically, CEM groups the samples together depending on their Booleanized encoding :

(3)

where is conventionally set to . In particular, two samples and belong to the same cluster if and only if their encodings and match. For example, consider the two node embeddings and . For these inputs, the normalized softmax would return the fuzzy encodings and , respectively. As their Booleanizations and do not match, we can then conclude that the two nodes belong to different clusters. Notice how our concept encoding is highly efficient, as it allows to learn up to different concepts on GNN embeddings of size . This way the GNN can dynamically find the optimal number of concepts/clusters, thus relieving users from this burden. In fact, users just need to choose an upper bound to the number of concepts rather than an exact value as in k-Means for GCExplainer. In order to account for graph classification, the concept encodings for a graph are pooled before being passed to the interpretable model predicting the task, as explained in the next paragraph.

Interpretable predictions

The second CEM step consists of using the extracted concepts to make interpretable predictions for downstream tasks. In particular, the presence of concepts enables pairing GNNs with existing concept-based methods which are explainable by design, such as Logic Explained Networks (LENs, (Ciravegna et al., 2021)). LENs are neural models providing simple concept-based logic explanations for their predictions. Specifically, LENs can provide class-level explanations which makes our approach the first at providing unique global explanations for GNNs. CEM uses a LEN as the readout function for the classification, applying it on top of concept representations . For graph classification tasks, the input data is composed of a set of graphs , where each graph is associated with a task label . In this setting, GNN-based models predict a single label for each graph by pooling its node-level encodings to aggregate over multiple concepts:

(4)

where is the number of nodes associated with graph . In our implementation, we use the entropy-based layer to implement LENs (Barbiero et al., 2021)) as it can provide high classification accuracy with high-quality logic explanations. This entropy-based layer implements a sparse attention layer designed to work on top of concept activations. The attention mechanism allows the model to focus on a small subset of concepts to solve each task. It also introduces a parsimony principle in the architecture corresponding to an intuitive human cognitive bias (Miller, 1956). This parsimony principle allows the extraction of simple logic explanations from the network, thus making these models explainable by design.

Concept-based and logic-based explanations

The proposed method provides two types of explanations: concept-based and logic-based explanations. Global concept-based explanations can be extracted in a similar manner as in GCExplainer: a concept for a node or graph is extracted by finding the cluster with which a node’s embedding is associated, and visualising the samples closest to the cluster centroid. The logic-based formula provided per class broadens the explanation scope, as it indicates which neurons of the concept encoding

are activated and representative of a class. This provides a more comprehensive explanation since a class can be associated with multiple concepts.

Concept interventions

As in Concept Bottleneck Models (Koh et al., 2020), our approach supports human interaction at concept level. In fact, in contrast to existing post-hoc methods, a explainable-by-design approach creates an explicit concept layer which can positively react to test-time human interventions. For instance, consider a misclassified node with concept encoding . Assume that the vast majority of nodes with the binary encoding are nodes of a grid-like structure, which allows a human to label this cluster as “grid nodes”. Now, a human expert can inspect the neighborhood of the misclassified node and realize that this node belongs to a circle-like structure and not to a grid structure. As the binary encoding for the concept “circle nodes” is , the user can easily apply an intervention to correct the misclassified concept by changing its encoding to . Such an update allows the interpretable readout function to act on information related to the corrected concept, thus improving the original model prediction.

4 Experiments

In our experiments we focus on the following research questions:

  • Task accuracy and completeness — What is the impact of our approach on the generalization error of a GNN? Is the identified concept set complete w.r.t. the task?

  • Concept interpretability — Are the unsupervised concepts identified by our model meaningful? Do they match ground truths or human expectations?

  • Explanation performance — Are concepts pure and coherent? Are the logic explanations provided by our model accurate? Are they simple enough to be interpretable?

With these questions in mind, we hypothesize that our approach can: (i) obtain similar task accuracy w.r.t. a standard GNN; (ii) extract the ground truth graph concepts aligned with human expectations, and (iii) identify pure concepts as well as simple and accurate logic explanations.

Metrics

In our evaluation, we measure model performance and interpretability based on five metrics. We measure model performance via classification accuracy to compare the generalization error of CGNs w.r.t. their equivalent vanilla GNNs. To evaluate model interpretability, we compute concept completeness (Yeh et al., 2020), which assesses whether the concepts discovered are sufficient to describe the downstream task. Following Yeh et al. (2020)

, we use a decision tree 

(Breiman et al., 1984) to predict the task labels given the concept encoding associated with each input instance. We also examine concept coherence via concept purity (Magister et al., 2021). Following Magister et al. (2021), we measure concept purity by considering the graph edit distance of samples’ neighborhoods within each cluster/concept. Having checked concept quality, we evaluate logic explanations in terms of their accuracy and complexity. We calculate the accuracy of logic explanations using the learnt logic formulas for classifying test samples based on their concept encoding as done by Ciravegna et al. (2021). This mirrors the computation of concept completeness, however, instead of a decision tree, we use the learnt logic formulas for classification. Lastly, we evaluate the complexity of logic explanations by measuring the number of terms in logic rules (Ciravegna et al., 2021). We compute all metrics on test sets across five random weight initializations and report their means and confidence intervals using the Box-Cox transformation for non-normal distributions. Further details on our evaluation can be found in the Appendix. We do not measure classical explainability metrics, such as sensitivity and sparsity (Zhou et al., 2021), as they apply to explainers of models rather than explainable-by-design networks themselves.

Datasets

We perform the experiments on the same set of datasets as GNNExplainer (Ying et al., 2019), as subsequent research established them as benchmarks (Luo et al., 2020; Vu and Thai, 2020; Magister et al., 2021). We use five synthetic node classification datasets, which have a ground truth motif encoded. A ground truth motif is a subgraph in the graph, which a successful GNN explainability technique should recognize. The node classification datasets are: (i) BA-Shapes, consisting of a single graph where the base structure is a Barabási-Albert (BA) graph (Barabási and Albert, 1999), which has house motifs and random edges attached to it; (ii) BA-Community, generated by the union of two BA-Shapes graphs with the task of classifying nodes belonging to house motifs, as well as the membership of the node in one of the two houses; (iii) BA-Grid, based on a grid motif attached to a BA graph, similarly to BA-Shapes; (iv) Tree-Cycles, formed by a binary tree with cycle structures attached; (v) Tree-Grid, consisting of a binary tree with grid structures attached. We also include two real-world datasets to evaluate model performance on less structured data and on graph classification tasks. The two real world datasets are: (i) Mutagenicity (MUTAG, (Morris et al., 2020)), a collection of graphs representing molecules labelled as mutagenic or non-mutagenic, and (ii) Reddit-Binary (Reddit, (Morris et al., 2020)), a collection of graphs representing the structure of Reddit discussion threads where nodes represent users and edges the interaction between users. A challenge in evaluating these datasets is that there are no ground truth motifs. However, Ying et al. (2019) suggests the ring structure and the nitrogen dioxide compound in Mutagenicity, and the star-like structure in Reddit-Binary as desirable motifs to be recovered.

Baselines and setup

To address our research questions, we compare our approach against an equivalent convolutional vanilla GNN explained by GCExplainer. Notice that we do not focus on other post-hoc explainability methods, such as GNNExplainer (Ying et al., 2019), PGExplainer (Luo et al., 2020) or PGM-Explainer (Vu and Thai, 2020), as to the best of our knowledge GCExplainer is the only explainability method providing global concept-based explanations for GNNs. We refer the reader to the Appendix for a comparison of the proposed method with GNNExplainer and for an evaluation of the proposed approach with further GNN architectures.

We select the models’ hyperparameters, such as the number of hidden units and learning rate, using a grid search. To ensure fairness in our results, we use the same architecture capacity and hyperparameters for our model as well as for its vanilla counterpart. We initialize the hyperparameters of GCExplainer to the values determined experimentally by 

Magister et al. (2021). We refer the reader to the Appendix for a detailed discussion of the setup.

5 Results

5.1 Task Accuracy and Completeness

Concept Graph Networks are as accurate as vanilla GNNs (Figure 2 left)

Our results show that CEM allows GNNs to achieve better or comparable task accuracy w.r.t. equivalent GNN architectures. Specifically, our approach outperforms vanilla GNNs on the Tree-Cycle dataset, having a higher test accuracy (plus

on average) and less variance across different parameter initializations. We hypothesize that this effect is due to more stable and pure concepts being learnt thanks to CEM, as we will see later when discussing Figure 

3. We do not observe any significant negative effect of using CEM on the generalization error of GNNs.

Figure 2: Model accuracy and concept completeness for the Concept-based Graph Network (CGN) and an equivalent vanilla GNN. For these results, and those that follow, we compute all metrics on test sets across five seeds and report their mean and confidence intervals.

The Concept Encoder Module discovers complete concepts (Figure 2 right)

Our experiments show that overall CEM discovers a more complete set of concepts w.r.t. the concept set extracted by GCExplainer on equivalent GNN architectures. This is particularly emphasized in the Tree-Grid, BA-Shapes, BA-Community, and Mutagenicity datasets, where CEM significantly outperforms GCExplainer by up to

. For the other datasets, the proposed approach matches the concept completeness scores of GCExplainer, except for the Reddit-Binary dataset. However, a detailed analysis of completeness scores across the five different runs reveals that the mean is skewed by a single run. In absolute terms, CEM discovers highly complete sets of concepts with completeness scores close to the model accuracy.

5.2 Concept Interpretability

The Concept Encoder Module identifies meaningful concepts (Table 1)

CEM discovers high-quality concepts, which are meaningful to humans. Similarly to GCExplainer, our results demonstrate that CEM can discover concepts corresponding to the ground truth motifs embedded in the toy datasets. For example, our approach recovers the “house motif” in BA-Shapes. Moreover, CEM proposes plausible concepts for the real-world datasets where ground truth motifs are lacking. In this case, the extracted concepts match the desirable motifs suggested by Ying et al. (2019), corresponding to ring structures and the nitrogen dioxide compound in Mutagenicity, and a star-like structure in Reddit-Binary. As we use the same visualization technique as GCExplainer the merit of our contribution lies in the discovery of a more descriptive set of concepts, which includes rare and fine-grained concepts. As a thorough qualitative comparison with GCExplainer requires exhaustive visualization, we refer the reader to the Appendix for a complete set of results.

BA-Shapes BA-Grid Tree-Grid Tree-Cycle BA-Community Mutagenicity Reddit-Binary
Ground Truth
Extracted Concept
Table 1: The Concept Graph Module detects meaningful concepts matching the expected ground truth. Blue nodes are the instances being explained, while orange nodes represent their -hop neighbors. Similar motifs are identified by GCExplainer.
Ground Truth Fine-Grained Concepts Rare Concepts
Table 2: The Concept Graph Module detects concepts more fine-grained than the simple ground truth motif encoded as well as rare motifs. Blue nodes are the instances being explained, while orange nodes represent their -hop neighbors. Notably, GCExplainer gives no indication of rare concepts.

The Concept Encoder Module identifies rare and fine-grained concepts (Table 2)

CEM discovers more fine-grained concepts than just the “house motif” suggested by GNNExplainer, as it can differentiate whether a middle or bottom node is on the far or near side of the edge attaching to the BA graph. This matches the quality of concepts extracted by GCExplainer. In contrast to GCExplainer, CEM also identifies rare concepts. Rare motifs are present in toy datasets through the insertion of random edges. As the proposed approach can find the optimal number of clusters/concepts dynamically, clusters of a very small size possibly represent rare motifs. To check the presence of rare concepts, we visualize the -hop neighbors of nodes found in small clusters. For example, CEM identifies a rare concept represented as a “house” structure attached to the BA graph via the top node of the house in the BA-Shapes dataset. This represents a rare concept as it is generated by the insertion of a random edge. We confirm this observations in other toy datasets, such as BA-Community and Tree-Cycle, where motifs with random edges are clearly identified. We have not identified rare concepts in BA-Grid or Tree-Grid, which may be attributed to the random edges being distributed within the base graph, which has a less definite structure. Due to the lack of expert knowledge, we cannot confirm whether the rare motifs found in Mutagenicity and Reddit-Binary align with human expectations.

5.3 Explanation performance

The Concept Encoder Module identifies pure concepts (Figure 3)

CEM discovers high-quality concepts, which are coherent across samples, as measured by concept purity. In terms of purity, our approach discovers concepts with nearly optimal scores in toy datasets, with a graph edit distance close to zero. For these datasets, CEM provides either better or comparable purity scores when compared to GCExplainer. CEM provides slightly worse purity scores in both the Mutagenicity and Reddit-Binary datasets. However, also in this case the absolute purity of CEM is almost optimal.     Figure 3: Purity scores for the concept extracted by Concept Graph Module and GCExplainer. Notice how the optimal purity score is zero, as it measures the graph edit distance between concept instances (Magister et al., 2021).

The Concept Encoder Module provides accurate logic explanations (Figure 4, Table 3)

LEN allows CEM to provide simple and accurate logic explanations for task predictions. The accuracy of the logic explanations extracted reaches at least across all synthetic datasets, indicating that CEM derives a precise description of the model decision process. Relating the accuracy of explanations back to the model accuracy, we observe that the explanation accuracy is bounded by task performance, as already noticed by Ciravegna et al. (2021). This explains the slightly lower logic explanation accuracy on the real-world datasets, which can be ascribed to the absence of definite ground-truth concepts and to the classification task being more complex. Besides being accurate, logic explanations are very short, with a complexity below terms. In conjunction with the explanation accuracy, this means that CEM finds a small set of predicates which accurately describes the most relevant concepts for each class.

Figure 4: Accuracy and complexity of logic explanations provided by the proposed Concept Graph Module. The accuracy is computed using logic formulas to classify samples based on their concept encoding. Explanation complexity measures the number of minterms in logic formulas.
Dataset Concept-based Logic Explanation Ground Turth Concepts
BA-Shapes OR Node in house motif
BA-Grid Node in grid motif
Tree-Grid OR Node in grid motif
Tree-Cycle OR OR Node in circle motif
BA-Community OR OR Node in house motif
Reddit-Binary OR Star motifs
Mutagenicity Ring motifs or NO
Table 3: An example of a concept-based logic explanations discovered by the Concept Encoder Module for each dataset. Blue nodes are the instances being explained, while orange nodes represent their -hop neighbors. For Mutagenicity the color of each node represents a different chemical element. The logic formulae describe how the presence of concepts can be used to infer task labels. As an example, the first logic rule states that the task label “middle nodes in house motifs” () can be inferred from the concepts: “middle node with attaching edge on the near side” or “middle node with attaching edge on the far side”.

5.4 Interventions

The Concept Encoder Module supports human interventions (Figure 5)

Supporting human interventions is one of the main benefits of more interpretable architectures that learn tasks as a function of concepts. In contrast to vanilla GNNs, CEM enables interventions at concept-level, which allows human experts to correct mispredicted concepts. Similarly to Concept Bottleneck Models (Koh et al., 2020), our results show that correcting concept assignments significantly improves the model test accuracy to over for the synthetic datasets, achieving test accuracy on BA-Grid and BA-Shapes. We also observe an increase in task accuracy in BA-Community, however, the increase is much more gradual. Most notably, in both real-world datasets CEMs allow GNNs to improve their task accuracy by up to with less than interventions.

Figure 5: The Concept Encoder Module supports interventions at concept-level, allowing human experts to correct mispredicted concepts (left), increasing human trust in the model (Shen, 2022). This interaction significantly improves task performance, achieving almost accuracy on synthetic datasets (right).

6 Discussion

Concept Graph Networks are accurate and self-explaining

In summary, our results demonstrate that CEM makes GNNs explainable by design without impairing their task performance. Specifically, our approach extracts high-quality concept-based and logic explanations. Our experiments show that the extracted concepts are pure, meaningful and interpretable, while task-specific logic explanations are simple and accurate. We also demonstrate that CEM supports human interventions at concept level, which is one of the main advantages of explainable-by-design architectures.

Related work

Graph Neural Network Explainer (GNNExplainer,  (Ying et al., 2019)) represents the first seminal work on GNN explainability. Specifically, GNNExplainer maximizes the mutual information between GNN predictions and the distribution of possible subgraphs for explanations. By focusing on individual predictions, the method is limited to explaining one instance at a time corresponding to a localized view of the data distribution. To get a full picture, Ying et al. (2019) suggest to perform subgraph matching on a substantial number of instances. However, this is not scalable as subgraph matching is NP-Hard (Ying et al., 2019). In an attempt to alleviate this issue, the Parameterised Graph Explainer (PGExplainer, (Luo et al., 2020)) and the Probabilistic Graphical Model Explainer (PGM-Explainer, (Vu and Thai, 2020)) parametrize the process of generating explanations using deep neural networks to provide multi-instance explanations. However, all these methods remain fundamentally limited in their locality as they cannot explain a class of samples in its entirety. In contrast, GCExplainer  (Magister et al., 2021) fills the gap of global explainability for GNNs using concept-based explanations. While the existing explainability techniques begin to address the lack of insight into the computations of GNNs, they are all post-hoc methods whose goal is to explain a trained GNN, not to make it more interpretable. The proposed method instead aims at filling this knowledge gap by making GNNs explainable by design.

From a broader perspective, our work borrows ideas from supervised and unsupervised concept-based methods. These methods have been explored in various ways for other neural networks, such as convolutional neural networks 

(Alvarez-Melis and Jaakkola, 2018; Ghorbani et al., 2019b; Chen et al., 2020; Kazhdan et al., 2020a; Koh et al., 2020) and recurrent architectures (Kazhdan et al., 2020b). From supervised concept-based methods, our approach inherits the ability to perform effective human interventions at concept level extending Concept Bottleneck Models (Koh et al., 2020) to graphs. As for unsupervised methods, our approach mainly draws from the Automatic Concept Extraction algorithm (ACE, (Ghorbani et al., 2019a)). The algorithm extracts visual concepts by performing k-Means clustering (Forgy, 1965) on image segments in the activation space of a convolutional network. The ACE approach is based on the observation that the learned activation space is similar to human perceptual judgement (Zhang et al., 2018). This was the main motivation behind GCExplainer, as well as our approach. However, in contrast to ACE and GCExplainer, we embed the clustering step within the network architecture, making GNNs explainable by design.

Strengths and limitations

The main limitation of our work is the association of only one concept per sample. However, this also applies to GCExplainer, as well as to state-of-the-art unsupervised explainability methods for convolutional networks, such as ACE (Ghorbani et al., 2019b). The second main limitation pertains the visualization technique inherited from GCExplainer. Visualizing a concept by simply exploring the -hop neighborhood may include nodes which are not relevant for identifying a concept. More specifically, the concept visualization technique could be improved by performing largest common subgraph matching across the samples representing a concept. However, such an approach would be extremely expensive in terms of computations even for small graphs and it would not scale for large concepts. In terms of novelties, the proposed approach is the first of its kind in terms of making GNNs explainable by design. Secondly, the approach allows to find the optimal number of concepts dynamically. While the size of the embedding space must still be defined, this size is just an upper bound, alleviating the user from the burden of tuning this hyperparameter as in other explainability methods, such as GCExplainer or ACE. This dynamic adaptation often produces a high number of clusters/concepts. While a higher number of concepts may appear redundant and be more complex to reason about, the extracted concepts accurately describe the dataset, as indicated by the high concept completeness scores and rare concepts found. Moreover, logic-based formulas allow to filter through the concepts relevant for each class. We note, however, that as stronger interpretable models are deployed, there are risks of societal harm which we must be vigilant to avoid.

Conclusion

In this work, we address the lack of human trust in GNNs caused by their opaque reasoning. To this aim, we propose the Concept Encoder Module which makes GNNs explainable by design. We demonstrate that the proposed method allows to discover and extract high-quality concept-based explanations. The proposed approach makes GNNs explainable by design without a reduction in performance, while also allowing for human intervention. Human intervention allows to alleviate dataset biases, further increasing trust in the model. The increased understanding of the model’s working through the proposed approach fosters an increase in trust and may open up the possibility to use GNNs in more high-stake scenarios.

References

  • J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim (2018) Sanity checks for saliency maps. Advances in neural information processing systems 31. Cited by: §1.
  • D. Alvarez-Melis and T. S. Jaakkola (2018) Towards robust interpretability with self-explaining neural networks. arXiv preprint arXiv:1806.07538. Cited by: §6.
  • M. Bajaj, L. Chu, Z. Y. Xue, J. Pei, L. Wang, P. C. Lam, and Y. Zhang (2021) Robust counterfactual explanations on graph neural networks. Advances in Neural Information Processing Systems 34. Cited by: §3.
  • A. Barabási and R. Albert (1999) Emergence of scaling in random networks. science 286 (5439), pp. 509–512. Cited by: §4.
  • P. Barbiero, G. Ciravegna, F. Giannini, P. Lió, M. Gori, and S. Melacci (2021) Entropy-based logic explanations of neural networks. arXiv preprint arXiv:2106.06804. Cited by: §3.
  • P. Battaglia, R. Pascanu, M. Lai, D. Jimenez Rezende, et al. (2016) Interaction networks for learning about objects, relations and physics. Advances in neural information processing systems 29. Cited by: §1.
  • L. Breiman, J. Friedman, R. Olshen, and C. Stone (1984) Classification and regression trees. Cited by: §4.
  • Z. Chen, Y. Bei, and C. Rudin (2020) Concept whitening for interpretable image recognition. Nature Machine Intelligence 2 (12), pp. 772–782. Cited by: §6.
  • G. Ciravegna, P. Barbiero, F. Giannini, M. Gori, P. Lió, M. Maggini, and S. Melacci (2021) Logic explained networks. arXiv preprint arXiv:2108.05149. Cited by: §2, §3, §4, §5.3.
  • A. Davies, P. Veličković, L. Buesing, S. Blackwell, D. Zheng, N. Tomašev, R. Tanburn, P. Battaglia, C. Blundell, A. Juhász, et al. (2021) Advancing mathematics by guiding human intuition with ai. Nature 600 (7887), pp. 70–74. Cited by: §1.
  • M. Defferrard, X. Bresson, and P. Vandergheynst (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems 29. Cited by: Appendix D.
  • J. M. Durán and K. R. Jongsma (2021) Who is afraid of black box algorithms? on the epistemological and ethical basis of trust in medical ai. Journal of Medical Ethics 47 (5), pp. 329–335. Cited by: §1.
  • EUGDPR (2017) GDPR. general data protection regulation.. Cited by: §1.
  • M. Fey and J. E. Lenssen (2019)

    Fast graph representation learning with PyTorch Geometric

    .
    In ICLR Workshop on Representation Learning on Graphs and Manifolds, Cited by: Appendix A, Appendix C.
  • E. W. Forgy (1965) Cluster analysis of multivariate data: efficiency versus interpretability of classifications. biometrics 21, pp. 768–769. Cited by: §2, §6.
  • A. Ghorbani, A. Abid, and J. Zou (2019a) Interpretation of neural networks is fragile. In

    Proceedings of the AAAI conference on artificial intelligence

    ,
    Vol. 33, pp. 3681–3688. Cited by: §1, §2, §6.
  • A. Ghorbani, J. Wexler, J. Zou, and B. Kim (2019b) Towards automatic concept-based explanations. arXiv preprint arXiv:1902.03129. Cited by: §1, §2, §6, §6.
  • J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl (2017) Neural message passing for quantum chemistry. In International conference on machine learning, pp. 1263–1272. Cited by: §2.
  • D. Kazhdan, B. Dimanov, M. Jamnik, P. Liò, and A. Weller (2020a) Now you see me (cme): concept-based model extraction. arXiv preprint arXiv:2010.13233. Cited by: §6.
  • D. Kazhdan, B. Dimanov, M. Jamnik, and P. Liò (2020b) MEME: generating rnn model explanations via model extraction. arXiv preprint arXiv:2012.06954. Cited by: §6.
  • B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas, et al. (2018)

    Interpretability beyond feature attribution: quantitative testing with concept activation vectors (tcav)

    .
    In International conference on machine learning, pp. 2668–2677. Cited by: §1.
  • P. Kindermans, S. Hooker, J. Adebayo, M. Alber, K. T. Schütt, S. Dähne, D. Erhan, and B. Kim (2019) The (un) reliability of saliency methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, pp. 267–280. Cited by: §1.
  • T. N. Kipf and M. Welling (2016) Semi-Supervised Classification with Graph Convolutional Networks. In 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings, pp. 11313–11320. External Links: 1609.02907, Link Cited by: Appendix A.
  • P. W. Koh, T. Nguyen, Y. S. Tang, S. Mussmann, E. Pierson, B. Kim, and P. Liang (2020) Concept bottleneck models. In International Conference on Machine Learning, pp. 5338–5348. Cited by: §1, §2, §3, §5.4, §6.
  • S. Lo Piano (2020) Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward. Humanities and Social Sciences Communications 7 (1), pp. 1–7. Cited by: §1.
  • A. Lucic, M. A. Ter Hoeve, G. Tolomei, M. De Rijke, and F. Silvestri (2022) Cf-gnnexplainer: counterfactual explanations for graph neural networks. In International Conference on Artificial Intelligence and Statistics, pp. 4499–4511. Cited by: §3.
  • D. Luo, W. Cheng, D. Xu, W. Yu, B. Zong, H. Chen, and X. Zhang (2020) Parameterized explainer for graph neural network. Advances in neural information processing systems 33, pp. 19620–19631. Cited by: §1, §3, §4, §4, §6.
  • L. C. Magister, D. Kazhdan, V. Singh, and P. Liò (2021) GCExplainer: human-in-the-loop concept-based explanations for graph neural networks. arXiv preprint arXiv:2107.11889. Cited by: Appendix A, Appendix A, Appendix A, Table 4, Appendix C, Figure 18, §1, §2, §3, §4, §4, §4, Figure 3, §6.
  • G. A. Miller (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological review 63, pp. 81–97. Cited by: §3.
  • C. Morris, N. M. Kriege, F. Bause, K. Kersting, P. Mutzel, and M. Neumann (2020) Tudataset: a collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663. Cited by: §4.
  • A. Pal, C. Eksombatchai, Y. Zhou, B. Zhao, C. Rosenberg, and J. Leskovec (2020) Pinnersage: multi-modal user embedding framework for recommendations at pinterest. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2311–2320. Cited by: Appendix D, §1.
  • A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. (2019) Pytorch: an imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703. Cited by: Appendix A.
  • F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al. (2011) Scikit-learn: machine learning in python. the Journal of machine Learning research 12, pp. 2825–2830. Cited by: Appendix A.
  • C. Rudin (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1 (5), pp. 206–215. Cited by: §1, §1, §3.
  • F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini (2008) The graph neural network model. IEEE transactions on neural networks 20 (1), pp. 61–80. Cited by: §1, §2.
  • M. W. Shen (2022) Trust in ai: interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient. arXiv preprint arXiv:2202.05302. Cited by: §1, §2, Figure 5.
  • J. M. Stokes, K. Yang, K. Swanson, W. Jin, A. Cubillos-Ruiz, N. M. Donghia, C. R. MacNair, S. French, L. A. Carfrae, Z. Bloom-Ackermann, et al. (2020) A deep learning approach to antibiotic discovery. Cell 180 (4), pp. 688–702. Cited by: §1.
  • M. Vu and M. T. Thai (2020) Pgm-explainer: probabilistic graphical model explanations for graph neural networks. Advances in neural information processing systems 33, pp. 12225–12235. Cited by: §1, §3, §4, §4, §6.
  • S. Wachter, B. Mittelstadt, and C. Russell (2017) Counterfactual explanations without opening the black box: automated decisions and the gdpr. Harv. JL & Tech. 31, pp. 841. Cited by: §1.
  • M. Y. Wang (2019) Deep graph library: towards efficient and scalable deep learning on graphs. In ICLR workshop on representation learning on graphs and manifolds, Cited by: Appendix C.
  • K. Xu, W. Hu, J. Leskovec, and S. Jegelka (2019) How powerful are graph neural networks?. In International Conference on Learning Representations, External Links: Link Cited by: Appendix D.
  • C. Yeh, B. Kim, S. Arik, C. Li, T. Pfister, and P. Ravikumar (2020) On completeness-aware concept-based explanations in deep neural networks. Advances in Neural Information Processing Systems 33, pp. 20554–20565. Cited by: Appendix A, §1, §2, §4.
  • Z. Ying, D. Bourgeois, J. You, M. Zitnik, and J. Leskovec (2019) Gnnexplainer: generating explanations for graph neural networks. Advances in neural information processing systems 32. Cited by: Appendix C, Appendix C, §1, §1, §3, §4, §4, §5.2, §6.
  • R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang (2018)

    The unreasonable effectiveness of deep features as a perceptual metric

    .
    In

    Proceedings of the IEEE conference on computer vision and pattern recognition

    ,
    pp. 586–595. Cited by: §2, §6.
  • J. Zhou, A. H. Gandomi, F. Chen, and A. Holzinger (2021) Evaluating the quality of machine learning explanations: a survey on methods and metrics. Electronics 10 (5), pp. 593. Cited by: §4.

Appendix A Experimental setup

Model training and evaluation

For each dataset, we train two types of models: a CGN, a model implementing the proposed approach, and a vanilla GNN. Both models have the same number of graph convolutional layers [Kipf and Welling, 2016], where we base the number of layers on those reported by  Magister et al. [2021]

for GCExplainer. In the CGN, these layers are then followed by the CEM. In lieu of the CEM, the vanilla GNN has a linear layer and a log softmax activation function to perform the read out. We perform a grid search on the CGN for each dataset to determine the number of hidden units, as well as the learning rate. The number of hidden units tested for the CGNs for the synthetic datasets are 10, 20 and 30, while we search over 20, 30 and 40 hidden units for the real-world datasets, as they are more complex. The different learning rates tested are 0.1, 0.001 and 0.0001. In the more complex datasets Mutagenicity and Reddit-Binary, we also search over the concept encoding size to encourage fewer clusters. The concept encoding size is the number of hidden units in the final graph convolutional layer. We vary this parameter rather than using the same number of hidden units throughout, as we observed a high number of very small clusters formed for those datasets. This can be attributed to the real-world datasets having a less regular structure than the synthetic ones. We determine the best hyperparameter combination based on model convergence and test accuracy. Table

4 summarises the model architecture and hyperparameters defined for each dataset.

Dataset
Number of
Graph Convolutions
Number of
Hidden Units
Concept
Encoding Size
Learning
Rate
Batch
Size
Number of
Epochs
BA-Shapes 4 10 10 0.001 N/A 7000
BA-Grid 4 10 10 0.001 N/A 3000
Tree-Grid 7 20 20 0.0001 N/A 20000
Tree-Cycle 3 10 10 0.001 N/A 7000
BA-Community 6 20 20 0.0001 N/A 10000
Mutagenicity 4 40 10 0.001 16 1000
Reddit-Binary 4 40 10 0.001 16 1000
Table 4: A summary of the model architecture and training hyperparameters for each dataset. The number of hidden units and learning rate where determined via a grid search, while the number of graph convolutional layers is based on the models reported for GCExplainer [Magister et al., 2021]. These parameters are kept the same between the Concept Graph Network and vanilla graph neural network to allow comparison.

In regard to the initialization of GCExplainer, we carry over the recommended values for in k-Means proposed by  Magister et al. [2021]. The recommended values for in BA-Shapes, BA-Grid, Tree-Grid and Tree-Cycle are , while for BA-Community, Mutagenicity and Reddit-Binary.  Magister et al. [2021] determine these experimentally by performing a grid search and comparing the concept completeness and purity scores for different initalizations of . We also carry over the initialization of for visualizing the -hop neighborhood, recommend by  Magister et al. [2021]. For BA-Shapes and BA-Community, to cover the ‘house-motif‘. For the remaining datasets, to visualise the encoded motifs.

For each dataset, we train and evaluate each of the models five times in order to be able to state confidence bounds. In order to allow reproducing the results, we fix the random seed for a run, using a different seed for each run. The randomly generated seeds for the five runs are: 42, 19, 76, 58 and 92.

Metrics

Here, we will detail how we compute the five metrics for evaluating task accuracy, task completeness, concept interpretability and explanation performance. The first metric is classification accuracy, which we compute by dividing the number of correctly classified samples by the total number of samples. Secondly, we calculate concept completeness [Yeh et al., 2020], which is the accuracy of a decision tree classifier trained to predict the output label from the concept encoding in the case of CGNs and cluster number in the case of GCExplainer. Thirdly, we compute concept purity. We follow the approach proposed by  Magister et al. [2021], which is to calculate the graph edit distance between the top representation of a concept and the two closest concept representations to it. We compute the minimum purity score, because we want to compare the best concepts found by CEM and GCExplainer. As the graph edit distance is expensive to compute, only smaller, meaningful graphs are considered. Table 5 summarises and justifies the graph sizes considered. We aim for the maximum graph size to be twice the size of the concept, however, we cap this threshold at 13 if the concept is too big. This is reasonable, as in the synthetic datasets noisy concepts representing the base graph are identified.

Dataset
Graph Sizes considered
in Purity Score
Comment
BA-Shapes 10 Twice the size of the desired motif.
BA-Grid 13 Capped due to computation cost.
Tree-Grid 13 Capped due to computation cost.
Tree-Cycle 12 Twice the size of the desired motif.
BA-Community 10 Twice the size of the desired motif.
Mutagenicity 13 Capped due to computation cost.
Reddit-Binary 13 Capped due to computation cost.
Table 5: A summary of the maximum graph size considered in the computation of the purity score.

Similar to the completeness score, we compute the accuracy of logic explanations by taking the accuracy of a decision tree classifier trained on mapping between concepts activated according to logic formula and the output label. Finally, we compute the complexity of logic explanations by measuring the length of the logic formula. The metrics reported are the mean of the five experiment runs for each dataset, including the confidence intervals computed using the Box-Cox transformation for non-normal distributions.

Implementation details

The code for the experiments is implemented in Python 3, relying upon open-source libraries 

[Pedregosa et al., 2011, Paszke et al., 2019, Fey and Lenssen, 2019]. Implementation can be found in the public repository anonymous using Apache v2.0 free and open source licence. All the experiments have been run on the same machine: Intel® Core™ i7-10750H 6-Core Processor at 2.60 GHz equipped with 16 GiB RAM and NVIDIA GeForce RTX 2060 GPU.

Appendix B Further concept visualisations

In this section we present further concept visualisation results for the datasets. As an exhaustive number of concepts is identified for more complex datasets, such as BA-Community, Mutagenicity and Reddit-Binary, we only present a selection of the concepts discovered.

BA-Shapes (Figure 6

The selection of concepts presented illustrates that a variety of concepts is found. Concepts 1 and 5 show the middle node of the house structure. In contrast, the fourth concept visualised shows a rare concept. Focusing on the first concept representation of the concept shown, it is evident that the concept visualised is a house structure attached via two edges, of which one is a random edge. The size of the corresponding cluster is only one, which explains why the four other concept representations do not match. This draws the user’s attention to the significance of the rare motif.

Figure 6: A selection of five concepts found for the BA-Shapes dataset, each represented by the top five concept representations. Blue nodes are the nodes clustered, while the orange nodes make up the -hop neighborhood. The greyed out area shows that the cluster size for the concept is only 1. This means that only the first concept is representative. In contrast to the other concepts visualised, it becomes evident that this is a rare concept of a top of the house node being connected to the base graph via a random edge.

BA-Grid (Figure 7)

We visualise a selection of the concepts found for BA-Grid in the same manner. In reference to concepts 1-3, it can be stated that the grid motif is successfully recovered. Moreover, the invariance of the graph is highlighted in concept 2, as we see two blue nodes per concept representation. Nodes on either side of the grid being grouped together shows that the GNN successfully classifies them as similar due to the neighbourhood having the same structure. In contrast, concepts 4 and 5 represent the BA base graph, which can be described as highly connected.

Figure 7: A selection of five concepts found for the BA-Grid dataset. Each concept is represented by the top five concept representations, the nodes closest to the center of a cluster. Blue nodes are the nodes clustered, while the orange nodes make up the -hop neighborhood.

Tree-Grid (Figure 8)

Similarly to BA-Grid, we successfully recover the grid concept, as shown in concepts 3-5. Moreover, concepts describing the more structured base graph are found. Concepts 1 and 2 clearly depict the binary tree structure, which forms the base graph, as well as parts of grid structures.

Figure 8: A selection of five concepts found for the Tree-Grid dataset, each represented by the top five concept representations. Blue nodes are the nodes clustered, while the orange nodes make up the -hop neighborhood.

Tree-Cycle (Figure 9)

We obtain similar results for the Tree-Cycle dataset. The subset of concepts visualised in Figure 9 shows that the cycle structure is successfully recovered in concepts 1-3. Concept 5 also represents the cycle structure, however, including random edges. Lastly, concept 4 encapsulates the binary-tree base graph of the dataset.

Figure 9: A selection of five concepts found for the Tree-Cycle dataset, each represented by the top five concept representations. Blue nodes are the nodes clustered, while the orange nodes make up the -hop neighborhood.

BA-Community (Figure 10)

Figure 10 shows concept representations of the concepts discovered for the BA-Community dataset. Concepts 1-3 successfully recover the house structure. However, concept 2 highlights the existence of random edges, creating rare structures. Finally, concepts 4 and 5 represent the highly connective nature of the BA base graph. Comparing concepts 1 and 2, it is evident that features play an important role and that feature importance is highlighted by CEM. Concept 1 and 2 both highlight the existence of the middle node attaching directly to the BA graph, however, reviewing the class labels, it is evident that house shapes belonging to different communities are clustered separated from one another. This indicates that the concept encoding differentiates between similar neighborhoods of the same structure based on feature importance.

Figure 10: A selection of five concepts found for the BA-Community dataset, each represented by the top five concept representations. Blue nodes are the nodes clustered, while the orange nodes make up the -hop neighborhood.

Mutagenicity (Figure 11)

Figure 11 visualises a subset of concepts discovered for the Mutagenicity dataset. We can clearly identify the concept of the ring structure in concepts 1 and 3, while concepts 2, 4 and 5 exhibit a more varied structure, lacking the ring motif. Examining concept 3 more closely, it becomes evident that the ring structure does not uniquely identify a molecule as mutagenic. The concept depicts molecules which all have a ring structure, despite some being mutagenic and some not mutagenic, as indicated by the class label. Examining concept 2 in conjunction with the cluster size, it becomes evident that a rare concept is depicted. The size of the cluster is only one, which means that the greyed-out representations do not belong to this concept. Due to the lack of expert knowledge and ground truths, it is more tricky to identify the meaning of this rare concept, however, it can be stated that the second closest representation to the cluster centroid still exhibits the ring structure.

Figure 11: A selection of five concepts found for the Mutagenicity dataset, each represented by the top five concept representations. The nodes are colored by the node feature, which is the atom label. The greyed out area highlights a rare motif, as the cluster size is only one and there is a stark contrast to the other closest concept representations discovered.

Reddit-Binary (Figure 12)

Finally, Figure 12 visualizes a subset of the concepts found for Reddit-Binary. We can see that multiple concepts exhibiting the star-like structure are discovered, with the difference lying in the density of the connections. Examining concept 5 more closely, it can be stated that a rare concept is discovered, as there are only 3 examples of the concept. This may be attributed to the nodes not only being connected to the center node, marked by node 45, but also to other nodes, showing activity in other threads. The remaining two concepts visualised are the closest embeddings to the cluster centroid but are part of another concept.

Figure 12: A selection of five concepts found for the Reddit-Binary dataset, each represented by the top five concept representations. Blue nodes are the nodes clustered, while the orange nodes make up the -hop neighborhood. The greyed-out area highlights that the concept only comprises 3 examples and can be identified as a rare motif.

Appendix C Comparison to GCExplainer and GNNExplainer

We perform a qualitative comparison of the explanations produced using the CEM, GCExplainer [Magister et al., 2021] and GNNExplainer [Ying et al., 2019]. We limit ourselves to a qualitative comparison against GCExplainer here, as we have already performed a quantitative comparison in the main section of the paper. We do not perform a quantitative evaluation against GNNExplainer, as the explanations are not concept-based and thus are evaluated in a different manner. GCExplainer and GNNExplainer are applied on the vanilla GNN. We focus on an evaluation of the BA-Shapes, BA-Grid and BA-Community datasets, as the ground truth motifs to be extracted are known for these dataasets.

BA-Shapes (Figure 13)

Figure 13 shows the explanations provides by CEM, GCExplainer and GNNExplainer for a node, which is part of the middle of a house. Both CEM and GCExplainer successfully identify the house structure. CEM performs slightly better than GCExplainer, as it does not include a concept with a random edge. In contrast, the explanation provided by GCExplainer does not visualise the house structure in full. Only the middle nodes of the house are visualised (purple), as well as a large part of the BA graph (turquoise). It can be argued that the explanations provided by CEM and GCExplainer are more intuitive, however, GNNExplainer highlights important edges.

Figure 13: Concept-based explanations produced using the Concept Encoder Module and GCExplainer, as well as the explanation produced by GNNExplainer for a node in the BA-Shapes dataset. In the explanations, the blue nodes are the nodes clustered together, while the orange nodes are the -hop neighborhood. GNNExplainer has its own coloring, where the purple nodes are node part of the middle of the house and the turquoise nodes are part of the BA base graph.

We struggle to reproduce the quality of explanations presented by  Ying et al. [2019] for GNNExplainer. We first adapted the threshold to observe an effect on the explanations, however, this only impacted the visualisation of the important edges. We fix the threshold at 0.8 after this. We then examined the implementation of GNNExplainer used. To ensure that the quality of explanations is not the fault of the PyTorch Geometric [Fey and Lenssen, 2019] implementation of GNNExplainer, we also used the implementation provided by the Deep Graph Library [Wang, 2019]. After obtaining similar results, we exhaustively visualize the explanations for class 1. We present a selection in Figure 14. In summary, we fail to produce the house motif using GNNExplainer, as the explanations provided mostly emphasize the importance of the BA base graph.

Figure 14: A selection of explanations produced using GNNExplainer for nodes of class 1. The purple nodes are nodes part of the middle of the house, while the turquoise ones are part of the BA base graph.

BA-Grid (Figure 15)

Figure 15 visualizes explanations produced for a grid node in the BA-Grid dataset. Based on the concept representations, it can be argued that CEM and GCExplainer produce explanations of similar quality, as they both successfully extract the grid structure. This is also supported by the comparable concept completeness scores achieved (Figure 2), however, CEM produces more pure results (Figure 3). In respect to GNNExplainer, our previous claim of more intutive explanations is supported, as it is harder to reason about the explanation provided. It can be inferred that the immediate neighbours part of the grid structure (purple) are the most important for prediction, as well as part of the BA graph (turquoise).

Figure 15: Concept-based explanations produced using the Concept Encoder Module and GCExplainer, as well as the explanation produced by GNNExplainer for a node in the BA-Grid dataset. In the explanations, the blue nodes are the nodes clustered together, while the orange nodes are the -hop neighborhood. GNNExplainer has its own coloring, where the purple nodes are part of the grid and the turquoise nodes are part of the BA base graph.

BA-Community (Figure 16)

Lastly, we compare the explanations for a node in the BA-Community dataset (Figure 16). Similar to our previous observations, both CEM and GCExplainer successfully identify the house structure. More importantly, they both identify the existence of random edges to explain the node. In contrast, the explanation provided by GNNExplainer is more elusive, highlighting mostly the BA base structure. In conclusion, it can be stated that the concept representations for CEM and GCExplainer are almost identical, which can be attributed to the same visualisation technique being used. However, we refer the read back to the quantative evaluation in the results section, which highlights the strengths of CEM over GCExplainer.

Figure 16: Concept-based explanations produced using the Concept Encoder Module and GCExplainer, as well as the explanation produced by GNNExplainer for a node in the BA-Community dataset. In the explanations, the blue nodes are the nodes clustered together, while the orange nodes are the -hop neighborhood. GNNExplainer has its own coloring, where the purple node is the ’top’ node in the house, the blue nodes are the ’middle’ of the house and the turquoise nodes are part of the BA base graph.

Appendix D Evaluation on further GNN architectures

To demonstrate that CEM can be applied to any GNN architecture, we perform a quantative analysis of the performance of CEM using different convolutional embedding layers. We keep the experimental setup the same and only vary the type of graph convolutional layer. We trial the chebyshev spectral graph convolution (Cheb,  [Defferrard et al., 2016]), graph isomorphism Network layer (GIN,  [Xu et al., 2019]) and Graph SAmple and AGgregate layer (SAGE, [Pal et al., 2020]), as they are prominent graph convolutions, covering both spectral and spatial approaches. We showcase the results for the BA-Shapes and BA-Community dataset.

d.1 Task Accuracy and Completeness

Concept Graph Networks are as accurate as vanilla GNNs (Figure 17, left)

Our results confirm our previous observations. Across the three different layers, CEM allows to achieve better or comparable task accuracy w.r.t. equivalent GNN architectures. For example, the CEM-based model using GIN convolutions performs better on the BA-Community dataset. Only one CEM-based model performs worse than its respective vanilla GNN counterpart. That is the model trained on BA-Community. In general task accuracy is low, which can be attributed to the convolutional operation being ill-suited to the task.

The Concept Encoder Module discovers complete concepts (Figure 17, right)

Our experiments confirm that CEM discovers a more complete set of concepts w.r.t. the concepts extracted by GCExplainer applied to the respective vanilla GNN. Most notably, CEM achieves significantly higher concept completeness scores on all runs, except for the Cheb-based model trained on BA-Community. However, the difference between the completeness score for concepts extracted using CEM and GCExplainer is not significant, despite a larger difference in model performance. This further confirms our conclusion that CEM discovers a more complete set of concepts than GCExplainer.

Figure 17: Model accuracy and concept completeness for the Concept-based Graph Network (CGN) and an equivalent vanilla GNN using different graph convolutional layers. For these results, and those that follow, we compute all metrics on test sets across five seeds and report their mean and confidence intervals.

d.2 Explanation performance

The Concept encoder Module identifies pure concepts (Figure 18)

We further demonstrate that CEM discovers high-quality concepts, which are coherent across samples. This is indicated by the low concept purity scores, where 0 is a perfect score. CEM outperforms GCExplainer applied on a vanilla GNN across all architectures on the BA-Shapes dataset. In contrast, it performs slightly worse on the BA-Community dataset, however, the variance of the purity score between the proposed approach and GCExplainer is not significant, with the exception for the model using the GIN convolution.

The Concept Encoder Module provides accurate logic explanations (Figure 19, left)

Our results support our previous statement that CEM provides accurate logic explanations, as the logic explanation accuracy ranges at model accuracy. The low accuracy attained by the Cheb-based model trained on the BA-Community dataset can be explained by the poor model accuracy. The drop in the overall logic explanation accuracy in comparison to the results reported earlier can be lead back to model performance. Moreover, it should be highlighted that logic-based explanations are a benefit of CEM over GCExplainer.

Figure 18: Purity scores for the concept extracted by Concept Graph Module and GCExplainer. Notice how the optimal purity score is zero, as it measures the graph edit distance between concept instances [Magister et al., 2021].
Figure 19: Accuracy and complexity of logic explanations provided by the proposed Concept Graph Module. The accuracy is computed using logic formulas to classify samples based on their concept encoding. Explanation complexity measures the number of minterms in logic formulas.