The question how neural activity gives rise to cognition is arguably one of the most interesting problems in neuroimaging [16, 42, 1]. Neuroimaging studies per se, however, only provide insights into neural correlates but not into neural causes of cognition [42, 33]. Nevertheless, causal terminology is often introduced in the interpretation of neuroimaging data. For instance, Hamann writes in a review on the neural mechanisms of emotional memory that “Hippocampal activity in this study was correlated with amygdala activity, supporting the view that the amygdala enhances explicit memory by modulating activity in the hippocampus” , and Myers et al. note in a study on working memory that “we tested […] whether pre-stimulus alpha oscillations measured with electroencephalography (EEG) influence the encoding of items into working memory”  (our emphasis of causal terminology). The apparent contradiction between the prevalent use of causal terminology and the correlational nature of neuroimaging studies gives rise to the following question: which causal statements are and which ones are not supported by empirical evidence?
We argue that the answer to this question depends on the experimental setting and on the type of model used in the analysis of neuroimaging data. Neuroimaging distinguishes between encoding and decoding models 
, known in machine learning as generative and discriminative models. Encoding models predict brain states, e. g. BOLD activity measured by fMRI or event-related potentials measured by EEG/MEG, from experimental conditions [10, 11, 6]25, 31]. Several recent publications have addressed the interpretation of encoding and decoding models in neuroimaging, discussing topics such as potential confounds [40, 44], the dimensionality of the neural code , and the relation of linear encoding and decoding models . We contribute to this discussion by investigating, for each type of model, which causal statements are warranted and which ones are not supported by empirical evidence. Our investigation is based on the seminal work by Pearl  and Spirtes et al.  on causal inference (cf. [32, 15, 41, 26] for applications of this framework in neuroimaging). We find that the distinction between encoding and decoding models is not sufficient for this investigation. It is further necessary to consider whether models work in causal or anti-causal direction, i. e. whether they model the effect of a cause or the cause of an effect . To accommodate this distinction, we distinguish between stimulus- and response-based paradigms. We then provide causal interpretation rules for each combination of experimental setting (stimulus- or response-based) and model type (encoding or decoding). We find that when considering one model at a time, only encoding models in stimulus-based experimental paradigms support unambiguous causal statements. Also, we demonstrate that by comparing encoding and decoding models trained on the same data, experimentally testable conditions can be identified that provide further insights into causal structure. These results enable us to reinterpret previous work on the relation of encoding and decoding models in a causal framework [40, 44, 17].
The empirical relevance of our theoretical results is illustrated on EEG data recorded during a visuo-motor learning task. We demonstrate that an encoding model allows us to determine EEG features that are effects of the instruction to rest or to plan a reaching movement, but does not enable us to distinguish between direct and indirect effects. By comparing relevant features in an encoding and a decoding model, we provide empirical evidence that sensorimotor - and/or occipital -rhythms (8–14 Hz) are direct effects, while brain rhythms in higher cortical areas, including precuneus and anterior cingulate cortex, respond to the instruction to plan a reaching movement only as a result of the modulation by other cortical processes.
We note that while we have chosen to illustrate the empirical significance of our results on neuroimaging data, and specifically on EEG recordings, the provided causal interpretation rules apply to any encoding and decoding model trained on experimental data. This provides a guideline to researchers on how (not) to interpret encoding and decoding models when investigating the neural basis of cognition. A preliminary version of this work has been presented in .
We begin this section by introducing the causal framework by Pearl  and Spirtes et al.  that our work is based on (section 2.1) and demonstrate how it leads to testable predictions for the causal statements cited in the introduction (section 2.2). We then introduce the distinction between causal and anti-causal encoding and decoding models (section 2.3) and establish a connection between these models and causal inference (section 2.4). This connection enables us to present the causal interpretation rules for encoding and decoding models in section 2.5. In section 2.6, we show that combining an encoding and a decoding model trained on the same data can provide further insights into causal structure. We conclude this section by providing a reinterpretation of previous work on encoding and decoding models in a causal framework (section 2.7).
2.1 Causal Bayesian Networks
By we denote the finite set of random variables representing the brain state features, i. e. . While these variables may correspond to any type of independent and identically distributed (iid) samples of
brain state features, it is helpful to consider bandpower features of different EEG channels, trial-averaged BOLD activity at various cortical locations, or mean spike rates of multiple neurons as possible examples. Bywe denote the random variable representing the (usually discrete) experimental condition. stands for a stimulus () or response () variable and it will be made clear when is restricted to either particular case. For convenience, we denote the set of all random variables by
. Throughout this article, we denote marginal, conditional and joint distributions by, and , respectively (overloading the notation of ). For our theoretical investigations, we assume that the involved distributions have probability mass or density functions (PMFs or PDFs) with values denoted by and respectively, again overloading the notation of while it is always clear from the argument which function is meant. We use the common notations for independence and conditional independence
In the framework of Causal Bayesian Networks (CBNs)[30, 38], a variable is said to be a cause of another variable if the distributions are sensitive to (cf. , p. 24f.). In this notation, the intervention signifies that is externally set to a constant , possibly resulting in a change of the distribution of . The framework of CBNs thus defines cause-effect relations in terms of the impact of external manipulations. This is in contrast to frameworks that define causality in terms of information transfer [13, 35, 22].
Causal relations between variables in CBNs are represented by directed acyclic graphs (DAGs). If we find a directed edge , we call a direct cause of and a direct effect of . In case there is no directed edge but at least one directed path , we call an indirect cause of and an indirect effect of . Note that the terms (in-)direct cause/effect depend on the set of observed variables: consider and the causal DAG . Then and wrt. , while wrt. . That is, whether a cause or effect is direct or indirect depends on the set of observed brain state features. We omit the considered set of nodes if it is clear from the context.
To establish a link between conditional independences and DAGs the following concepts are required:
Disjoint sets of nodes and are d-separated by another disjoint set of nodes if and only if all and are d-separated by . Two nodes are d-separated by if and only if every path between and is blocked by . A path between nodes and is blocked by if and only if there is an intermediate node on the path such that (i) and is a tail-to-tail () or head-to-tail () or (ii) is head-to-head () and neither nor any of its descendants is in .
- Causal Markov Condition (CMC)
The CMC expresses the notion that each node in a causal DAG becomes independent of its non-descendants given its direct causes, i. e. that the causal structure implies certain (conditional) independences.
The faithfulness assumption states that all (conditional) independences between the random variables of a DAG are implied by its causal structure, i. e. there are no more (conditional) independences than those implied by the CMC.
Assuming faithfulness and the causal Markov condition, d-separation is equivalent to conditional independence, i. e. d-separates and if and only if and are independent given . The following three examples are the most relevant instances of d-separation for our following arguments. Firstly, consider the chain . Here, d-separates and by blocking the directed path from to . This implies that . Secondly, consider the fork . Here, d-separates and , as is a joint cause of and . This again implies that . Thirdly, consider the collider . In this case, does not d-separate and . As is a joint effect of and , it unblocks the previously blocked path between and , implying that .
The equivalence between d-separation and conditional independence enables us to infer causal relations between variables in from observational data. By identifying conditional independences that hold in our data, and mapping them onto the equivalent d-separations, we gain knowledge about the causal structures that can give rise to our data. This link forms the basis of the inference rules we describe in section 2.5 and 2.6. We refer the interested reader to  for a more exhaustive introduction to this causal inference framework in the context of neuroimaging.
2.2 Causal terminology and CBNs
We now demonstrate how the causal statements, that we cited in the introduction to motivate our work, can be expressed in the framework of CBNs.
Firstly, consider the statement “Hippocampal activity in this study was correlated with amygdala activity, supporting the view that the amygdala enhances explicit memory by modulating activity in the hippocampus” . Here, it is implicitly assumed that hippocampal activity is a cause of explicit memory; that is, manipulating hippocampal activity results in measurable changes in explicit memory. In the notation of CBNs, this is expressed as hippocampal activity explicit memory. Further, it is implied that the amygdala enhances explicit memory via modulating activity in the hippocampus; that is, manipulating activity in the amygdala leads to measurable changes in hippocampal activity, which results in changes in explicit memory. This gives the causal hypothesis amygdala activity hippocampal activity explicit memory. Assuming faithfulness and the CMC, this causal hypothesis makes the empirically testable predictions amygdala activity explicit memory and amygdala activity explicit memory hippocampal activity.
Secondly, consider the statement “we tested […] whether pre-stimulus alpha oscillations measured with electroencephalography (EEG) influence the encoding of items into working memory” . The authors thereby express the notion that pre-stimulus alpha oscillations are a cause of working memory; that is, manipulating pre-stimulus alpha oscillations has a measurable effect on working memory. This gives the causal hypothesis pre-stimulus alpha oscillations working memory. Again assuming faithfulness and the CMC, this causal hypothesis makes the empirically testable prediction pre-stimulus alpha oscillations working memory.
In the following, we investigate how these empirical predictions can be tested by encoding and decoding models, and under which conditions these predictions are sufficient to prove a causal hypothesis. In section 2.7.3 we revisit those two examples illustrating our theoretical results.
2.3 Causal and anti-causal encoding and decoding models
An encoding model represents how various experimental conditions are encoded in the brain state feature vector. We ask “How does the brain state look like given a certain experimental condition?”. Examples for encoding models are the general linear model  or the class-conditional mean .
A decoding model represents how experimental conditions can be inferred from the brain state feature vector as the independent and as the dependent variable.
A priori, encoding and decoding models are oblivious to the causal relation between experimental conditions and brain state features, i. e. they disregard whether they model the effect of a cause or the cause of an effect . As we show in section 2.4, however, this has implications for their interpretation. We hence introduce the distinction between stimulus- and response-based experimental paradigms.
We categorize an experimental setup as stimulus-based, if the experimental conditions precede the measured brain states. An example of a stimulus-based setup is the investigation of the brain’s activity when exposed to auditory stimuli. As the auditory stimuli precede the measured brain activity, the brain state features cannot be a cause of the stimuli. In the stimulus-based setting, an encoding model works in causal direction, as it models the effect of a cause. A decoding model works in anti-causal direction, as it models the cause of an effect. We note that in the stimulus-based setting we can control for and randomize the stimulus, i. e. we can externally set the stimulus to a desired value denoted by .
We categorize an experimental setup as response-based, if the measured brain states precede the experimental conditions. An example of a response-based setup is the prediction of the laterality of a movement from pre-movement brain state features. As the measured brain state features precede the actual movement, the movement cannot be a cause of this brain activity. In the response-based setting, an encoding model works in anti-causal direction, as it models the cause of an effect. A decoding model works in causal direction, as it models the effect of a cause. We note that in contrast to the stimulus-based setting, we cannot control for and randomize the response, i. e. we cannot set the response to a desired value by an external intervention.
We also note that more complex experimental paradigms, in which responses act again as stimuli , can also be categorized in this way by considering time-resolved variables, e. g. stimulus brain activity response stimulus.
In the following, we hence distinguish between four types of models:
Causal encoding models –
Anti-causal decoding models –
Anti-causal encoding models –
Causal decoding models –
2.4 Relating encoding and decoding models to causal inference
In this section, we establish a link between causal inference and the identification of relevant features in encoding and decoding models. This link forms the basis for the causal interpretation rules in the next section.
When using an encoding model to analyze neuroimaging data, we wish to identify features that show a statistically significant variation across experimental conditions. In practice, this can be carried out by a variety of methods including but not limited to a general linear model , class-conditional differences in mean activation , and non-linear independence tests [14, 15]
. Common to all these approaches is that they admit univariate statistical tests to quantify the likelihood of the data under the null-hypothesis. Features, for which the null-hypothesis of independence is rejected, are considered relevant for the encoding model in this experimental paradigm. Features with insufficient evidence for rejection of the null-hypothesis are considered irrelevant for the present encoding model. The set of relevant and irrelevant features in an encoding model is subsequently denoted as and , respectively. Assuming faithfulness, the relevance of features in an encoding model can be translated into d-separation properties that provide insights into the causal structure of the data-generating process. In particular, features in are d-separated from the experimental condition by the empty set while features in are not d-separated from the experimental condition by the empty set.
When using a decoding model to analyze neuroimaging data, we wish to identify features that help in decoding the experimental condition; that is, we wish to determine for each feature if its removal increases the minimum Bayes error. If
the Bayes classifierand hence also the Bayes error rate remain unchanged. This leads to the mathematical concept of conditional independence for a feature’s relevance in decoding : a feature is relevant for decoding if . In practice, one commonly used approach is to permute a feature with respect to the experimental conditions and check if this results in a statistically significant decrease in decoding accuracy . Features, for which the null-hypothesis of conditional independence is rejected, are considered relevant for the decoding model. Features with insufficient evidence for rejection of the null-hypothesis are considered irrelevant for decoding the experimental condition given the other features. The set of relevant and irrelevant features in a decoding model is subsequently denoted as and , respectively. Assuming faithfulness again, every feature is d-separated from the experimental condition by the set , while every feature is not d-separated from the experimental condition by the set .
The link between the relevance of features in encoding and decoding models and d-separation properties allows us to provide the causal interpretation rules given in the next section. For our theoretical arguments, we assume that we can identify all relevant features for each type of model, that is, we assume we can identify all (conditional) (in-)dependence relations. We discuss the practical intricacies of identifying independence relations on finite empirical data in section 4.
2.5 Causal interpretation rules for encoding and decoding models
2.5.1 Causal encoding models
According to Reichenbach’s principle , the dependency between and implies that , , or with a joint common cause of and . In the stimulus-based setting, we can control for and randomize the stimulus. This enables us to rule out the last two cases and conclude that , i. e. the features in are effects of .
In contrast, for we have , which allows us to conclude that features in are not effects of .
As such, all relevant features in a causal encoding model are effects of , while irrelevant features are not effects of . We hence have the following two interpretation rules:
Interpretation rule S1.
Interpretation rule S2.
2.5.2 Anti-causal decoding models
We describe two counterexamples that show that features in are not necessarily effects of and that features in can be effects of .
Firstly, assume . Since does not d-separate and , i. e. , we obtain that although is not an effect of .
Secondly, assume . Since d-separates and , i. e. , we have although is an effect of .
This establishes that interpreting relevant features in anti-causal decoding models has two drawbacks. Firstly, features in are only potentially effects of . And secondly, effects of might be missed, since effects are not necessarily relevant for decoding the experimental condition. This yields the following two interpretation rules:
Interpretation rule S3.
Interpretation rule S4.
2.5.3 Anti-causal encoding models
According to Reichenbach’s principle, the dependency between and implies that , , or with a joint common cause of and . A priori, we know that brain activity response. This enables us to rule out the case . We now show that one cannot uniquely determine which of the last two scenarios is the case, i. e. features in are only potentially causes of . Consider . In this case, we have and and therefore . But note that while , i. e. is not a cause of . This shows that features in are not necessarily causes of .
Features in , on the other hand, are independent of and hence cannot be causes of .
As such, not all relevant features in anti-causal encoding models are causes of , while irrelevant features are indeed not causal for . We hence have the following two interpretation rules:
Interpretation rule R1.
Interpretation rule R2.
2.5.4 Causal decoding models
We describe two counterexamples that show that one can neither conclude that features in are causes of nor that features in are not causes of .
Firstly, consider where is a hidden common cause of and , yet is a non-observable brain state feature. We have in this example, as as well as are d-separated from only by the non-observable common cause . But both and are not causes of .
Secondly, assume . Since d-separates and , i. e. , we have although is a cause of .
This establishes that interpreting relevant features in causal decoding models has two drawbacks. Firstly, features in are not necessarily causes of . Secondly, causes of might be missed, since causes are not necessarily relevant for decoding the experimental condition. This yields the following two interpretation rules:
Interpretation rule R3.
Interpretation rule R4.
In the previous subsections, we showed that the interpretation of relevant features in encoding and decoding models depends on the experimental setting. This justifies our argument that the distinction of encoding and decoding models is not sufficient to determine their causal interpretation. In particular we argued that, without employing further assumptions,
causal encoding models identify all effects of in .
anti-causal decoding models only identify some features being potentially effects of in .
anti-causal encoding models identify all features being potentially causes of in .
causal decoding models only identify some features being potentially causes of in .
2.6 Causal interpretation rules for combined encoding and decoding models
In section 2.5, we showed that only encoding models in a stimulus-based setting support unambiguous causal statements. This result appears to imply that decoding models, despite their gaining popularity in neuroimaging, are of little value for investigating the neural causes of cognition. In the following, we argue that this is not the case. Specifically, we show that by combining encoding and decoding models, we gain insights into causal structure that are not possible by investigating each type of model individually. In analogy to section 2.5, we again distinguish between stimulus- and response-based paradigms.
For a combined analysis of encoding and decoding models, we intuitively extend our notation and define the following four sets of brain state features, which yield a disjoint partition of :
We now provide causal interpretation rules for each type of feature set and experimental setting.
2.6.1 Stimulus-based setting
Features relevant in encoding and relevant in decoding:
As , it is an effect of (rule S1). Intuitively, the fact that furthermore suggests that is in some sense closer to . Figure 1, however, establishes that this intuition is not correct. Since and , we have even though is an indirect effect of . Hence, features that are relevant in both encoding and decoding do not provide further insights into causal structure. This leads to the following interpretation rule (note the missing bi-implication compared to interpretation rule S1):
Interpretation rule S5.
Features relevant in encoding and irrelevant in decoding:
As , it holds that is an effect of (rule S1). Only looking at the encoding side, however, does not determine whether is a direct () or indirect effect () of wrt. . As , however, and are d-separated by the set (cf. section 2.4). This rules out that is a direct effect of wrt. , leaving as the only explanation. This leads to the following interpretation rule:
Interpretation rule S6.
Features irrelevant in encoding and relevant in decoding:
As , it is not an effect of (rule S2). As further , and are not d-separated by . This implies that is either a cause of variables in or that and share at least one common cause. Examples of these scenarios are and . In both cases, knowledge about can be used to better decode from by removing variations in that are not due to . This leads to the following interpretation rule:
Interpretation rule S7.
Features irrelevant in encoding and irrelevant in decoding:
Features in are neither effects of (rule S2) nor do they provide brain state context wrt. . Hence, they can be considered irrelevant for the present experimental context. This is summarized in the following interpretation rule:
Interpretation rule S8.
2.6.2 Response-based setting
Features relevant in encoding and relevant in decoding:
As , it is potentially a cause of (rule R1). Intuitively, the fact that furthermore suggests that is in some sense closer to , e. g. that is a cause of . The DAG , however, establishes that this intuition is not correct. Since , we have , even though is not a cause of . Hence, features that are relevant in both encoding and decoding do not provide further insights into causal structure. This leads to the following interpretation rule (note the missing bi-implication compared to interpretation rule R1):
Interpretation rule R5.
Features relevant in encoding and irrelevant in decoding:
As , it holds that or (rule R1). Although we cannot distinguish between those two scenarios, we can learn more about from its irrelevance in decoding. would imply that and are not d-separated by and hence . Thus, cannot be a direct cause of wrt. . This leads to the following interpretation rule:
Interpretation rule R6.
Features irrelevant in encoding and relevant in decoding:
As , it is not a cause of (rule R2). As further , and are not d-separated by . This implies that is either a cause of variables in or that and share at least one common cause. An example of this scenario is . Here, knowledge about can be used to better decode from by removing variations in that are not due to . In analogy to the stimulus-based setting, this leads to the following interpretation rule:
Interpretation rule R7.
Features irrelevant in encoding and irrelevant in decoding:
Features in are neither causes of (rule R2) nor do they provide brain state context wrt. . Hence, they can be considered irrelevant for the present experimental context. This leads to the following interpretation rule:
Interpretation rule R8.
We summarize all causal interpretation rules in Table 1. Combining encoding and decoding models is particularly useful if features turn out to be relevant in only one type of model: features only relevant in encoding are not direct effects/causes, while features only relevant for decoding do provide brain state context wrt. while not being effects/causes.
We further note that due to the possibility of hidden confounders, features potentially being a cause and genuine causes cannot be distinguished in the response-based setting without introducing further assumptions.
2.7 Reinterpretation of previous work in a causal framework
In this section, we discuss previous work on the interpretation of encoding and decoding models in light of the causal interpretation rules that we introduced in the previous sections.
2.7.1 Potential confounds
In [40, 44] the problem of potential confounds in multi-voxel pattern analysis (MVPA) has been discussed. In particular, Todd et al. demonstrated that decoding models may determine brain state features as relevant that are statistically independent of the experimental condition. This finding is confirmed by interpretation rules S7 and R7 that we presented in section 2.6. These rules reveal that what Todd et al. termed confounds are exactly those features that are irrelevant in encoding and relevant in decoding. Hence, our reinterpretation in a causal framework re-emphasizes the potential problem highlighted in  and additionally allows to specify the characteristics of such features, i. e. being irrelevant in encoding and relevant in decoding.
In contrast to Todd et al. we do not term those features confounds, as this terminology suggests that such features invalidate interpretation of other features and cannot be interpreted. Instead, we propose to use the terminology features that provide brain state context. Knowledge about the specific characteristics of features that provide brain state context wrt. can indeed lead to interesting causal statements, as demonstrated in : under the additional assumption of causal sufficiency, the assumption that all causally relevant features have been observed, a causal influence of gamma oscillations () on the sensorimotor rhythm was concluded from the fact that .
As we have shown in the previous section, encoding and decoding models provide complementary information. We hence argue that the problem discussed in [40, 44] is not a shortcoming of decoding models, but rather a useful feature. Decoding models provide insights into causal structure that cannot be gained by only investigating encoding models. Pitfalls are not due to weaknesses of decoding models, but a result of negligent interpretation of relevant features. Our findings are thus in line with Todd et al. and Woolgar et al. and strengthen the point that MVPA results need to be interpreted carefully.
2.7.2 Linear encoding and decoding models
In  Haufe et al. demonstrated that linear backward models, i. e. linear decoding models, “cannot be interpreted in terms of the studied brain processes”, as a large weight does not necessarily correspond to a feature that picks up the signal and, vice-versa, a feature that picks up the signal does not necessarily have a large weight. The causal interpretation rules S3, S4 and R3, R4 extend this argument to non-linear decoding models and yield a reinterpretation of these findings in the framework of CBNs. What is more, our distinction of stimulus- and response-based experimental settings allows us to specify the finding that linear forward models, i. e. linear encoding models, are not affected by this problem: in accordance with Haufe et al. we showed that in the stimulus-based setting encoding models, both linear and non-linear, allow unambiguous causal statements. However, in the response-based setting only those features irrelevant in encoding are unproblematic to be interpreted as non-causes of , while the meaning of relevant features remains ambiguous in this setting.
For the linear case Haufe et al. presented an intriguing way to obtain an encoding from a decoding model. Linear models only allow to test for correlation, and jointly Gaussian random variables are the only general case for which the lack of correlation implies independence. As such, if one is willing to assume that all variables are jointly Gaussian, this method together with our findings yields an easy way to enrich causal interpretation when using linear models. Firstly, in contrast to the decoding model one started with, the encoding model obtained by inversion allows for causal statements on its own (interpretation rules S1, S2 or R2). Secondly, since then both an encoding and a decoding model are at hand, also interpretation rules S5-8 or R5-8 can be applied.
2.7.3 Revisiting the introductory examples
We revisit the introductory examples for which we derived testable predictions in section 2.2.
Firstly, consider the causal hypothesis amygdala activity hippocampal activity explicit memory. The two predictions derived in section 2.2, i. e. that amygdala activity is marginal dependent on explicit memory and becomes conditionally independent given hippocampal activity, can be tested by assessing the relevance of amygdala activity in a response-based encoding and decoding model, respectively. Without employing further assumptions these conditions are not sufficient to prove the hypothesized causal structure, though, as amygdala activity hippocampal activity explicit memory or amygdala activity h hippocampal activity explicit memory, where is hidden, are also consistent with the tested conditions. A statement that is warranted by interpretation rule R6 in case that both predictions hold true is the following: amygdala activity is not a direct cause of explicit memory.
Secondly, consider the causal hypothesis pre-stimulus alpha oscillations working memory. The prediction derived in section 2.2, i. e. pre-stimulus alpha oscillations working memory, can be tested by assessing the relevance of pre-stimulus alpha oscillations in a response-based encoding model. Testing this prediction is again not sufficient to conclude the hypothesized causal structure: in case the prediction holds true, pre-stimulus alpha oscillations is only potentially a cause of working memory (cf. interpretation rule R1), as pre-stimulus alpha oscillations working memory, where is hidden, is also consistent with the tested condition. Under the rather strong assumption of causal sufficiency the existence of hidden confounders is denied, and the hypothesized causal structure may indeed be concluded.
Note that a hypothesized causal structure can be rejected whenever one of the conditional independences implied by the DAG is not present in the data.
3 Experimental Results
In this section, we demonstrate the empirical significance of our theoretical results on EEG data recorded during a visuo-motor learning task. We investigate the neural basis of planning a reaching movement by training encoding and decoding models on EEG bandpower features derived from trial-periods in which subjects have either been instructed to rest or to plan a reaching movement. We chose this stimulus-based setting to illustrate our theoretical results, as it admits less ambiguous causal interpretations than a response-based setting (cf. Table 1).
We introduce the experimental setup and recorded data in section 3.1. For simplicity, we omit details of the experimental setup that are irrelevant for the stimulus-based setting. We then describe the data preprocessing steps and the derivation of our brain state features in section 3.2. In section 3.3, we train a nonlinear encoding and a nonlinear decoding model and determine the set of (ir-)relevant features for each model type. The causal conclusions, that can be derived from our analysis, are presented in section 3.4. We first focus on each model type independently, and then discuss further causal insights that are obtained by combining encoding and decoding models.
3.1 Experimental setup and data
Subjects were trained in a visuo-motor learning paradigm to maximize the smoothness of three-dimensional reaching movements performed with their right hand. Specifically, subjects were placed approximately 1.5 m in front of a computer screen displaying a 3D virtual reality scene. At the beginning of each trial, subjects were shown the blank 3D virtual reality scene, instructing them to rest (figure 1(a)). After five seconds, subjects were shown the position of their arm, indicated by a ball with a black and white checkerboard texture, and the target for the upcoming reaching movement, indicated by a ball with a yellow and black checkerboard texture (figure 1(b)
). This instructed subjects to plan but not yet initiate the reaching movement. After a variable delay of 2.5 to 4 seconds, drawn from a uniform distribution, the yellow ball turned green, instructing subjects to carry out the movement (figure1(c)). At the end of each trial, subjects received feedback on the smoothness of their reaching movement, derived from position measurements recorded by a motion tracking device (Impulse X2 Motion Capture System, PhaseSpace, San Leandro, CA, U.S.), and were instructed to return to the initial starting position (figure 1(d)). Each subject carried out a total of 250 movements to randomly chosen locations within the subject’s movement range, distributed across 5 blocks with brief intermissions in between.
Throughout the training session, a 120-channel EEG, with electrodes placed according to the extended 10-20 system, was recorded at 500 Hz sampling rate using actiCAP electrodes and four BrainAmp DC amplifiers (BrainProducts, Gilching, Germany). Prior to each training session, a five-minute resting-state baseline was recorded, for which subjects were instructed to relax while looking at a fixation cross displayed centrally on the screen. All recordings were carried out with a reference electrode behind the left ear and converted to common average reference offline.
Eighteen healthy male subjects (mean age ) participated in this study. All subjects indicated that they are right-handed and conducted the study with their right arm. Subjects gave informed consent in accordance with guidelines set by the Max Planck Society. The study was approved by an ethics committee of the Max Planck Society.
3.2 Data preprocessing and feature computation
In order to facilitate an across-subject analysis, we chose to train our models on bandpower features derived from a group-wise independent component analysis (ICA). This is particularly helpful as it allows to reduce the dimensionality of the feature space and leads to physiologically interesting sources. We note that this cannot introduce artificial dependences between subjects’ trial data as long as the ICA spatial filters obtained from resting state data are being held fixed throughout further analysis. Furthermore, although the group-wise ICA seeks independent components in the resting state data, the dependency structure between those components may be very different during a task.
We first high-pass filtered the resting-state data of each subject at a cut-off frequency of 3 Hz, and then performed a group-wise ICA on the combined resting-state data of all subjects. We chose the SOBI algorithm 
for this purpose, as ICA algorithms based on second-order statistics have been shown to outperform methods based on higher moments in group-wise analyses. We manually inspected the topography and spectrum of every resulting independent component (IC) and rejected all ICs as non-cortical that did not exhibit a clear dipolar topography . The topographies and equivalent dipoles of the six cortical ICs that we retained for further analyses are shown in figure 3. Equivalent dipole locations were derived with a three-shell spherical head model with standardized electrode locations, using the EEGLAB toolbox . We note that ICs 1 and 2, located in sensorimotor and occipital cortex, represent sensorimotor and low-level visual processes, respectively. In contrast, ICs 3, 4 and 5, located in precuneus, in the anterior cingulate, and at the intersection of cuneus and precuneus, respectively, are generated in cortical areas linked to fronto-parietal attention networks . IC 6 appears to represent a sub-cortical source.
To compute the brain state features for the causal analysis, we first applied the spatial filters of the six ICs in figure 3 to each subject’s EEG data recorded during the training session, and then computed log-bandpower in the -range (8–14 Hz) for each subject, IC, and trial in two temporal windows. For the first window, we selected a random 1.5 s window from each trial’s rest phase, excluding the first and last 500 ms to avoid carry-over effects. For the second window, we selected a random 1.5 s window from each trial’s planning phase, again avoiding the first and last 500 ms. Trials, in which subjects already initiated the movement during the planning phase or did not reach the target within 10 s, were excluded as invalid trials. We limited our analysis to the -band, as we found -rhythms most relevant for predicting the progress of visuo-motor learning in a previous study . We excluded one subject from further analyses, as we found this subject’s spectra to be contaminated by EMG activity.
For each of the remaining 17 subjects, we thus obtained between 444 and 498 samples of a six-dimensional feature vector, representing log-bandpower in the -range of six cortical ICs in windows of 1.5 s length. In half of these samples, subjects had been instructed to rest, while in the other half subjects had been instructed to plan a reaching movement.
Topographies and equivalent dipole locations of the six cortical ICs from the group-wise ICA. The residual variances of the equivalent dipole fits are denoted by RV.
3.3 Model training and feature analysis
In the present context, the experimental condition corresponds to the instruction to either rest or to plan a reaching movement. As this instruction always preceded the time-window used to compute the corresponding brain state features, we are working in a stimulus-based setting, i. e. (cf. section 2.3). Our brain state features correspond to log-bandpower in the -range of the cortical ICs in figure 3. In the following, we first describe how we trained a nonlinear encoding and a nonlinear decoding model to distinguish between relevant and irrelevant brain state features in each model type. The models were trained on each subject’s features individually. We then discuss how we combined the experimental results across subjects to identify the (ir-)relevant feature sets on a group-level. These sets form the basis for the causal interpretation in section 3.4.
3.3.1 Encoding analysis
To quantify the relevance of features in encoding, we employed a non-linear independence test based on the Hilbert–Schmidt independence criterion (HSIC) [14, 15]. The HSIC test utilizes a kernel independence measure, capable of detecting linear as well as non-linear dependences between arbitrary input variables. The p-values of the HSIC tests were computed using a permutation analysis with permutations. The kernel size for the HSIC tests was set to the median distance between points in input space . For each subject and feature, this resulted in a p-value that quantifies the probability that the observed data has been generated under the precondition (cf. Table 2).
|Subject||IC 1||IC 2||IC 3||IC 4||IC 5||IC 6|
3.3.2 Decoding analysis
We used a random forest classifier (RF) to decode the experimental condition from the brain state features. For each RF model, trees were grown using bagging, i. e. for every tree a training subset ( of the whole training set) was drawn by sampling with replacement. While growing the trees, at each node the best split of (as suggested in 
) randomly chosen features was used. Estimates of prediction accuracy (PE*) were obtained by leave-one-trial-out cross-validation. This resulted in the subject-specific decoding accuracies shown in Table3. A Wilcoxon signed rank test rejected the null-hypothesis of chance-level decoding accuracies on the group-level with .
To quantify the relevance of features in decoding, we then successively permuted every feature times and compared the resulting PEs to the PE with all features intact (PE*). For each subject and feature, this resulted in a p-value that quantifies the probability that the observed data has been generated under the precondition (cf. Table 4).
3.3.3 Identifying (ir-)relevant feature sets on the group-level
In order to identify the (ir-)relevant features on the group-level, we make use of the fact that by construction p-values are uniformly distributed under the null-hypothesis. In the present context, this implies that if a feature is irrelevant in encoding/decoding, the 17 subject-specific p-values of this feature are drawn from a uniform distribution. This enables us to use a Kolmogorov-Smirnov permutation test (KS test) (with permutations), for each feature and model type, to quantify the probability of the set of observed p-values under the assumption of (conditional) independence. We reject the null-hypothesis of or , for the encoding or decoding side, respectively, if the resulting p-value of the KS test is below the significance level . Conversely, we accept the null-hypothesis if the p-value across subjects exceeds . In this way, we can test each feature’s significance for encoding and decoding on a group-level, while avoiding problems in causal analysis related to pooling features across multiple subjects .
The resulting p-values are shown in Table 2 and 4 for the encoding and decoding analysis, respectively. For the encoding side, we find all six cortical ICs to be relevant. For the decoding side, we only find ICs 1 and 2 to be relevant. We hence obtain the following sets of (ir-)relevant features on the group-level:
This enables us to distinguish between ICs that are relevant only for encoding and ICs that are relevant in both model types: and .
3.4 Causal analysis
Having identified the sets of (ir-)relevant features, we proceed to apply our causal interpretation rules. We first consider each model type independently, and then discuss causal conclusions that can be derived from considering both models jointly.
3.4.1 Encoding model only
From and interpretation rule S1, it follows that is causal for the -bandpower of every IC. Accordingly, we find that every IC in figure 3 responds to the instruction to plan a reaching movement. The specific role of each of these cortical processes, however, remains speculative.
3.4.2 Decoding model only
The above chance-level decoding accuracy on the group-level indicates that at least one of the six ICs is an effect . Interpretation rules S3 and S4, however, forbid any further causal interpretation.
3.4.3 Combination of the encoding and the decoding model
From in combination with interpretation rule S6, we conclude that ICs 3–6 are only indirect effects of . As , we can further conclude that at least one of ICs 1 and 2 is a direct effect of . This follows from the fact that if and were both indirect effects, all observed brain state features would be indirect effects.
We note that these causal conclusions are in line with the roles commonly attributed to the corresponding cortical areas. Specifically, the higher-level cortical areas represented by ICs 3–5 are only found to be indirectly modulated by the instruction to plan a reaching movement. In contrast, low-level sensorimotor and visual areas, represented by ICs 1 and 2, respectively, are found to contain at least one direct effect of the instruction to plan a movement.
|Subject||IC 1||IC 2||IC 3||IC 4||IC 5||IC 6|
The rules presented in this work provide a guideline to researchers which causal statements are and which ones are not supported by empirical data when analyzing encoding and decoding models. In particular, we argued that only encoding models in stimulus-based paradigms support unambiguous causal statements. We demonstrated that further causal insights can be derived by combining encoding and decoding models, and illustrated the significance of this theoretical result on experimental data. While we have chosen an EEG study for this illustration, our causal interpretation rules apply to any type of brain state feature. In the following, we discuss limitations and potential extensions of this framework.
4.1 The iid assumption in neuroimaging
It is likely that neither of the iid assumptions is met by neuroimaging data. Consider for example subjects getting tired through the course of an experiment. In this case, the features for later trials may follow another distribution than those for earlier trials, violating the assumption of identical distributions. Also the brain’s state may depend on previous activations and hence later trials may be dependent on previous trials, violating the independence assumption. As statistical tests rest on the iid assumption, it is important to consider this limitation when interpreting test results.
4.2 Finite empirical data
Our theoretical arguments rest on the assumption that it is possible to distinguish between relevant and irrelevant features in encoding and decoding models, i. e. that we have access to an oracle for univariate independence and multivariate conditional independence tests. In practice, we are faced with several interrelated problems. Firstly, the identification of irrelevant features rests on the readiness to interpret negative results. We need to interpret the lack of evidence against independence as evidence in favor of it. As such, we need to keep in mind that observing more data and/or using more powerful (conditional) independence tests may falsify previous statistical tests, thereby altering our causal conclusions. Secondly, we either need to employ non-linear encoding and decoding models to test for (conditional) independences, as in the analysis of our experimental data in section 3, or we need to assume that all observed brain state features are jointly Gaussian. This follows from the fact that linear models only test for uncorrelatedness, and in general uncorrelatedness only implies independence if all variables are jointly Gaussian. Note that other frameworks enable causal discovery in linear models, e.g. by introducing the additional assumption of additive non-Gaussian noise [37, 19]. Lastly, it is difficult to base conditional independence tests on permutation approaches, as these are biased towards conditional dependence . The development of unbiased conditional independence tests is an area of active research .
4.3 Univariate vs multivariate analysis
We based our causal analysis on commonly employed and intuitive notions of feature relevance (cf. section 2.4). In encoding models, a feature is relevant if it varies with the experimental condition. This corresponds to an univariate independence test . In decoding models, a feature is relevant if it cannot be removed without increasing the minimum Bayes error. This corresponds to a multivariate conditional independence test . The interpretation rules presented in this work apply whenever these tests are employed, independently of whether encoding and decoding models or direct statistical tests for (conditional) independence are being used [14, 45].
We note that there are instances in which decoding models are used to carry out an encoding analysis. Consider the searchlight technique . Here, it is tested whether a set of voxels as a whole contains information about the experimental condition. In this case, the decoding model is used for a marginal independence test . This approach is oblivious to the causal structure within the set . As such, the searchlight technique does not provide causal insights beyond those implied by an encoding model. The additional insights offered by decoding models rest on multivariate conditional independence tests.
4.4 Whole brain analysis
In case of a high-dimensional feature space, e. g. voxels in fMRI or bandpower features in high-density EEG recordings, training a decoding model on the whole feature set may be infeasible. To harness the additional insights provided by a decoding model, it may be necessary to reduce the feature space dimensionality before training a decoding model, e. g. by clustering for fMRI or ICA for EEG recordings. We note that it is not trivial to reduce the dimensionality of the feature space without discarding causally relevant information.
4.5 Untestable assumptions
As it is the case for any type of empirical inference, causal inference also rests on a set of untestable assumptions. In particular, causal inference in the framework of CBNs rests on the CMC and the assumption of faithfulness. Theoretical results show that the set of unfaithful distributions has measure zero relative to all probability distributions that can be generated by a given DAG. As long as nature has no particular reason to favor unfaithful distributions, we are thus unlikely to encounter them in practice. We note that further assumptions might allow stronger causal statements, e. g. the assumption of causal sufficiency rules out the existence of hidden common causes.
4.6 Causal inference in neuroimaging
We note that the rules presented here have, to a certain extent, already been applied in the context of neuroimaging [32, 15, 41, 26]. The primary contribution of our work is to point out their relation to widely used methods for analyzing neuroimaging data. In particular, we show that interpreting encoding and decoding models is a form of causal inference. Given the prevalent use of causal terminology in the interpretation of neuroimaging studies, we believe it is essential to make the inherent assumptions and limitations explicit. We further note that combining encoding and decoding models is only a first step towards a causal analysis of empirical data. More detailed insights can be obtained by additional conditional independence tests, e. g. by training decoding models on subsets of variables and/or permuting subsets of variables. Causal inference algorithms like the PC or FCI algorithm are designed for tackling such questions and hence can yield more detailed causal insights [30, 38]. In contrast, encoding and decoding models are not especially designed for these purposes but, as shown, might still warrant some causal interpretation.
We close by emphasizing that, if relevant features in encoding and decoding models are interpreted in a causal sense, one inevitably accepts the untestable assumptions and limitations expatiated in this article. The only way to elude this situation, in case one is not willing to make those assumptions, is to resign from causal interpretations.
If not only correlational statements but ultimately neural causes of cognition are of interest, further assumptions and causal inference algorithms, which go beyond encoding and decoding models, should be considered. Future research may investigate how causal inference methods can be facilitated in neuroimaging, discuss the appropriateness of different assumptions, and explore possible ways to weaken or refine those assumptions in case of neuroimaging data.
The authors want to thank the reviewers for their encouraging, very concise and constructive feedback, which significantly improved this manuscript.
-  L.Y. Atlas, N. Bolger, M.A. Lindquist, and T.D. Wager. Brain mediators of predictive cue effects on perceived pain. The Journal of Neuroscience, 30(39):12964–12977, 2010.
-  A. Belouchrani, K. Abed-Meraim, J.-F. Cardoso, and E. Moulines. A blind source separation technique using second-order statistics. IEEE Transactions on Signal Processing, 45(2):434–444, 1997.
-  Leo Breiman. Random forests. Machine Learning, 45(1):5–32, 2001.
-  Leo Breiman. Manual On Setting Up, Using, And Understanding Random Forests V3.1. 2002. http://oz.berkeley.edu/users/breiman/Using_random_forests_V3.1.pdf; accessed 2013-03-22.
-  S.L. Bressler and V. Menon. Large-scale brain networks in cognition: emerging methods and principles. Trends in Cognitive Sciences, 14(6):277–290, 2010.
-  Olivier David, Stefan J Kiebel, Lee M Harrison, Jérémie Mattout, James M Kilner, and Karl J Friston. Dynamic causal modeling of evoked responses in EEG and MEG. NeuroImage, 30(4):1255–1272, 2006.
Tyler Davis, Karen F LaRocque, Jeanette A Mumford, Kenneth A Norman, Anthony D
Wagner, and Russell A Poldrack.
What do differences between multi-voxel and univariate analysis mean? How subject-, voxel-, and trial-level variance impact fMRI analysis.NeuroImage, 97:271–283, 2014.
-  A. Delorme and S. Makeig. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1):9–21, 2004.
-  A. Delorme, J. Palmer, J. Onton, R. Oostenveld, and S. Makeig. Independent EEG sources are dipolar. PLoS One, 7(2):e30135, 2012.
-  Karl J Friston, Andrew P Holmes, Keith J Worsley, J-P Poline, Chris D Frith, and Richard SJ Frackowiak. Statistical parametric maps in functional imaging: a general linear approach. Human Brain Mapping, 2(4):189–210, 1994.
-  K.J. Friston, L. Harrison, and W. Penny. Dynamic causal modelling. NeuroImage, 19(4):1273–1302, 2003.
-  M Gomez-Rodriguez, J Peters, J Hill, B Schölkopf, A Gharabaghi, and M Grosse-Wentrup. Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery. Journal of Neural Engineering, 8(3):036005, 2011.
-  C.W.J. Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: Journal of the Econometric Society, 37(3):424–438, 1969.
-  Arthur Gretton, Kenji Fukumizu, Choon H Teo, Le Song, Bernhard Schölkopf, and Alex J Smola. A kernel statistical test of independence. In Advances in Neural Information Processing Systems, pages 585–592, 2008.
-  Moritz Grosse-Wentrup, Bernhard Schölkopf, and Jeremy Hill. Causal influence of gamma oscillations on the sensorimotor rhythm. NeuroImage, 56(2):837–842, 2011.
-  Stephan Hamann. Cognitive and neural mechanisms of emotional memory. Trends in cognitive sciences, 5(9):394–400, 2001.
-  Stefan Haufe, Frank Meinecke, Kai Görgen, Sven Dähne, John-Dylan Haynes, Benjamin Blankertz, and Felix Bießmann. On the interpretation of weight vectors of linear models in multivariate neuroimaging. NeuroImage, 87:96–110, 2014.
-  Paul W. Holland. Statistics and causal inference. Journal of the American Statistical Association, 81(396):945–960, 1986.
Patrik O Hoyer, Aapo Hyvarinen, Richard Scheines, Peter Spirtes, Joseph Ramsey,
Gustavo Lacerda, and Shohei Shimizu.
Causal discovery of linear acyclic models with arbitrary
Proceedings of 24th Conf. on Uncertainty in Artificial Intelligence (UAI 2008), pages 282–289, 2008.
-  Nikolaus Kriegeskorte, Rainer Goebel, and Peter Bandettini. Information-based functional brain mapping. Proceedings of the National Academy of Sciences of the United States of America, 103(10):3863–3868, 2006.
-  Guillaume Lio and Philippe Boulinguez. Greater robustness of second order statistics than higher order statistics algorithms to distortions of the mixing matrix in blind source separation of human eeg: Implications for single-subject and group analyses. NeuroImage, 67:137–152, 2013.
-  Joseph T Lizier and Mikhail Prokopenko. Differentiating information transfer and causal effect. The European Physical Journal B-Condensed Matter and Complex Systems, 73(4):605–615, 2010.
-  Christopher Meek. Strong completeness and faithfulness in Bayesian networks. In Proceedings of the Eleventh Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-95), pages 411–418, San Francisco, CA, 1995. Morgan Kaufmann.
-  Timm Meyer, Jan Peters, Thorsten O Zander, Bernhard Schölkopf, and Moritz Grosse-Wentrup. Predicting motor learning performance from electroencephalographic data. Journal of Neuroengineering and Rehabilitation, 11(1):24, 2014.
-  Tom M Mitchell, Rebecca Hutchinson, Radu S Niculescu, Francisco Pereira, Xuerui Wang, Marcel Just, and Sharlene Newman. Learning to decode cognitive states from brain images. Machine Learning, 57(1-2):145–175, 2004.
-  Jeanette A Mumford and Joseph D Ramsey. Bayesian networks for fMRI: A primer. NeuroImage, 86:573–582, 2014.
-  Nicholas E Myers, Mark G Stokes, Lena Walther, and Anna C Nobre. Oscillatory brain state predicts variability in working memory. The Journal of Neuroscience, 34(23):7735–7743, 2014.
-  Thomas Naselaris, Kendrick N Kay, Shinji Nishimoto, and Jack L Gallant. Encoding and decoding in fMRI. NeuroImage, 56(2):400–410, 2011.
-  Andrew Y. Ng and Michael I. Jordan. pages 841–848, 2002.
-  Judea Pearl. Causality: models, reasoning and inference. Cambridge University Press, 2000.
-  Francisco Pereira, Tom Mitchell, and Matthew Botvinick. Machine learning classifiers and fMRI: a tutorial overview. NeuroImage, 45(1):S199–S209, 2009.
-  JD Ramsey, SJ Hanson, C. Hanson, YO Halchenko, R.A. Poldrack, and C. Glymour. Six problems for causal inference from fMRI. NeuroImage, 49(2):1545–1558, 2010.
-  Geraint Rees, Gabriel Kreiman, and Christof Koch. Neural correlates of consciousness in humans. Nature Reviews Neuroscience, 3(4):261–270, 2002.
-  H. Reichenbach. The Direction of Time. University of California Press, Berkeley, 1956.
-  Alard Roebroeck, Elia Formisano, and Rainer Goebel. Mapping directed influence over the brain using granger causality and fMRI. NeuroImage, 25(1):230–242, 2005.
-  Bernhard Schölkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. On causal and anticausal learning. In 29th International Conference on Machine Learning (ICML 2012), pages 1255–1262, 2012.
-  Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, and Antti Kerminen. A linear non-gaussian acyclic model for causal discovery. The Journal of Machine Learning Research, 7:2003–2030, 2006.
-  Peter Spirtes, Clark N Glymour, and Richard Scheines. Causation, prediction, and search. MIT press, 2000.
-  Carolin Strobl, Anne-Laure Boulesteix, Thomas Kneib, Thomas Augustin, and Achim Zeileis. Conditional variable importance for random forests. BMC bioinformatics, 9(1):307, 2008.
-  Michael T Todd, Leigh E Nystrom, and Jonathan D Cohen. Confounds in multivariate pattern analysis: Theory and rule representation case study. NeuroImage, 77:157–165, 2013.
-  Lourens Waldorp, Ingrid Christoffels, and Vincent van de Ven. Effective connectivity of fMRI data using ancestral graph theory: dealing with missing regions. NeuroImage, 54(4):2695–2705, 2011.
-  Lawrence M Ward. Synchronous neural oscillations and cognitive processes. Trends in cognitive sciences, 7(12):553–559, 2003.
-  Sebastian Weichwald, Bernhard Scholkopf, Tonio Ball, and Moritz Grosse-Wentrup. Causal and anti-causal learning in pattern recognition for neuroimaging. In Pattern Recognition in Neuroimaging, 2014 International Workshop on, pages 1–4, June 2014.
-  Alex Woolgar, Polina Golland, and Stefan Bode. Coping with confounds in multivoxel pattern analysis: What should we do about reaction time differences? a comment on Todd, Nystrom & Cohen 2013. NeuroImage, 2014.
-  Kun Zhang, Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Kernel-based conditional independence test and application in causal discovery. CoRR, abs/1202.3775, 2012.