A common goal of modern scientific studies is determining statistically significant relationships between multiple sets of features. In a collaborative work with behavioral scientists, the key interest of our research is to determine the extent to which genetic factors explain the variability in brain function (activation and connectivity) and how both genetics and brain function may influence human behavior. In most biomedical studies, these sets of features are often complexly structured, with each set having dimension orders of magnitude larger than the sample size. As a consequence of the “curse of dimensionality” in high-dimensional settings, only factors with large effect sizes can be detected after adjusting for multiple comparisons. The deficiencies of massive univariate analyses have motivated methods that aggregate effects from many individual variables. The goal of such methods is to assess the joint effects of a set of variables, rather than separately estimating many univariate effects on the outcome of interest. Examining the overall effects is particularly relevant in the presence of two sets of high-dimensional data from distinct measurement modalities. For instance, in our collaborative work, we examine data from different imaging modalities, such as functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) which measure different aspects of brain function. Thus, our goal is to develop a method to determine whether these two modalities are significantly related and how that association might vary across different experimental conditions. Several parametric and non-parametric approaches to multi-modal association testing have been developed and revisited in the scientific and statistics literature.
Mantel’s test Mantel (1967) is one of the earliest formulations of a distance-based, ostensibly nonparametric, test for association of features between two observational modalities. To conduct the Mantel test, one computes a similarity or distance between each pair of subjects in each modality, and tests for significance of the correlation of similarities via an asymptotic distribution or permutation testing procedure. Formally, let and be measurements on subjects from two observational modalities X and Y, with metrics and giving rise to (dis)similarity matrices and respectively, where represents the dis(similarity) between subjects and with respect to the modality X, and similarly for with respect to the modality Y. The original form of the MT statistic can then be stated as
from which significance can be assessed using an asymptotic distribution or permutation procedure.
Alternative, but closely related tests, have been developed from the RV coefficient Robert and Escoufier (1976) and the distance covariance Székely et al. (2007). The RV coefficient is defined as the correlation of pairwise similarities across the two modalities, and thus is a scaled version of a similarity MT. Distance covariance Székely et al. (2007)
is a more recent approach, which is defined as the covariance of distances between random vectorsand . A detailed discussion of the connections of MT, the RV coefficient, and distance covariance is given in Omelka and Hudecová (2013), as well as simulation results indicating that tests with distance covariance have higher power than the standard Mantel test (using Euclidean distance matrices) in some situations. However, if we choose and in the Mantel test to be the Gower-centered distance matrices, then the Mantel test and the dCov tests are equivalent Omelka and Hudecová (2013).
The primary contribution of this article is the introduction of the adaptive Mantel test as a method for high-dimensional inference, which improves upon the classical Mantel test by simultaneously testing across a set of similarity measures without the need to explicitly adjust for multiple comparisons. Section 2 reviews the original MT and establishes the relationship of MT and linear model score statistics, from which a unified formulation of the fixed effects, random effects, and ridge regression score tests is developed. Section 3 describes the implementation and use of the adaptive Mantel test (AMT). Section 4 evaluates the performance of the test through simulations, and illustrates the use of AMT with an investigation of EEG and genetic data from a working memory study of 350 college students. Section 5 concludes with a discussion of practical implications and directions for future work.
2 Mantel’s Test for Linear Association
2.1 Forms of Mantel’s Test
If two statistics and produce equivalent test results when calculated on the same size sample, we refer to these statistics as testing equivalent, and write . When the tests from and are asymptotically equivalent as , we say is asymptotically testing equivalent to , and write . To develop the theoretical foundation of the adaptive Mantel test, we first discuss the original form of Mantel’s test and introduce a modified form that is asymptotically testing equivalent. Consider independent observations each measured on , where is the design matrix from the -dimensional feature space and the response vector from a univariate feature space , and further suppose is column-centered, and is centered. Let be an similarity matrix that measures the similarity of observations in . For a similarity or dissimilarity metric on , let be the corresponding Gram matrix with . Similarily, for some metric defined on , let be the corresponding Gram matrix of
. The original Mantel test statistic is defined as
The reference distribution under the null hypothesis of no association between the measures and measures, can be obtained from the observed features and by permuting the observation labels for one set of features and calculating the empirical reference distribution for . Equivalently, one can hold one matrix fixed, say , and simultaneously permute the rows and columns of . In Mantel’s original formulation, uses only the upper triangle of each matrix, excluding the diagonal. In this paper, we define a modified Mantel test statistic, denoted , as
Testing will not be equivalent to testing when the diagonals of and are both nonconstant, but it can be shown that Martins-Filho and Yao (2006). For the remainder of this article, we will focus on tests with similarity measures that can be written as weighted Euclidean inner products, as these tests have a close connection with linear models. Extending these results to a broader class of kernel similarity measures is a direction for future work.
We now define three Gram matrices calculated from weighted inner products, which we denote and since these will be shown in the following section to correspond to the score tests for the random effects, fixed effects, and ridge regression models respectively. For a positive semi-definite (p.s.d.) weight matrix , the corresponding weighted inner product is calculated as . Choosing gives the standard Euclidean inner product, with Gram matrix
where , . Another natural choice for weight matrix is (assuming the inverse exists), which is the projection matrix into , the column space of . The Gram matrix is
is also recognizable as the “hat matrix” from the fixed effects model, and is related to the Mahalanobis distance. When is not full rank, such as when , we can replace with a generalized inverse , since the similarity matrix is invariant to the choice of inverse. Alternatively, we can pre-condition the weight matrix by adding a positive constant to the diagonal, i.e. . This gives similarity matrix
2.2 Linear Model Score Tests
In general, the score test is defined as follows. For a stochastic model with parameters and observations , and with likelihood , score vector , and Fisher information , the score test statistic for testing is
The classical fixed effects model is robust and broadly applicable for simple association testing, but requires , and so is not feasible for high-dimensional settings. It does however serve as a useful theoretical comparison to the random effects and ridge regression models to be discussed next. The fixed effects model can be written
The global score test statistic is straightforward to calculate, and can be written as
In practice, the nuisance parameter can be replaced with , which is the REML estimator when there are no adjustment covariates. This is a scalar that is fixed under permutations of . Therefore,
Now consider the random effects model
where , . Under the null hypothesis of no association between and , we have . Similar to Liu et al. (2007), we first calculate the score and use the term that involves both and as our score test statistic. Thus a testing equivalent form for the random effect score test statistic is
The relationship between similarity-based tests and score tests of random-effect or kernel machine regression has been previously discussed in detail (Kwee et al. (2008); Tzeng et al. (2009); Pan (2011)).
The ridge regression model is a penalized form of 1, for which the estimator of is defined as the minimizer of the penalized residual sum of squares:
Although the above penalized function does not correspond to a likelihood conditional on ), it can instead be formulated from the likelihood for the augmented data model
From this augmented likelihood, we can derive the score statistic as
2.3 (B)ridging the Fixed-effects and Random-effects Models
In this section we demonstrate that is closely related to well-known correlations in several linear models. The sample correlation of the similarity measures of all subjects is defined as
Since is a correlation, we have in general, and for and p.s.d., this is further restricted to . When permutations are used to assess the strength of association, using is equivalent to testing the significance of , since the denominator of is fixed when simultaneously permuting the rows and columns of either or . The sample correlation of similarities for the three choices of
above can be conveniently related through the singular value decomposition (SVD) of.
Theorem. Let be an column centered matrix of covariates with and singular value decomposition , with squared singular values . Let be an centered vector of scalar responses, and let and .
Relation to Correlation of and
The preceding theorem also gives a straightforward derivation of the null distributions for the score tests. Since the matrix from the SVD of is orthogonal, the distribution of is
. The asymptotic distributions of the score test statistics then follow from standard results on the distribution of quadratic forms of normal random variables.
Corollary. Connections between the Mantel test statistics
Let and .
These results provide a description of the ridge regression score test as a natural intermediary between the fixed and random effects score tests. A crucial difference of the ridge test is the presence of the tuning parameter . It is obvious that when , reduces to , and reduces to . On the other hand, when , the test based on a ridge-penalty converges to that based on the random-effect model. Note that in permutation based methods, multiplying a constant does not change the permuted p-value. Therefore, , which converges to , i.e., .
Remark To better understand the differences in the tests based on and , a geometric comparison is helpful. Recognizing as the projection into and writing , we can interpret the test of as testing the norm of projected image of into . Equivalently, tests the significance of the norm which is the Mahalanobis norm, since is the Fisher information matrix of the fixed effects model. In contrast, the statistic tests the unweighted norm of . Since the th component of the vector is the covariance of the th feature of with , the test of can be understood as giving equal weight to each feature in in measuring the strength of the overall relationship of and , whereas the test of weights the contribution of each feature of
according to the Mahalanobis norm. In the case of independent features, these weights are inversely proportional to the observed variance of that feature.
after transformation by the left eigenvectors of. From 2, we see that the fixed effects score test is equivalent to testing the Euclidean norm of . Whereas from 3, the random effects score test statistic is equivalent to testing the weighted norm of , where the th component (corresponding to the th eigenvector) is weighted by . This has the effect of emphasizing the influence of directions in X for which has large variance and reducing the influence of directions with small variance. The ridge regression score test is a compromise between the fixed and random effects, with small yielding a test close to the fixed effects (or identical at ), and large yielding a test close to the random effects score test, and identical tests for . Geometrically, the ridge test weights the proportional to the as in the random effects test, but flattens each weight by a factor of .
3 Adaptive Mantel Test
Effectively using kernel methods requires an appropriate selection of the kernel function and tuning parameters for the particular setting. Selection methods have been extensively considered in the context of prediction problems, with cross-validation as the de facto standard. Cross-validation is a straight-forward and practical selection method for prediction, but may be difficult to implement for hypothesis testing, since the type I error rate needs to be controlled. Furthermore, the tuning parameter selected by minimizing the CV MSE may not necessarily yield the highest powered test. This section introduces the adaptive Mantel test (AMT), which extends the classical MT to simultaneously test across a set of tuning parameters and kernels without the need to directly apply adjustments for multiple comparisons.
3.1 Algorithm for the Adaptive Mantel Test
The “adaptive” procedure used here is similar to the adaptive sum of powered score test algorithm described in Xu et al. (2017). The procedure receives as input a list of pairs of metrics/kernels from which the matrices and are computed for each metric pair, . These metrics may be from a single family with varying tuning parameters, such as ridge kernels with different penalization terms, or may include kernels from different families.
For each , is calculated as the -value of the Mantel test with metrics and for and respectively. The AMT test statistic is defined as the minimum of these values,
A permutation procedure can be used to calculate the reference distribution for . For each , and , is generated by permuting rows and columns of simultaneously, and the corresponding test statistic is calculated. The AMT -value is then calculated as
General pseudocode for the adaptive Mantel test is given in Algorithm 1.
3.2 Computational Methods
If the feature space is very high-dimensional or if is large, a straightforward implementation of Algorithm 1 may be computationally impractical. However, when only ridge kernels with varying values of are included in AMT, there are two approaches that can be used to greatly reduce the computational cost.
The first approach utilizes the SVD . The computational complexity needed for finding the SVD for is . Once the SVD is computed, we can compute and , which has a total complexity of . Note that when , the rank is often the same as ; as a result, the cost needed for calculating is . Calculating the test statistics for permutations requires , for a total computational complexity of .
Alternatively, when is very large relative to , we can use the identity so that the matrix inverse is applied to an , rather than , matrix. From this identity, can be rewritten as
Note that calculating involves multiplying the matrix and the matrix , multiplying two matrices, and inverting an matrix. When , the computation cost is dominated by calculating , which has a complexity of . The Mantel test statistic can be calculated as
which has a complexity of . With permutations, the total computational complexity is , which is less than the required computational complexity using SVD. Thus, switching from the feature space to the subject space (i.e., from a similarity matrix of the features to an similarity matrix of the subjects), has a computational advantage. Additionally, in some situations the SVD computation may be unstable, thus the matrix identity method may be recommended as the more robust approach.
3.3 Variance Explained and the Ridge Penalty
By allowing for simultaneous testing over a set of tuning parameter values, AMT lessens the challenge of parameter selection, but does not completely resolve it. Although it does not require an overly conservative adjustment for multiple testing, the power of AMT does decrease as the number of metrics considered increases. Conversely, the test results are highly sensitive to the choice of parameters to test. Consequently, one must still take some care in the selection of the included parameters, balancing the desire to use a wide range of parameter values, with the gains of using a small set of parameters. When only ridge kernels are included in AMT, previous results on the role of the ridge penalty term in predictive modeling can help with the identification of a reasonable set of values to test. Specifically, it has been shown that when the ridge penalty is chosen to be the noise to signal ratio, the resulting shrinkage estimator is identical to the best linear unbiased predictor (BLUP) for the random effects model, and moreover, for a new observation with unknown response value the predictions using the ridge and random effects models are the same de los Campos et al. (2013). Thus, when using ridge regression for prediction, it is recommended that the penalty should reflect the relevant level of noise versus signal, i.e., .
To apply this result to practical settings, if one can determine a priori a likely range for the noise to signal ratio or a related quantity, this will determine a reasonable range of penalty terms. For instance, in assessing the genetic influence on observed phenotypes, the noise to signal ratio is related to what is known as the heritability of the phenotype, which can be understood as the proportion of variance in the observed trait explained by the genetic data. Formally, for (standardized) genetic data matrix consisting of alleles for single nucleotide polymorphisms (SNPs), and observed phenotype vector , the random effects model given in Eq. 3 is commonly used to estimate the genetic heritability of the trait Yang et al. (2011); Liu et al. (2007). From this model, the narrow-sense heritability of the phenotype is defined as
Plugging in gives
We see then that if is known, the optimal penalty (for prediction) can be found by solving for the noise to signal ratio. In practice, the scientific interpretation and range of plausible values of will depend on the specific modalities of X and Y. For instance, in the genetics literature, would generally indicate high heritability, while a heritibility of
is probably not scientifically interesting. As a point of reference, most estimated heritability in the UK Biobank data is between 0.1 to 0.4Ge et al. (2017).
4 Simulations and Applications to Imaging Genetics Data
To verify the theoretical connection between the fixed, random, and ridge regression models and to illustrate the differences in testing results, we now compare the power of these models via a simulation study and application to a real-world imaging genetics data set.
For this simulation study of the adaptive Mantel test, the simulated data was generated from the random effects model (Eq. 3), with observations, number of covariates ranging from 250 to 2000, and fixed. For each setting, 500 simulations were run. The design matrix was generated from draws from a
-variate normal distribution, where is chosen to have an structure with and th diagonal element equal to . The hypothesis of interest for heritability analyses is , which is equivalent to . AMT was applied with and for , where corresponds to the similarity measure, with 1000 permutations.
Overall, the simulation results show that the power of the adaptive Mantel test is not substantially lower than the -values for the individual Mantel tests. Figure 1 shows the simulation results for the heritability fixed at , requiring the effect size decrease as increases. Comparing the power of AMT and the simple MT for each of the , we observe that the power of AMT is competitive with the best of the simple Mantel tests for the considered. For this setting, and exhibited the highest power among the simple Mantel tests; smaller values had lower power relative to the larger values for all values of . The power of all the tests decreased as increased, leveling off with approximately 45% power for . In Figure 1, is fixed, resulting in increasing as increases. In this setting, the power of AMT and MT increase as increases since each added feature increases . AMT is again competitive with the best simple Mantel tests, and again the smaller penalty terms perform relatively poorly across the range of -values. All of the tests converge to roughly the same power when , for which the power is near 90%.
4.2 Association of EEG Coherence and Selected SNPs
We next consider data from 350 healthy college students from Beijing Normal University who participated in a visual working memory task, during which 64-channel EEG was recorded at 1 kHz. The total duration of the experiment was approximately 10 minutes for each subject. Approximately SNPs were also measured for each subject. Standard pre-processing and quality control steps were applied to both the EEG and genetic data. We are interested in testing the association of alpha and theta band coherence with a group of 11 SNPs that have been identified as potentially related to Alzheimer’s Disease.
The coherence between two EEG channels at a particular frequency is a measure of the oscillatory concordance of the the two signals at . The pairwise coherence for EEG channels is a symmetric matrix, from which we extract the upper triangle and vectorize to form the matrix . This results in 2080 distinct features when using all 64 channels, and 300 distinct features for the 25 selected frontal channels. The adaptive Mantel test was performed with and using 1000 permutations. Genetic similarity of subjects was calculated as the inner product of the centered standardized SNP data for all tests. For the alpha band, all 64 channels gave , and the selected frontal channels gave . Results for the theta band were and for all 64 channels and frontal channels respectively. Since the adaptive Mantel test was used, these -values already take into account testing across multiple . In the case of the alpha band, the test results suggest that coherence involving channels outside of the selected frontal channels may be associated with genetic similarity determined by the 11 AD SNPs, whereas for the theta band, the SNP association appears stronger when considering only the frontal channels. For a better sense of the significance of these 11 SNPs, the AMT -values for 200 sets of 11 randomly selected SNPs were calculated for each of the four tests considered here. The boxplot of these values are given in Figure 2. For the alpha – all channels test and the theta – frontal channels test, the -values from the AD SNPs (0.065 and 0.085 respectively) are outside the ranges of -values from the randomly selected SNPs.
|Test||-value (Best )|
|, All Channels||0.065 (5)|
|, Frontal Channels||0.381 (1)|
|, All Channels||0.416 (0.5)|
|, Frontal Channels||0.085 (0.5)|
To assess possible contributions of individual channel pairs, each channel pair was separately tested for significance with the AD SNPs with the adaptive Mantel test. Figure 2 shows the most significant channel pairs across all channel pairs for the alpha band. The TP8 – CP2 connection is the most significant at . Other top pairs are C4 – PO4, C4 – AF7, and T8 – FT7. The most significant connections are mostly between the right temporal regions with the left frontal regions. For the theta band, the most significant channel pairs were P9 – Cz, P9 – CP3, CP3 – AF3, and P8 – F3. The P9 channel also had relatively significant connections with many other channels in the frontal left hemisphere. These results are supported by a number of previous studies that have established links between working memory performance and features measured by EEG. For instance, Onton et al. (2005) found increases in frontal midline theta power with increasing memory load during a verbal-working memory task; Sauseng et al. (2005) also found that alpha coherence plays a significant role in “top-down” control during working memory tasks; and Simons and Spiers (2003) identified important interactions between the prefrontal and medial temporal lobes for the processing of long-term memory.
Similarly, the association of each individual SNP with EEG coherence was assessed with AMT. For alpha coherence with all channels, the most significant SNP (
) is rs2227564, a functional polymorphism within plasminogen activator urokinase (PLAU) gene. An allele of this SNP has been linked to significantly high plaque counts in AD, although its role is not well-established. The second most significant SNP from the individual tests is rs3851179, a SNP upstream of the PICALM gene. This SNP has been repeatedly implicated as a factor in AD, as well as Parkinson’s Disease and schizophrenia, although there are dissenting results regarding its significance in particular Chinese populations. In the genetics literature on cognitive function in healthy subjects, polymorphisms in neurotransmitter genes, such as those in the dopamine pathway, have been shown to be significantly associated with increased neuronal activity in the prefrontal cortex during working memory tasks, as measured by fMRIBertolino et al. (2006). Vogler et al. analyzed data from the -back memory task for 2298 subjects, and estimated genome-wide heritability of working memory accuracy to be 41% (95% CI: 0.13, 0.69) Vogler et al. (2014). Taken together, these results make it plausible genetic factors have an important influence on brain function related to working memory, but determining the specific nature of the role of genetics in brain function remains a challenging problem that will require repeated validation through a variety of different studies and experiments.
While the results of the present analysis are preliminary, they do suggest that the selected SNPs may influence characteristics of brain connectivity during a working memory task in the alpha and theta bands. If these selected SNPs truly are significant factors of memory-related brain function in healthy individuals, and given that many of these SNPs are known to discriminate healthy individuals from AD cases, further studying these SNPs in healthy subjects may lead to identification of new targets for treatment, or provide insight into the genetic mechanisms of AD.
Stated as a test of the correlation of similarities or distances, the Mantel test is a geometrically motivated method for association testing. We have here shown that the Mantel test with ridge kernel similarity measures is in fact equivalent to the score tests for the fixed effects, ridge regression, and random effects models for particular choices of similarity on the covariate space X. In high-dimensional settings, the random effects score test has been shown to have reasonable performance when a large number of covariates contribute small independent effects, but with real genetic data the validity of this assumption is questionable, as it is known that there exists correlation and more complicated relationships between SNPs, and the proportion of unrelated SNPs for a given trait is difficult to know a priori. Ridge regression is designed to both address collinearity between covariates and reduce the influence of noisy covariates in prediction settings. In predictive modeling, tuning parameters are often selected via cross-validation to minimize squared error loss. This is sensible and practical when the end goal is prediction, but has two major drawbacks for inference. Firstly, one must compute the post-selection null distribution of the test statistic; secondly, the best predictive model is not necessarily the highest powered for null hypothesis testing. The topic of post-selection inference has received increasing attention in recent years, leading to important developments for general post-selection inference Lee et al. (2016), and for post-selection estimation of heritability Gorfine et al. (2017), although these methods still rely on CV and minimizing squared error loss to select the tuning parameters.
We have here proposed the adaptive Mantel test as a procedure to simultaneously test across a range of tuning parameters as an alternative to other selection methods for testing. The adaptive Mantel method is also comparatively simple to describe and implement, and can naturally accomodate selecting across different families of kernels (or models). As a tool for high-dimensional inference, the AMT is a straightforward and flexible method for quickly testing the strength of association between different features of the data, and can be used as a sanity check before proceeding with more complicated modeling.
There are a number of other generalizations and extensions that can be implemented within the framework of the Mantel test. A necessary extension for application purposes is the inclusion of adjustment covariates. Specifically, suppose is a matrix of covariates on subjects, and we wish to test the association of and adjusting for using the Mantel test. A straightforward solution is to apply a restricted maximum likelihood approach, for which and are each separately regressed on , and then the Mantel test is performed with and replaced by their corresponding residuals. Future work is also needed to characterize the class of metrics that admit a likelihood model, and describe the mapping of a metric in this class to its associated model, which could have important implications for kernel selection and geometric interpretations of model-based tests.
- Bertolino et al. (2006) Bertolino, A., Blasi, G., Latorre, V., Rubino, V., Rampino, A., Sinibaldi, L., Caforio, G., Petruzzella, V., Pizzuti, A., Scarabino, T., Nardini, M., Weinberger, D. R., and Dallapiccola, B. (2006). Additive effects of genetic variation in dopamine regulating genes on working memory cortical activity in human brain. Journal of Neuroscience, 26(15):3918–3922.
- de los Campos et al. (2013) de los Campos, G., Vazquez, A. I., Fernando, R., Klimentidis, Y. C., and Sorensen, D. (2013). Prediction of complex human traits using the genomic best linear unbiased predictor. PLoS genetics, 9(7):e1003608.
- Ge et al. (2017) Ge, T., Chen, C.-Y., Neale, B. M., Sabuncu, M. R., and Smoller, J. W. (2017). Phenome-wide heritability analysis of the uk biobank. PLoS genetics, 13(4):e1006711.
- Gorfine et al. (2017) Gorfine, M., Berndt, S. I., Chang-Claude, J., Hoffmeister, M., Le Marchand, L., Potter, J., Slattery, M. L., Keret, N., Peters, U., and Hsu, L. (2017). Heritability estimation using a regularized regression approach (herra): Applicable to continuous, dichotomous or age-at-onset outcome. PloS one, 12(8):e0181269.
- Kwee et al. (2008) Kwee, L. C., Liu, D., Lin, X., Ghosh, D., and Epstein, M. P. (2008). A powerful and flexible multilocus association test for quantitative traits. The American Journal of Human Genetics, 82(2):386–397.
- Lee et al. (2016) Lee, J. D., Sun, D. L., Sun, Y., Taylor, J. E., et al. (2016). Exact post-selection inference, with application to the lasso. The Annals of Statistics, 44(3):907–927.
Liu et al. (2007)
Liu, D., Lin, X., and Ghosh, D. (2007).
Semiparametric regression of multidimensional genetic pathway data: Least-squares kernel machines and linear mixed models.Biometrics, 63(4):1079–1088.
- Mantel (1967) Mantel, N. (1967). The detection of disease clustering and a generalized regression approach. Cancer research, 27(2 Part 1):209–220.
- Martins-Filho and Yao (2006) Martins-Filho, C. and Yao, F. (2006). A note on the use of v and u statistics in nonparametric models of regression. Annals of the Institute of Statistical Mathematics, 58(2):389–406.
- Omelka and Hudecová (2013) Omelka, M. and Hudecová, Š. (2013). A comparison of the mantel test with a generalised distance covariance test. Environmetrics, 24(7):449–460.
- Onton et al. (2005) Onton, J., Delorme, A., and Makeig, S. (2005). Frontal midline eeg dynamics during working memory. Neuroimage, 27(2):341–356.
- Pan (2011) Pan, W. (2011). Relationship between genomic distance-based regression and kernel machine regression for multi-marker association testing. Genetic epidemiology, 35(4):211–216.
- Robert and Escoufier (1976) Robert, P. and Escoufier, Y. (1976). A unifying tool for linear multivariate statistical methods: the rv-coefficient. Applied statistics, pages 257–265.
- Sauseng et al. (2005) Sauseng, P., Klimesch, W., Schabus, M., and Doppelmayr, M. (2005). Fronto-parietal eeg coherence in theta and upper alpha reflect central executive functions of working memory. International Journal of Psychophysiology, 57(2):97–103.
- Simons and Spiers (2003) Simons, J. S. and Spiers, H. J. (2003). Prefrontal and medial temporal lobe interactions in long-term memory. Nature reviews neuroscience, 4(8):637–648.
- Székely et al. (2007) Székely, G. J., Rizzo, M. L., Bakirov, N. K., et al. (2007). Measuring and testing dependence by correlation of distances. The annals of statistics, 35(6):2769–2794.
- Tzeng et al. (2009) Tzeng, J.-Y., Zhang, D., Chang, S.-M., Thomas, D. C., and Davidian, M. (2009). Gene-trait similarity regression for multimarker-based association analysis. Biometrics, 65(3):822–832.
- Vogler et al. (2014) Vogler, C., Gschwind, L., Coynel, D., Freytag, V., Milnik, A., Egli, T., Heck, A., De Quervain, D. J., and Papassotiropoulos, A. (2014). Substantial snp-based heritability estimates for working memory performance. Translational psychiatry, 4(9):e438.
- Xu et al. (2017) Xu, Z., Xu, G., and Pan, W. (2017). Adaptive testing for association between two random vectors in moderate to high dimensions. Genetic Epidemiology.
- Yang et al. (2011) Yang, J., Lee, S. H., Goddard, M. E., and Visscher, P. M. (2011). Gcta: a tool for genome-wide complex trait analysis. The American Journal of Human Genetics, 88(1):76–82.