One important task in biology is to identify signalling pathways caused using a stimulus defined by an interventional experimental setting. The resulting cascade of chemical reactions on proteins in a cell has to be understood and quantified. A well known and publicly available data set for this kind of set up is the data collected and initially analyzed by Sachs2005. It involves measurements on proteins on cells under different experimental conditions. Often the underlying pathways are learned from these cell measurements using graphical models on directed acyclic graphs (DAGs), which are also called Bayesian networks (Pearl1988, Lauritzen1996, KollerFriedman2009). Here the associations between the random variables are modelled by a statistical model on a graph with node set and edge set . Using Markov assumptions and allowing only for acyclic graphs, the joint density of can be expressed as product of conditional densities
where is the parent set of , i.e the set of nodes which have an edge going to . More precisely . Note that can be the empty set. The set at observed values of the parent nodes is denoted by .
Sachs2005 used a single DAG to model the pooled data collected under all experimental conditions. Since the Sachs data involves only continuous data, Gaussian DAGs or undirected Gaussian graphical models (dempster1972covariance) are often utilized. Gaussian DAG models are often constructed using a structural equation model (SEM) (mulaik2009linear, kaplan2008structural).
Many approaches have been followed to learn the underlying pathway network from this data. To name a few, friedman2008sparse used graphical lasso to construct an undirected graph for the pooled data, while ellis2008learning discretized the data and used a graphical model for discrete data. peterson2015bayesian constructed a selection prior for different experimental conditions to conduct a joint Bayesian MCMC analysis. Earlier luo2011bayesian developed a non graphical Bayesian hierarchical approach, where the regression coefficients for each experimental condition are linked by a prior. scutari2013identifying
used bootstrapping methods to identify significant edges for the Sachs data. A penalized maximum likelihood estimator of the interventional Markov equivalence classes under linear Gaussian assumptions was derived inhauser2015jointly. castelletti2018learning, castelletti2019objective use an objective Bayes approach to learn the Markov equivalence classes in a Gaussian DAG for the Sachs data. ramsey2018fask
introduced a fast adjacency skewness (FASK) algorithm, which orients edges derived from a fitted undirected network by incorporating skewness properties of the variables. Genetic algorithms have also been applied to this data set (jose2020bayesian).
Many of the pathway learning approaches assume that there is an underlying joint Gaussian model available which allows the conditional densities in (1) as Gaussian. However this might not be true for the data considered. Already hauser2015jointly mentioned that the Gaussian assumption for the Sachs data is questionable. This was also noticed by voorman2014graph, zhang2017mixture. voorman2014graph developed a nonparametric approach to include non Gaussian behavior in a graphical model, while zhang2017mixture build a Bayesian network using a mixture copula on dimensions to model the conditional distribution of a node given the observed values of the parents to accommodate the non Gaussianity of the data and the pooling over the different experimental conditions. Here he followed the approach of elidan2010copula, who was the first to use a copula based representation of the conditional density in (1).
One special feature of the Sachs data is that there exist a biological consent graph given in Figure 3 of Sachs2005. It consists of 20 edges connecting the eleven nodes of the Sachs data. This consent graph is shown in Panel (a) of Figure 3.
In this paper we want to extend the flexibility by modeling the conditional densities in Equation (1) using a copula based approach. For this we suggest the use of vine copulas (joe2014dependence, czadobook), which have become very popular among practitioners. One major reason for this is the enormous flexibility of this class compared to standard multivariate copula families such as the class of elliptical or Archemedian copulas. In particular the vine copula class is constructed using only bivariate copulas, which can be chosen separately and data driven. For the conditional densities needed in the DAG model we use the ones derived from a fitted vine distribution of the node and its parents . The vine distribution is chosen in such a way that the conditional density of a node given its parents is available in closed form. The choice of the vine model is further optimized such that the conditional log-likelihood is maximized when parent nodes are included in a forward selection. For this we use the approach taken in kraus2017d
derived for vine copula based quantile regression models. The forward selection of the parent nodes will also be used to identify which edges in the consent graph are supported by the data and allows for a potential reduction of the edges in the graph.
Overall this approach allows us to fit very general vine copula based Bayesian networks, thus extending the copula Bayesian network of elidan2010copula. We call the resulting models vine copula based SEMs and show the need of this novel model class for one experimental setting of the Sachs data using the consent graph. We restrict to a single experimental setting to avoid accommodating clusters present in the pooled data set arising from different experimental conditions.
This new approach allows us to quantify the effects of ignoring the non Gaussianity in the data with regard to the changes in the estimated conditional densities given the parent nodes as well as on simulated conditional quantiles based on the fitted models. In summary this extension provides an important building block for learning non Gaussian networks. It is easy to fit to a given graph, can remove nonsignificant edges in the graph and allows to quantify differences in conditional densities as well as probabilities. For one experimental setting of the Sachs we show that this approach leads to a better understanding and quantification of the graphical structure.
The paper is organized as follows: In Section 2 we introduce statistical models on DAGs including Gaussian DAGs more formally. The needed background on copulas and especially on vine copulas is given in Section 3. This includes the subclass of D-vine copulas. General vine copula based regression models are discussed in Section 4. The included D-vine regression forms the building block of the novel D-vine copula based SEMs introduced in Section 5. In Section 6 we present D-vine copula based structural equation analysis for the experimental setting of the Sachs data. In Section 7 we provide simulation based evidence for the better fit of the D-vine copula based SEM compared to the standard linear Gaussian Bayesian network fit. In Section 8 we quantify differences between the fitted models based on conditional probabilities and densities. The paper closes with a summary and an outlook to future research directions.
2 Statistical models on DAGs
To describe the behavior of many variables it is important to identify how the variables interact. For many applications it is useful to describe these interaction by a network with directed edges. Following Pearl1988 a Bayesian network is a directed acyclic graph (DAG), where each node represents a random variable
. The edges connecting the nodes indicate direct causal influence between the linked variables and their strengths are quantified by conditional probabilities or densities in the case of continuous random variables.
Since we are dealing with continuous data in this paper we start with a linear SEM. For this we assume that we have a graph for the node set and edge set with directed weighted edges. Let be an associated dimensional adjacency matrix with th element given by weight , which will be estimated. Further if and only if , i.e. there is an edge . Let be a
dimensional multivariate normal random vector with zero mean vector and diagonal covariance matrix. Then a linear SEM for is defined as
It follows that is multivariate normal with zero mean vector and covariance matrix satisfying . Using conditioning arguments the factorization (1) follows, where the the conditional densities are univariate normal densities. In general such a factorization is equivalent to the Markov assumption with respect to the graph (see Theorem 3.27 of Lauritzen1996).
Since the graph is acyclic there is at least one permutation of the numbers 1 to such that for all . Such a permutation is called a topological order. One such topological order resulting from the consent graph in Figure 3 applied to the eleven protein variables on the logarithmic scale
of the Sachs data is given in Table 1.
Reordering the columns and rows of according to the topological order allows to express A as a strict upper triangular matrix and thus (2) can be expressed recursively using the topological order. Utilizing (1) and the topological order we can therefore express the joint density of the Sachs data with the eleven protein variables based on the consent graph as
Hence instead of estimating one eleven dimensional density we can estimate eleven univariate conditional densities, where each of them has at most three conditioning variables. We use the special case of a linear Gaussian Bayesian network (LGBN) as a reference model, where the weights of the conditional densities derived from the SEM (2) are estimated. In particular it satisfies
for regression parameters
and residual variancesfor . The corresponding maximized log-likelihoods as well as AIC and BIC statistics are given in Table 2 for the experimental setting of the Sachs data. We see that the protein contributes more to the overall fit compared with the other proteins, while does the opposite. This is expected in this experimental setting since we have activation of the levels for and inhibition for .
Since the SEM (2) and the LGBN (4) induce marginal normality an inspection of the marginal histograms of the Sachs experiment shows skewed and multimodal histograms (see first column of Figure A4 of Appendix A1) thus indicating that the reference model might be inadequate. To extend beyond the LGBN we discuss now a copula based approach. This will allow us to construct nonlinear non Gaussian SEMs.
3 Copulas and vine distributions
For the analysis of large multivariate data sets multivariate statistical models are required, which can adequately describe not only the data center but also their tail behavior. For this standard multivariate distributions such as the multivariate normal or Student distribution are insufficient. They often require that all univariate and multivariate marginal distributions are of the same type and might result in symmetric tail dependence of pairs of variables. These characteristics are often not satisfied and so the copula approach developed by Sklar allows to separate the marginal behavior from the dependence structure. In particular a -dimensional copula is a multivariate distribution function on the -dimensional hyper cube
with uniformly distributed marginals. The corresponding copula density for an absolutely continuous copula we denote by. The fundamental theorem of Sklar for a -dimensional random vector
with joint distribution functionand marginal distribution functions is given by
for some -dimensional copula . When is absolutely continuous the associated density can be expressed as
with marginal densities and copula density .
For absolutely continuous distributions the copula C is unique. Using (5) or (6) flexible multivariate distributions can be constructed if -dimensional copulas are used. By inversion of (5) we can use any -dimensional distribution function to obtain the corresponding copula. Examples for such copulas are the Gaussian and the Student copula. Note that using these copula families together with arbitrary margins results in multivariate distributions, sometimes also called meta distributions, which are much more flexible than the multivariate distribution classes used in the inversion. Another class of parametric copulas are built directly using generator functions is the class of Archimedean copulas. The Gumbel, Clayton and Frank copula families are prime examples. A nice introduction to copulas is given in genest2007 and more theoretical treatments are the books by nelsen2007introduction and Joe1997.
From (6) for we immediately can derive expression for the conditional density and distribution functions, which are needed later. In particular the conditional density and distribution function can be expressed as
Since vine copulas are built out of bivariate copulas we give some properties of bivariate copulas. For this we use pairwise dependence measures such as Kendall’s and Spearman’s , which are completely determined by the copula to measure the overall strength of the dependence. To assess the tail dependence upper and lower tail dependence coefficients can be used. It is easy to see that the Gaussian and the Frank copula do not have tail dependence, while the Student copula has symmetric tail dependence. Further the Clayton copula has no lower tail dependence while the Gumbel has upper tail dependence. For more details see for example Chapter 2 of czadobook.
To allow for a visual comparison between different bivariate copula families marginally normalized contour plots are helpful. For this we consider three different scales: the original scale , the copula scale and the marginally normalized scale (z-scale) . Here denotes the distribution function of a random variable. Comparison of contours on the copula scale for different families is difficult, since copula densities are in general unbounded at the corners of . This is not the case if one works on the z-scale. Here has margins and thus any non-elliptical contour shape indicates a deviation from a Gaussian dependence.
We now turn to estimation in a parametric setup. In a copula based model specified by (5) or (6) we have to estimate both the marginal and copula parameters. For this joint maximum likelihood estimation can be used, if the number of parameters are not too large. However it is more common to use a two step approach, by first estimating the marginal parameters based on i.i.d observations for . This can be done separately for each of the margins. In a second step pseudo copula data is formed by setting
using the estimated probability integral transforms (PIT) . Then the copula parameters are estimated based on the pseudo copula data. If parametric marginal models are used we speak of an inference for margins approach (IFM) and if the empirical distribution is applied to the margins we have a semiparametric approach. The efficiency of the IFM approach has been investigated by joe2005, while the semiparametric approach has been proposed by Genest.
In the case where parametric marginal and copula models do not fit the data, we can also use a nonparametric approach. For the margins kernel density based estimates of the distribution function can be used. For the bivariate copula case also many nonparametric estimators are available. Their efficiency have been investigated in nagler2014kernel. The results showed that bivariate kernel density based estimation is preferred and this option is also available in rvinecopulib of rvinecopulib2019 to fit the more general class of vine distributions, which are build using bivariate copulas. We now give a short exposition of this class.
While the catalogue of bivariate parametric copula families is large, this is not the case for . Therefore the aim of early research in vine copulas was to find a way to construct multivariate copulas using only bivariate copulas as building blocks. The appropriate tool to obtain such a construction is conditioning and Joe96 gave the first pair copula construction of a multivariate copula in terms of distribution functions, while Bedford3, Bedford2 independently developed constructions in terms of densities. Additionally they provided a general framework to identify all possible constructions. We shortly illustrate this construction for starting with the recursive factorization
and treat each term separately. To determine we consider the bivariate conditional density . This density has and as marginal distributions (densities) with associated conditional copula density . More specifically denotes the copula density associated with the conditional distribution of given . Applying (6) to gives
Finally direct application of (7) gives
We see that the joint three dimensional density can be expressed in terms of bivariate copulas and (conditional) distribution functions. However this decomposition is not unique, since
are two different decompositions using a reordering of the variables in (10). In these decompositions we have in general different conditional copula densities when the value of the conditioning variable is changing. This is intractable for estimation and therefore often the so called simplifying assumption is made, i.e the dependence on the conditioning value is ignored. For example we set in (15) for every choice of . Bivariate copulas are also referred to as pair copulas. Using the simplifying assumption in (15) we speak of a pair copula construction and not of a decomposition. In this setup we can estimate both marginal and copula parameters in a parametric setup or use a nonparametric approach for both margins and bivariate copulas. Note that the specification of marginal and pair copula distributions can be done independently thus we have constructed a very flexible class of three dimensional densities.
This construction principle involving only marginal distributions and pair copulas can be extended to arbitrary dimensions. For arbitrary dimension we need a building plan identifying which bivariate conditional distributions and their associated bivariate copulas are needed in the construction. This building plan is specified through a set of linked trees, where nodes in the next tree are edges of the current tree. Further the edges of the next tree have to satisfy the so called proximity condition. This tree structure was called by Bedford3, Bedford2 a vine tree structure. The corresponding edges identify the pair copulas needed to construct the -dimensional density called a regular (R-) vine distribution. Since the number of such vine tree structures grows super exponentially (morales2011count), we need a greedy algorithm such as the one developed by dissmann2013selecting. Under the simplifying assumption parameter estimation can be performed in a stepwise fashion over the pair copula terms in a tree. This approach was developed by aas2009pair and applied to financial data. There are two simple sub classes of R-vine distributions. The first one is called a D-vine distribution and uses as allowed tree structures only a path of all nodes in the tree. The class of C-vines occur in the case that all trees are stars with a root node in the center.
For a detailed introduction to pair copula constructions and the resulting vine distributions consult czadobook and joe2014dependence. The use of such vine copula based models has grown tremendously. aas2016pair and czadonagler2022 give reviews and point to current developments.
4 Vine copula based regression models
After constructing multivariate distributions we are interested in constructing a vine copula based regression model to allow for nonlinear regression effects. In a first approach chang2019prediction, chang19a, cooke2019vine build a vine distribution for the the covariates only and then add the response to the vine tree structure such that the resulting structure remains a vine tree structure and allows to specify the conditional response distribution in closed form. In this way numerical integration over the covariates is avoided. This first focus on the joint distribution of the covariates is not natural in the context of a regression problem, therefore an alternative is to start with the response as first node and add covariates one by one in such a way that again the resulting structure is a vine tree structure and the associated density of the response given the covariates is available in closed form. The adding a covariates stops if the conditional (penalized) log-likelihood does not increase any more when a further covariate is added. This approach was first developed for D-vine models by kraus2017d and later extended to certain R-vine structures by zhu2021simplified. tepegjozova2021nonparametric considered both D- and C-vines, but generalized the procedure to look two steps ahead before adding a new covariate. While the D- and C-vine based regression methods are feasible in large dimensions, the search for R-vines based regression is restricted to smaller dimensions because of the huge number of allowed R-vine tree structures.
5 D-vine copula based structural equation models
We now extend the LGBN model (4) to allow for non linear effects of the parents on a node. For this we recall the factorization (1) and formulate for the conditional density of the node given its parents a vine copula based regression model. If we use the D-vine based regression formulation discussed in Section 4 then we can express the conditional density of node given its parents , where is the number of parents of node for as
The order of the parent nodes in the D-vine based conditional density (18) is apriori not fixed, we will however later choose the order as obtained by using the methods developed by kraus2017d and described in Section 4. Thus we start with the most important parent node with regard to the conditional (penalized) log-likelihood of the node and continue adding parent nodes until this quantity can no longer improved or all parent nodes are included.
Since the forward selection of the parent nodes also includes the possibility of not including all parents this gives a way to remove edges from the starting consent graph. These edges are then not supported by the data. We will now apply this class of copula based SEMs to the experimental setting of the Sachs data.
6 Analysis of experimental setting from the Sachs data
For this analysis we used the data from https://science.sciencemag.org/content/308/5721/523/tab-figures-data on the logarithmic scale. Recall that using the consent graph given in Figure 3 results in the density decomposition (2) for the eleven protein variables. Next we want to utilize the D-vine copula based SEM introduced in Section 5. This entails that every density term in (2) is modeled by using the D-vine regression introduced in Section 4. Using (18) it follows that (2) can be rewritten as
Note that if a node has more than one parent, the order of the D-vines might change depending on the chosen marginals and copulas. In this case we use numbers to denote the position in the order of the D-vine instead of the explicit nodes. Hence, ”” denotes the first parent of the modeled node, ”” the second parent in the D-vine regression and so on. For our analysis we have 845 observations available for the experiment. In this experiment the reagent Anti-CD3/CD28 111General perturbation: Activates T cells and induces proliferation and cytokine production. Induced signaling through the T cell receptor (TCR), activated ZAP70, Lck, PLC-g, Raf, Mek, Erk, and PKC. The TCR signaling converges on transcription factors NFkB, NFAT, and AP-1 to initiate IL-2 transcription (Sachs2005) and AKT 222Inhibitor specific perturbation: Binds inositol pleckstrin domain of AKT and blocks AKT translocation to the membrane where normally AKT becomes phosphorylated and active [median inhibitory concentration (IC50) 0 5 mM]. Inhibition of AKT and phosphorylation of AKT substrates are needed to enhance cell survival (Sachs2005) are having an activation effect on the protein and and an inhibitor effect on according to Figure 2 of Sachs2005.
To estimate the parameters of the marginal and copula terms in Equation (6) we follow the IFM approach discussed in Section 3. So in a first step we have to estimate the margins and then form appropriate pseudo data as in (9). In the second step we fit a D-vine copula to the node as first variable and its parents as remaining variables in the described forward approach.
To see how the fit of the model varies, when using different marginals (M) or copulas (C
) we fit three different copula based SEMs. For the first model we use kernel density estimates as margins and allow for parametric and non-parametric copulas (MC), for the second one we use Gaussian mixture margins since we observe some multi nodal margins and the same set of copulas (MC) and for the last model we only allow for Gaussian margins and Gaussian copulas (MC). A non parametric pair copula is chosen, when it fits better than any implemented parametric family and is implemented in vinereg of vinereg.
The fit of the different marginals is shown in Figure A4 of Appendix A1 comparing marginal histograms to the fitted density, qq plots and checking if the data after applying the PIT using the estimated marginal densities is approximately uniformly distributed. We observe that using Gaussian margins for the PIT in general does not result in uniformly distributed data indicating a bad fit. On the other hand, Gaussian mixture margins and kernel density estimates fit the margins very well. It is hard to distinguish between these two models. We also consider the goodness of fit measures log-likelihood, AIC and BIC of the fitted marginals. The results are displayed in Table A1 in Appendix A2 and show that Gaussian margins give the worst fit.
To support the use of copula based SEMs instead of linear Gaussian ones, we give normalized contour plots in Figure A8 of Appendix A1. From there we see non elliptical shapes in the lower triangular panels regardless which marginal models were used, thus pointing towards the need to capture non Gaussian dependence.
In the next step we use the fitted marginals to transform the data to the copula scale by applying the estimated PIT. On these new data sets we then fit copulas according to three defined models.
Using kernel density estimates for the PIT and then fit a D-vine regression model for each node with at least one parent results in a log-likelihood of 2316.23, an AIC of -4228.73 and a BIC of -3272.06 of the copula terms. Using Gaussian mixture margins and the same set of copula families results in a log-likelihood of 2292.34, an AIC of -4189.49 and a BIC of -3252.98 of the copula terms. The last model where Gaussian margins and Gaussian copulas are used worsens the three measures to a log-likelihood of 1497.84, an AIC of -2971.68 and a BIC of -2914.80 of the copula terms. More detailed results are displayed in Table A2 and Table A3 on the copula scale as well as contour plots of the fitted copulas in Figure A12 of Appendix A3. They show that Gaussian pair copulas are only seldom chosen in the non Gaussian SEMs. Further the parametric pair copula families contained in vinereg are not sufficient and many nonparametric pair copulas (tll) are selected. We see strong similarities on the fitted copula scale between the MC model and the MC model.
If a parent is not selected in the D-vine regression for the node , this indicates that the copula density is very close to the bivariate independence copula density ( for every ) and thus the edge is not supported by the data. This implies that in both copula based non Gaussian SEMs only 17 of the 20 dependencies given by the biological consent graph are present for this experimental setting (see Panel (b) of Figure 3). The removed edges are and . The deletion of implies that there is no pathway from to . This might be a consequence of the inhibiting effect on . Further the deletion of implies that can only be reached from over , which is sensible since in this experiment is activated. The removal of is also sensible since is inhibited in this setting. The edges and also were identified as edges to be removed from the consent graph in the original analysis of Sachs2005.
The number of supported edges decreases for the MC model further. Here only 12 edges are modelled. The removal of these additional four edges (colored red in Panel (b) of Figure 3) is questionable, since as we show now that the fit of the MC is inferior to the other two copula based SEM’s.
We now include the marginal fit for the different SEMs based on Equation (18) to compute overall fit measures, which allow us to also compare to the performance of the standard LGBN. These are contained in Table 7.
(a) Log-likelihood of the conditional densities
(b) AIC of the conditional densities
(c) BIC of the conditional densities
(d) Number of (effective) parameters of the conditional densities
We observe that the LGBN and the D-vine based SEM with Gaussian margins and copulas result in almost identical results for all three goodness of fit measures. This is reasonable as both of them specify a multivariate Gaussian distribution following KollerFriedman2009 and MORALES2008699. The parameters of the conditional Gaussian distribution given by each node in the two models are displayed in Table S3 in the Supplement S1. Many similarities between these two models are visible even though all possible dependencies are modelled in the LGBN, as it was fitted optimizing the conditional log-likelihood with all parents of a node, whereas this is not the case for the MC.
Comparing the two models to the MC model and the MC model we can see a strong improvement in all goodness of fit measures in the models with parametric and non-parametric copulas. This is no surprise since we have already seen that Gaussian mixture margins and kernel density estimates provide a much better fit than Gaussian margins and allowing flexibility in the choice of the pair copula families provides an advantage. We observe that between the non Gaussian models only slight differences are visible mostly due to the large mumber of effective parameters needed for the kernel density estimates, which influences the BIC.
7 Model validation
Up to this point we have only compared the goodness of fit measures between the four model but have not evaluated if they really reproduce structures visible in the observed data. For this we sampled 845 times from each model by starting in the node
and sampling it using its fitted marginal density. We then follow the topological order, i.e for each variable in this order we simulate data for the node using the already simulated data for its parents. For the LGBN we use that we can express the conditional distribution as a conditional normal distribution. To sample from the D-vine copula based SEMs we utilize the algorithm presented inbevacqua2017multivariate.
To compare the induced pairwise dependence structures we consider the scatter plots for node pairs of the observed data and of the simulated data sets from the different models in Figure 8. Looking at the scatter plots of the MC model and the MC model we observe almost no differences. This does not surprise as we have already seen that their fitted copulas are almost identical. Comparing the scatter plots to the observed scatter from the data set we see that they closely match. For all variable pairs the shapes, even more complicated ones like between the node pairs or , are similar from the simulated data of the two fitted non Gaussian SEMs and the original data.
However comparing the scatter plots of the MC model and the LGBN instead, it is clear that both of them have problems modeling more complicated shapes as they are only able to model elliptical shaped dependencies. It seems that the MC model does this a little bit better looking at the pairs plots of and which are closer to the ones from the original data set than the ones from the LGBN. Since in Figure 8 we only considered bivariate dependence properties, we check if high/low values in several variables at once appear in a similar frequency in the data and the simulated data. Hence, we sum up over the variables in each sample and then fit kernel density estimates to the resulting data. We can then compare the kernel density estimates fitted to the sum over the data and simulated data from the investigated models. The results are displayed in Figure 9.
Comparing the fitted kernel density estimates of the sum over all nodes we see that especially the ones fitted to the simulated data from the MC model and the MC model are very close to the original data set and are only slightly shifted to the left. On the other hand, we observe that the ones fitted to the sum of the simulated data from the MC model does not reach the height of the mode compared to the original data set whereas the LGBN overestimates the height of the mode. Hence, considering the ability of the different models to recreate the data set they are fitted on the MC model and the MC model outperform the other two.
8 Conditional density and medians for given values of parent nodes
To see the effect of allowing a more general dependence structure in the SEM (18) compared to a LGBN we investigate the behavior of the conditional density of a node given its parents for the studied models. Here we like to investigate the effect when median and extreme values of the parents are used. If a node only has one parent in the DAG, we chose the , and empirical quantile of the parent node as conditioning values. This is appropriate for the nodes and and the associated conditioning values are shown in Figure S1 of Supplement S2.
Since there is no simple equivalent of quantiles in two or three dimensional distributions a different approach had to be used for all other nodes. If a node has two or three parents, we fitted a two/three-dimensional kernel density estimate to the joint distribution of the parent nodes using the data. We then manually choose one point close to the mode of the fitted density, two in the tails and two more which are neither close to the mode nor in the tails. This is illustrated in Figure S2 for nodes with two parents and Figure S9 for nodes with three parents, respectively, in Supplement S2. Note that while six nodes have two parents, (, , , , , ), three of them (, and ), have the same set of parents, namely and , so we only obtain four cases for the values of the conditioning variables. The chosen values of the parent nodes are summarized in Table 8.
|p38, jnk, raf||pka||6.00||6.00||7.00||7.50||6.50|
In the following we will denote the points which we chose close to the the mode of the fitted parent distribution as ”Mode”, the points in the tails as ”Tail” and the points between them as ”Middle”. If a node only has one parent, we will assign the points from the and the quantile to ”Tail” and the point from the quantile to ”Mode”. Note that as in the MC model not all dependencies have been modeled, in this model sometimes the number of parents change with the model.
As earlier we sample 845 times from each node of the model given the values of the parent nodes and then fitted kernel density estimates to this sample. The resulting density plots are displayed in Figure 10. As we expect, we observe only small differences in the general shape of the conditional density plots between MC and MC. The same holds when comparing the MC model and the LGBN. However the shapes of the conditional density can be very different for the non Gaussian copula based SEMs and the Gaussian SEMs, thus illustrating very different conditional effects for the chosen parent values.
As a final visualisation of the differences on relevant conditional quantities we consider the conditional median for each node given its parents. We follow the topological order as given in Table 1. For we use a sample quantile of level denoted by as value of pip3. Next we determine the median of plc given using the D-vine quantile regression of kraus2017d, we denote this median as . For the node we determine the conditional median given using D-vine quantile regression. We denote this conditional median by . We continue in this way until all conditional medians are determined. The needed parents for the different models can be found in Table A3 of Appendix A3. We investigate three different and levels and the results are visualized in Figure 14. We see pronounced differences for the nodes and .
9 Summary and future research directions
In this paper we suggested to use a D-vine copula based SEM to analyze data which obeys a graphical structure. At least a starting graph has be known, but the data does not need to follow a joint Gaussian distribution. The appropriate copula regression model for the conditional distribution of the node given its parents is very flexible, since the bivariate building blocks of the D-vine regression can be independently chosen in addition to the marginal models for the node and its parents.
The proposed approach uses data reduction in two ways, first by the choice of the graph and second by the forward selection of parent nodes ordered by importance. Edges pointing to a node in the starting graph can be removed when in the D-vine regression of the node the corresponding parent node is not selected as a covariate. This allows for an edge selection without relying on the Gaussian assumption of the data. This approach identified three edges in the network, which are not no longer supported by the experimental conditions investigated. The removal of these edges were found to be plausible given the specific experimental conditions.
We illustrated the approach using the experimental setting of the Sachs data. Here we showed a much better fit of the copula based SEM with margins fitted by mixtures of univariate normals or nonparametric kernel density estimation and using non Gaussian pair copulas.
These D-vine copula based SEMs are easy to fit and can be applied to large networks, since D-vine regressions are feasible for several hundred covariates. In addition it is easy to simulate from the model and assess conditional densities or probabilities of the nodes given observed values of the selected parent nodes.
These D-vine copula based SEM’s can also be utilized to conduct a causal analysis using the pooled Sachs data. For this we would subdivide the data in a node specific way. For a given node we would select only observations from experimental settings which do not inhibit or activate the node. This would remove the interventional effects present in the data and such an analysis is currently investigated.
While it seems like that the knowledge of an initial graph is restrictive, the approach can be combined with any structure learning approach. In particular to allow for non Gaussian structure selection the vine copula based approach of bauer2012pair to test for conditional independence in a PC algorithm (kalisch2007estimating, spirtes2011causation) can be used and the selected graph can be fitted using the proposed D-vine copula based SEM.
Two immediate extensions of the proposed modeling approach are to allow for R-vine based SEMs or to improve the forward selection procedure by looking two steps ahead instead of a step as has been proposed by tepegjozova2021nonparametric.
A non Gaussian analysis of the complete Sachs data would require the development of efficient mixtures of vine regression models to accommodate the different clusters induced by the different experimental settings. A first step in this direction is sahin2021vine, who developed a data driven algorithm for the specification of vine distributions in a mixture setup for clustering. Such a development might further improve the fit of the experimental, since for some nodes we can observe clusters in the data (see Figure A8).
A1: Marginal fits for the copula based structural equation model
|Kernel density estimates||Gaussian mixture margins||Gaussian margins|
|Node||Log-lik.||AIC||BIC||# Par.||Log-lik.||AIC||BIC||# Par.||Log-lik.||AIC||BIC|
We see that all three measures are extremely similar for the fitted Gaussian mixture margins and kernel density estimates. For the log-likelihood and the AIC the kernel density estimates are slightly better whereas for the BIC the Gaussian mixture margins marginally outperform kernel density estimates. This difference is due to the higher number of parameters necessary to fit kernel density estimates. Compared to these two approaches fitting Gaussian margins results overall in a worse fit.
A2: Exploring pairwise dependence under different marginal specifications
A3: Copula fits for the copula based SEM
|pip2||pip3, plc||293.69||-509.66||-325.46||pip3, plc||290.73||-498.59||-302.22||pip3||88.80||-175.61||-170.87|
|p38||pkc, pka||519.59||-982.02||-846.57||pkc, pka||515.32||-977.28||-850.79||pkc||354.14||-706.28||-701.54|
|jnk||pkc, pka||86.66||-124.26||-7.99||pkc, pka||84.79||-128.34||-30.61||pkc||15.54||-29.08||-24.34|
|raf||pka, pkc||20.00||-6.88||71.59||pka, pkc||19.32||-9.59||59.28||pka, pkc||2.15||-0.30||9.18|
|mek||raf, pkc, pka||310.82||-574.83||-463.88||raf, pkc, pka||308.73||-571.78||-463.55||raf, pkc||232.86||-461.72||-452.24|
|akt||erk, pka||787.00||-1517.80||-1384.65||erk, pka||777.31||-1495.02||-1353.81||erk, pka||715.74||-1427.47||-1417.99|
|(a) MC||Pair copula||Family||# parameters||log-lik.||AIC||BIC||Est. Ken.|
|pip2 plc; pip3||tll||19.10||118.10||-198.00||-107.48||0.29|
|p38 pka; pkc||joe||1.00||1.70||-1.40||3.34||-0.01|
|jnk pka; pkc||gumbel||1.00||1.08||-0.15||4.59||0.00|
|raf pkc; pka||clayton||1.00||1.26||-0.52||4.22||0.01|
|mek pkc; raf||gaussian||1.00||4.54||-7.09||-2.35||-0.06|
|mek pka; raf pkc||gumbel||1.00||2.28||-2.55||2.19||0.01|
|akt pka; erk||bb8||2.00||122.11||-240.21||-230.73||0.26|
|(b) MC||Pair copula||Family||# parameters||log-lik.||AIC||BIC||Est. Ken.|
|pip2 plc; pip3||tll||21.12||116.59||-190.95||-90.87||0.29|
|p38 pka; pkc||joe||1.00||1.69||-1.39||3.35||-0.01|
|jnk pka; pkc||gumbel||1.00||1.10||-0.21||4.53||0.00|
|raf pkc; pka||clayton||1.00||1.21||-0.43||4.31||0.01|
|mek pkc; raf||gaussian||1.00||4.67||-7.33||-2.59||-0.06|
|mek pka; raf pkc||gaussian||1.00||2.17||-2.34||2.39||0.01|
|akt pka; erk||bb8||2.00||124.38||-244.75||-235.27||0.26|
|(c) MC||Pair copula||Family||log-lik.||AIC||BIC||Par.||Est. Ken.|
|raf pkc; pka||gaussian||1.08||-0.15||4.59||0.05||0.01|
|mek pkc; raf||gaussian||5.27||-8.55||-3.81||-0.11||-0.06|
|akt pka; erk||gaussian||184.92||-367.84||-363.10||0.60||0.26|
S1: Comparison of linear Gaussian Bayesian network and structural equation model based on a Gaussian copula with Gaussian margins