Modeling dependence for multivariate time series is essential to statistical applications in various fields. For instance, see Patton (2012) and Brechmann et al. (2012) in finance, Smith (2015) and Smith and Vahey (2016) in economics, and Erhardt et al. (2015) in climate monitoring. Roughly speaking, there are two types of dependence embedded in multivariate time series. One is the temporal dependence within each component univariate time series. The other is the cross-sectional dependence across all the component univariate time series. Multivariate time series often presents complicated dependence structures, such as nonlinear dependence, tail dependence, as well as asymmetric dependence, which makes dependence modeling a challenging yet crucial task. A desirable feature of a multivariate time series model is being able to accommodate the complex dependence in both temporal and cross-sectional dimension.
In the literature, a copula is one of the most widely used tools for introducing flexible dependence structures among multivariate outcomes. A -dimensional copula is a multivariate distribution function on with uniform margins. By Sklar (1959)’s theorem, any multivariate distribution can be separated into its marginals and a copula
, where the copula captures all the scale-free dependence of the multivariate distribution. In particular, suppose there is a random vectorsuch that follows , we have , where is a realization of . If all the marginals of are absolutely continuous, the copula is unique.
Most existing copula-based time series models focus on the cross-sectional dependence of multivariate time series, see, for example, the semiparametric copula-based multivariate dynamic models (SCOMDY) in Chen and Fan (2006a). Under the SCOMDY framework, standard univariate time series models, such as ARMA and GARCH (Engle, 1982; Bollerslev, 1986)
, are used to capture the temporal dependence in the conditional mean and variance of each component univariate time series. A parametric copula is then used to specify the cross-sectional dependence across thestandardized innovations of all the component univariate time series. See Patton (2006), Brechmann et al. (2012), Almeida et al. (2016) and Oh and Patton (2017) for related models under the SCOMDY framework.
consider copula-based Markov chains, where copulas and flexible marginal distributions are used to specify the transitional probability of the Markov chains.Ibragimov (2009), Chen et al. (2009) and Beare (2010) study the probabilistic properties of copula-based Markov chains. See Joe (2014) for a nice presentation of copula-based Markov chains. However, most of the literature focus on first-order Markov chains using bivariate copulas, possibly due to the variety of choices and mathematical tractability in the low dimensional setting.
To extend the copula-based univariate time series model to higher-order Markov chains, a framework to generate flexible yet tractable multivariate copulas is required. A promising direction is the vine-copula (see Joe, 1996; Bedford and Cooke, 2002; Aas et al., 2009), which generates multivariate copulas based on iterative pairwise construction of bivariate copulas. See Kurowicka and Cooke (2006) and Kurowicka and Joe (2011) for more details of the vine-copula. The D-vine, which is a specially structured vine-copula, is of particular interest due to its simplicity and natural interpretation under time series setting. For example, Smith et al. (2010) and Shi and Yang (2017) employ D-vine to account for the temporal dependence in longitudinal data, and Loaiza-Maya et al. (2017) use D-vine to capture the temporal dependence in stationary heteroskedastic time series.
According to Aas et al. (2009), the density of a -dimensional random vector based on the D-vine is given by the marginal distributions of and bivariate copulas such that
where is the pdf of , and are conditional cdf of and given variables , and can be calculated recursively based on and by the algorithm in Aas et al. (2009). The parameter of the bivariate copula is denoted by and .
An example of the D-vine for is exhibited in Figure 1. The nodes in tree 1 (top) represent the probability integral transformed marginals and the edges in each tree becomes the nodes in the next tree. From left to right, the th edge in tree corresponds to the (conditional) bivariate copula that is used in
to specify the conditional joint distribution ofgiven variables . The edges of the entire D-vine indicate the bivariate copulas that contribute to the pair copula constructions. The key feature of the D-vine is that the edges of each tree only connect adjacent nodes, which makes it simple to understand and naturally interpretable for time series. Note that if represents a univariate time series, the D-vine then provides a valid univariate time series model.
Although copulas have been proposed for modeling temporal and cross-sectional dependence in the aforementioned two separate strands of studies, there are few multivariate time series models that use copulas to account for both types of dependence simultaneously. Some notable exceptions are Smith (2015) and Beare and Seo (2015), where the authors first stack the multivariate time series into a univariate time series and then design copula-based dependence structures for the resulted univariate time series. Brechmann and Czado (2014) use an R-vine to simultaneously model the temporal and cross-sectional dependence. These approaches demonstrate flexible dependence structures and show superior performance to the standard multivariate time series models, such as Vector AR, in various applications. One potential drawback is that these models are technically complicated and can be difficult to implement. For example, all the proposed methods involve a direct copula-based joint distribution of a high-dimensional vector of length , which is challenging both analytically and computationally, especially when the cross-sectional dimension is high. Another potential disadvantage is that it can be hard for these models to impose desirable (parsimonious) structures into the cross-sectional dependence, such as time-varying, and spatial or factor-structured dependence, which may further hinder their abilities in modeling high-dimensional time series such as large panel data or spatio-temporal data.
In this paper, we aim to design a simple, intuitive and flexible multivariate time series model that enables the simultaneous copula-based modeling of both temporal and cross-sectional dependence. Specifically, based on pair copula construction, we first design a semiparametric univariate D-vine time series model (uDvine) that generalizes the existing first-order copula-based Markov chain to an arbitrary-order Markov chain. In line with the SCOMDY framework, we then further propose a multivariate time series model named Copula-linked univariate D-vines (CuDvine), where a parametric copula is employed to link multiple uDvines and specify the (conditional) cross-sectional dependence.
The main contributions of this paper are two-fold. In terms of statistical modeling, thanks to the use of a novel hybrid modeling approach, the proposed CuDvine achieves a nice balance between model flexibility and (analytical and computational) tractability. As demonstrated in real data applications, CuDvine can readily handle complicated marginal behavior and temporal dependence of time series, as well as model sophisticated cross-sectional dependence structures such as time-varying and parsimonious spatio-temporal dependence. In terms of statistical theory, we give a complete treatment of model selection and estimation for both uDvine and CuDvine, where robust and computationally efficient procedures are proposed. Although the idea of using D-vine to capture temporal dependence is not new, to our best knowledge, we are the first one to systematically study the probabilistic properties of D-vine based time series and the statistical properties of its estimators.
The rest of the paper is organized as follows. Section 2 presents the uDvine and CuDvine, and investigates their probabilistic properties. In Section 3, a sequential model selection procedure and a two-stage maximum likelihood estimator (MLE) are proposed for model inference and estimation. Their statistical properties are investigated as well. Numerical experiments are conducted in Section 4 to demonstrate the flexibility of the CuDvine, and to examine the performance of the sequential model selection procedure and the two-stage MLE. Real data applications on the Australian electricity price and the Ireland wind speed are considered in Section 5, where significant improvement over traditional time series models is observed. We conclude the paper in Section 6. The supplementary material contains the proofs of the theorems and other technical materials.
2 The D-vine based Time Series Models
2.1 Univariate D-vine time series model (uDvine)
In this section, we introduce the univariate D-vine time series model (uDvine) and study its probabilistic properties. Throughout the section, we use to denote a univariate time series and we assume the time series is strictly stationary. Note that the general formula for the density of based on the D-vine is given by , which depends on marginal distributions of and bivariate copulas .
2.1.1 Model specification of uDvine
The strict stationarity of implies that the marginal distribution for all and that all bivariate copulas in the same tree must be identical, i.e. if . We call this the homogeneity condition. Thus, under the stationarity assumption, to fully specify the joint distribution of , one needs to specify a marginal distribution and bivariate copulas for tree 1 to tree . This is unrealistic when is large.
A natural solution is to ‘truncate’ the D-vine after a certain level (say tree ) and set all bivariate copulas beyond tree , i.e. , to be independent copulas111See Brechmann et al. (2012) and Brechmann and Czado (2014) for a similar idea on truncating R-vine., where . We call the univariate D-vine time series model truncated at tree the uDvine() model. As shown later in Proposition 1, the uDvine() is a -order homogeneous Markov chain. To maximize the flexibility of marginal behavior, we do not impose any parametric assumption on and only assume it to be absolutely continuous, which makes the uDvine a semiparametric time series model.
The joint distribution of based on the uDvine() can be written as
where is the bivariate copula in tree with parameter , and are the conditional cdfs of and given . By the homogeneity condition, we have and for all such that . We denote as the collection of all parameters for the bivariate copulas and denote .
For the purposes of estimation and prediction, the conditional distribution of the uDvine is needed and can be easily derived from the joint distribution. By the Markovian property of uDvine(), it can be shown that, for , the conditional pdf of takes the form
which can be shown to be a function of , and . For simplicity of notation, we denote
where can be derived222See Section §2 of the supplementary material for the derived formulas for a uDvine(2). based on the algorithm in Aas et al. (2009). Together, we have .
Similarly, it can be shown that, for , the conditional cdf of given is a function of and . To simplify notation, we denote
Unlike many “conditional” univariate time series models, such as ARMA and GARCH, the uDvine directly specifies the joint distribution of the univariate time series, instead of specifying the conditional distribution of given
. Most univariate time series models that are based on the conditional approach specify the temporal dependence via first and second order moments, which can be restrictive. On the contrary, the uDvine does not impose constraints on either the marginal behavior ofor the temporal dependence due to the use of the semiparametric D-vine. Depending on the choices of bivariate copulas in each tree, the uDvine can generate nonlinear, asymmetric, and tail dependence. The flexibility of the uDvine is demonstrated through numerical experiments in Section 4.1 and through real data applications in Section 5.
The uDvine() is a general model that nests many commonly used time series models as special cases. All the first-order copula-based Markov chains, e.g. Chen and Fan (2006b), are essentially a uDvine(1). In fact, all the stationary first-order Markov chains in , e.g. AR(1) models and ARCH(1) models in Engle (1982), are special cases of the uDvine(1). Another important special case of the uDvine() is a stationary AR() process with Gaussian innovations. Loaiza-Maya et al. (2017) show numerically that certain D-vine based time series model can generate volatility clustering effects as in GARCH model, Example 3 in Section 1 of the supplementary material gives an analytical explanation of such phenomenon.
2.1.2 Stationarity and ergodicity of uDvine
Note that under the homogeneity condition, the univariate time series generated by the uDvine() is strictly stationary. In this section, we study the probabilistic properties of the uDvine and show that under certain conditions, is ergodic. To our best knowledge, this is the first formal result on ergodicity of D-vine based time series, which extends the result of first-order copula-based Markov chains in Chen and Fan (2006b).
Under the homogeneity condition, the univariate time series generated by uDvine() is a -order homogeneous Markov chain.
Proposition 1 is in line with the Markov properties of D-vine studied in Smith (2015) and Beare and Seo (2015). By Proposition 1, if we define , the new process is a first-order homogeneous Markov chain with state space . Since the marginal distribution of the uDvine is absolutely continuous, we know that
marginally follows the uniform distribution on. As noted in Chen and Fan (2006b), the stationarity and ergodicity of and are equivalent due to the absolute continuity of the marginal distribution . Theorem 1 gives sufficient conditions for the ergodicity of and thus that of .
Under the homogeneity condition and Assumptions 1 and 2 in Section §2 of the supplementary material, is positive Harris recurrent and geometrically ergodic, thus is , which follows the uDvine().
A direct result of Theorem 1 is the -mixing property of the uDvine().
If Theorem 1 holds, uDvine is -mixing with an exponential decaying rate.
2.2 Copula-linked univariate D-vines (CuDvine) time series model
Our ultimate goal is to develop a flexible statistical model for multivariate time series. The uDvine accounts for marginal behavior and temporal dependence of the univariate time series. To fully specify a multivariate model, one also needs to take into consideration the cross-sectional dependence across all component univariate time series. Using a similar idea as the SCOMDY framework in Chen and Fan (2006a), we propose the Copula-linked univariate D-vines (CuDvine) time series model. Throughout this section, denotes a -dimensional multivariate time series, denotes the sigma field of all past information and denotes the sigma field of the past information from the th component univariate time series.
The time series is defined as a CuDvine if its component univariate time series follows a uDvine(), for , and the conditional joint distribution of given can be written as
where is a -dimensional copula with parameter that captures the conditional cross-sectional dependence given history , and are the conditional marginal distribution of given its own history .
Note that is a direct result from the conditional Sklar’s theorem (Patton, 2006). Since uDvine() is a -order Markov chain, we have . Given the marginal distribution and the parameter of the bivariate copulas in the th uDvine(), is a function of and such that
where is defined in (3) in Section 2.1.1. In the following, without loss of generality, we assume that the order of all uDvines to be .
The specification of the cross-sectional copula is flexible and can take a variety of forms depending on the applications. A popular assumption in the multivariate time series literature is that the conditional copula of given does not depend on , which implies that is a static copula . For low-dimensional applications, can be an unstructured copula such as elliptical copula or Archimedean copula. For high-dimensional applications, can be a parsimonious factor-structured or spatial-structured copula. A time-varying where the cross-sectional dependence evolves according to can also be readily implemented. Section 5 demonstrates the applications of CuDvine with a time-varying and a spatial-structured cross-sectional copula.
Note that the parametric form of is not restricted and can be any copula, this is an important difference between the CuDvine and the vine-copula based multivariate time series in Beare and Seo (2015) and Smith (2015), where both temporal and cross-sectional dependence can only be generated by D-vine copula.
One implicit assumption of CuDvine is — (A1) the conditional marginal distribution of the th component univariate time series given only depends on its own history . A1 can be restrictive in certain applications although various models under the SCOMDY framework are based on A1 and are shown to perform well in real data applications, see, for example, Chen and Fan (2006a), Patton (2006), Dias and Embrechts (2010), Almeida et al. (2016) and Oh and Patton (2017). One advantage of A1 is that it drastically reduces the number of parameters for temporal dependence from to and enables the use of two-stage MLE. Together with the parsimonious structure of the cross-sectional copula, CuDvine can easily handle high-dimensional multivariate time series such as spatio-temporal data and large panel data of stock returns.
2.2.1 Relationship with existing modeling approaches
Most existing multivariate time series models, such as the SCOMDY framework, follow a purely “conditional” modeling approach in the sense that both the temporal and cross-sectional dependence are specified via conditional distributions of given . As discussed in Section 2.1.1 and noted by Smith and Vahey (2016), the conditional approach can be restrictive in terms of modeling the marginal behavior and temporal dependence of the component univariate time series. In contrast, the copula time series models in Brechmann and Czado (2014), Beare and Seo (2015) and Smith (2015) follow a purely “joint” modeling approach in the sense that the joint distribution of all the observations of are specified directly, which helps offer great modeling flexibility. The joint approach is computationally and analytically complicated, and may be difficult to incorporate structured cross-sectional dependence such as time-varying and factor/spatial-structured dependence.
The CuDvine follows a unique “hybrid” modeling approach – the marginal behavior and temporal dependence are modeled by a joint approach via the uDvine, and the cross-sectional dependence is modeled by a conditional approach via a -dimensional copula. The D-vine based joint approach for the component univariate time series allows the CuDvine to accommodate sophisticated marginal behavior and temporal dependence, which is demonstrated later by numerical experiments and real data applications. The copula-based conditional approach enables the CuDvine to generate flexible cross-sectional dependence and makes the estimation and prediction procedure straightforward and computationally efficient as shown in Section 3. The CuDvine can readily model time-varying cross-sectional dependence and high-dimensional spatio-temporal dependence as demonstrated in Section 5. To summarize, the novel hybrid modeling approach makes the CuDvine achieve highly flexible modeling ability and remain analytically and computationally tractable.
3 Estimation and Inference
As pointed out by Aas et al. (2009), the inference for the D-vine consists of two parts: (a) the choice of bivariate copula types and (b) the estimation of the copula parameters. The same tasks apply to the uDvine and CuDvine. In Section 3.1, we discuss the model selection for the CuDvine. In particular, we propose a sequential model selection procedure for the component uDvine. In Section 3.2, we propose a two-stage MLE for the estimation of parameters in a given CuDvine.
3.1 Selection of bivariate copulas for the uDvine
To implement a CuDvine, one needs to specify the order and the bivariate copulas for each component uDvine, and one also needs to specify the cross-sectional copula . The selection of can rely on standard procedures such as AIC or BIC. Here, we focus on the model selection for the component uDvine.
Given a set of candidate copulas (say different copulas) and an order , the number of possible uDvines is , which can be quite large even for moderate and . For computational feasibility, we adapt the tree-by-tree sequential selection procedure as described in Shi and Yang (2017).
The basic procedure is as follows. We start with the first tree, selecting the appropriate copula from a given set of candidates and estimating its parameters. Fixing the selected copula and its estimated parameters in the first tree, we then select the optimal copula and estimate its dependence parameters for the second tree. We continue this process for the next tree of a higher order while holding the selected copulas and the corresponding estimated parameters fixed in all previous trees. If an independent copula is selected for a certain tree, we then truncate the uDvine, i.e. assume conditional independence in all higher order trees (see, for example, Brechmann et al., 2012). The commonly used BIC is employed for the copula selection for each tree.
3.2 Two-stage MLE for the CuDvine
Given the parametric form of the CuDvine, there are three components to be estimated: (a) the marginal distributions of the component uDvines, (b) the parameters of bivariate copulas in the component uDvines, (c) the parameter of the cross-sectional copula333Throughout this section, we assume the cross-sectional copula to be a static copula with parameter . The asymptotic result for time-varying copulas is similar but more complicated and is beyond the scope of this paper. . Throughout this section, we assume that the parametric form (i.e. the bivariate copula types for each uDvine() and the cross-sectional copula type for ) of the CuDvine is known, and we present the properties of the two-stage MLE under the correct model specification.
Denote as the observations of the multivariate time series. By differentiating (4), the conditional likelihood function of can be obtained as
where the conditional marginal distributions and are defined in Section 2.1.1 and can be derived from the th uDvine().
Based on (6), the conditional log-likelihood function is
The number of parameters to be estimated in (7) is at least even if we assume all the bivariate copulas of the uDvines are single-parameter copulas. The full likelihood estimation can be computationally expensive especially when the dimension is large. To improve computational efficiency, we adapt the two-stage maximum likelihood estimator (MLE) in the copula literature(e.g. Joe and Xu, 1996; Chen and Fan, 2006a). The basic idea is to decompose into several components and optimize each component separately.
In the first stage, for , the marginal distribution and the parameter in the uDvine are estimated using the th component univariate time series . Specifically, the marginal distribution is estimated by the rescaled empirical distribution function , where Given , the MLE for can be calculated by maximizing
where the last equality follows from (2).
In the second stage, given estimators and , the MLE for can be calculated by maximizing
where the last equality follows from (3).
3.3 Consistency and normality of the MLE
Both the first stage MLE of parameters in the uDvines and the second stage MLE of the parameter in the cross-sectional copula are essentially the so-called semiparametric two-stage estimator. A general treatment on its asymptotic properties can be found in Newey and McFadden (1994). In the following, we provide the results on consistency and normality for both and . To our best knowledge, this is the first formal treatment for asymptotic properties of two-stage MLE under the context of D-vine based time series.
3.3.1 Asymptotic properties of
Given the estimated marginal distribution , each is calculated by maximizing the log-likelihood function (8). In Chen and Fan (2006b), the authors provide asymptotic properties of such two-stage MLE when the univariate time series is generated by a first-order Markov chain based on a bivariate copula. Here, we extend the result to the uDvine(), which is an arbitrary-order Markov chain based on a D-vine.
Since the uDvine() is a generalization of the bivariate copula based first-order Markov chain in Chen and Fan (2006b), it is natural to expect that the theoretical properties of are similar to the ones in Chen and Fan (2006b).
Assume conditions C1-C5 in Chen and Fan (2006b) hold for the th uDvine, we have i.e. is consistent.
Before stating the result for asymptotic normality, we first introduce some notations for the ease of presentation. Denote , , and
, for .
Further denote , and
where . Define .
Assume conditions A1-A6 in Chen and Fan (2006b) hold for the th uDvine, we have: (1) ; (2) in distribution.
As noted in Chen and Fan (2006b), the appearance of the extra terms in is due to the nonparametric estimation of the marginal distribution , and if is known, the terms will disappear.
3.3.2 Asymptotic properties of
Given and , can be calculated by maximizing the log-likelihood function (9). Compared to , is obtained based on a log-likelihood function that depends on both the estimated infinite-dimensional functions and the extra finite-dimensional estimators . The presence of the extra is the main difference between the setting of and the setting of . However, the consistency and normality results still hold, with an extra term in the asymptotic covariance due to the presence of .
Chen and Fan (2006a) provides asymptotic properties of such second stage MLE under the SCOMDY framework, where the component univariate time series follow conditional univariate models such as ARMA and GARCH. As discussed in Section 2.2, the CuDvine is constructed via a hybrid modeling approach with the component univariate time series being semiparametric uDvines. This difference makes parts of the asymptotic result of for the CuDvine distinct from the one in Chen and Fan (2006a).
Assume conditions D and C in Chen and Fan (2006a) hold for the CuDvine, we have i.e. is consistent.
Given the true marginal distributions and true uDvine parameters , we denote where can be thought as the unobserved i.i.d. copula process generated by the cross-sectional copula . Denote and for .
We further denote , , and for . Denote and
where , and are defined in Theorem 3. Finally, denote and .
Assume conditions D and N in Chen and Fan (2006a) hold for the CuDvine, we have: (1) ; (2) in distribution.
Notice that the asymptotic result for is similar to the one for . The extra terms are introduced by the nonparametric estimation of the marginal distributions , and the extra terms are introduced by the estimation of the uDvine parameters . As observed in Newey and McFadden (1994), the estimation of does not influence the asymptotic covariance of if . In Chen and Fan (2006a), there are no terms in , due to the conditional modeling approach of the component univariate time series.
There is no closed form solution for the asymptotic covariance for the second-stage MLE. Though the standard plug-in estimator can be constructed, it will be quite complicated to implement. A practical solution to the estimation of the asymptotic covariance is parametric bootstrap, e.g. see Zhao and Zhang (2017).
4 Numerical Experiments
4.1 Flexibility of the uDvine
In this section, we demonstrate the flexibility of the uDvine in terms of how well it approximates a GARCH (Bollerslev, 1986) or GJR-GARCH process (Glosten et al., 1993). The GARCH process is one of the most widely used univariate time series models in financial markets and is able to capture the unique features observed in stock returns, such as heavy tailedness and volatility clustering. The GJR-GARCH process further introduces asymmetry to the GARCH process by allowing the conditional variance to respond differently to positive and negative stock returns, and it contains the GARCH process as a special case. Specifically, a univariate time series follows a GJR-GARCH process, if
If , then reduces to a GARCH process. We set the parameters to be for the GARCH process and for the GJR-GARCH process. According to Oh and Patton (2013), the parameters broadly match the values of estimation from the real world financial data.
We use the uDvine to model