A Frequency Domain Bootstrap for General Stationary Processes

Existing frequency domain methods for bootstrapping time series have a limited range. Consider for instance the class of spectral mean statistics (also called integrated periodograms) which includes many important statistics in time series analysis, such as sample autocovariances and autocorrelations among other things. Essentially, such frequency domain bootstrap procedures cover the case of linear time series with independent innovations, and some even require the time series to be Gaussian. In this paper we propose a new, frequency domain bootstrap method which is consistent for a much wider range of stationary processes and can be applied to a large class of periodogram-based statistics. It introduces a new concept of convolved periodograms of smaller samples which uses pseudo periodograms of subsamples generated in a way that correctly imitates the weak dependence structure of the periodogram. bootstrap procedure means that cannot be mimicked by existing procedures. We show consistency for this procedure for a general class of stationary time series, ranging clearly beyond linear processes, and for general spectral means and ratio statistics. Furthermore, and for the class of spectral means, we also show, how, using this new approach, existing bootstrap methods, which replicate appropriately only the dominant part of the distribution of interest, can be corrected. The finite sample performance of the new bootstrap procedure is illustrated via simulations.



There are no comments yet.


page 1

page 2

page 3

page 4


A Frequency Domain Bootstrap for General Multivariate Stationary Processes

For many relevant statistics of multivariate time series, no valid frequ...

EstimatedWold Representation and Spectral Density-Driven Bootstrap for Time Series

The second-order dependence structure of purely nondeterministic station...

Information theoretic results for stationary time series and the Gaussian-generalized von Mises time series

This chapter presents some novel information theoretic results for the a...

Bootstrapping Whittle Estimators

Fitting parametric models by optimizing frequency domain objective funct...

Nonparametric robust monitoring of time series panel data

In many applications, a control procedure is required to detect potentia...

Reconciling the Gaussian and Whittle Likelihood with an application to estimation in the frequency domain

In time series analysis there is an apparent dichotomy between time and ...

Time series models with infinite-order partial copula dependence

Stationary and ergodic time series can be constructed using an s-vine de...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

Frequency domain bootstrap methods for time series are quite attractive because in many situations they can be successful without imitating the (potentially complicated) temporal dependence structure of the underlying stochastic process, as is the case for time domain bootstrap methods. Frequency domain methods mainly focus on bootstrapping the periodogram which is defined for any time series by


The periodogram is an important frequency domain statistic and many statistics of interest in time series analysis can be written as functions of the periodogram. Furthermore, it obeys some nice nonparametric properties for a wide class of stationary processes, which make frequency domain bootstrap methods appealing. In particular, periodogram ordinates rescaled by the spectral density are asymptotically standard exponential distributed and, moreover, periodogram ordinates corresponding to different frequencies in the interval

are asymptotically independent. This asymptotic independence essentially means that the classical i.i.d. bootstrap of drawing with replacement, as has been introduced by Efron (1979), can potentially be applied to bootstrap the periodogram, in particular the properly rescaled periodogram ordinates. Motivated by these considerations, many researchers have developed bootstrap methods in the frequency domain which generate pseudo-periodogram ordinates with the intent to mimic the stochastic behavior of the ordinary periodogram.

A multiplicative bootstrap approach for the periodogram has been investigated by Hurvich and Zeger (1987), Franke and Härdle (1992) and Dahlhaus and Janas (1996). The main idea is to exploit the (asymptotic) independence of the periodogram and to generate new pseudo periodogram ordinates by multiplying an estimator of the spectral density at the frequencies of interest with pseudo innovations obtained by an i.i.d. resampling of appropriately defined frequency domain residuals. Franke and Härdle (1992) proved validity of such an approach for estimating the distribution of nonparametric spectral density estimators for linear processes of the form



denotes an i.i.d. white noise process. Shao and Wu (2007) established validity of this procedure for the same statistic but for a much wider class of stochastic processes.

However, beyond nonparametric spectral density estimators, the range of validity of this bootstrap approach is limited. In fact, even in the special case of linear processes as given by (1.2) with i.i.d. white noise, Dahlhaus and Janas (1996) showed that the multiplicative approach fails to consistently estimate the limiting distribution of very basic statistics like sample autocovariances. This failure is due to the following: For many periodogram-based statistics of interest consistency is achieved by letting the number of frequencies at which the periodogram is evaluated increase with increasing sample size. The dependence between periodogram ordinates at different frequencies vanishes asymptotically, but the rate of this decay typically just compensates the increasing number of frequencies. This leads to the fact that the dependence structure actually shows up in the limiting distribution of many statistics of interest. Since the bootstrap pseudo-periodogram ordinates generated by the multiplicative approach are independent, this approach fails to imitate the aforementioned dependence structure of periodogram ordinates.
As a consequence, beyond the class of nonparametric spectral density estimators, validity of this frequency domain bootstrap approach can be established only for a restricted class of processes and statistics. To be precise, even in the special case of linear processes (1.2) with i.i.d. white noise, the approach works only under additional assumptions, such as Gaussianity of the time series, or for specific statistics such as ratio statistics. For nonlinear processes, even for processes with a linear structure as in (1.2) but with non-i.i.d. noise, the approach fails for most classes of statistics. Notice that the aforementioned limitations of the multiplicative periodogram bootstrap are common to other frequency domain bootstrap methods which generate independent pseudo-periodogram ordinates. The local periodogram bootstrap introduced by Paparoditis and Politis (1999) is such an example.
Thus, for a frequency domain bootstrap procedure to be successful for a wider class of statistics and/or a wider class of stationary processes, it has to take into account the dependence structure of the ordinary periodogram at different frequencies. ttempts in this direction are the approach proposed by Dahlhaus and Janas (1994), the autoregressive-aided periodogram bootstrap by Kreiss and Paparoditis (2003), and the hybrid wild bootstrap, cf. Kreiss and Paparoditis (2012). Beyond sample mean and nonparametric spectral density estimation, however, the second approach only works for linear stochastic processes which obey a finite or infinite order autoregressive structure driven by i.i.d. innovations. Although the idea behind the hybrid wild bootstrap is different, and this approach extends the validity of the frequency domain bootstrap to a wider class of statistics compared to the multiplicative periodogram bootstrap, its limitation lies, as for the approach proposed by Janas and Dahlhaus (1994), in the fact that its applicability is also restricted to linear processes.
The above discussion demonstrates that a frequency domain bootstrap procedure which is valid for a wide range of stationary stochastic processes and for a rich class of periodogram-based statistics is missing. This paper attempts to fill this gap. We propose a new bootstrap approach which is based on bootstrapping periodograms of subsamples of the time series at hand and which considerably extends the range of validity of frequency domain bootstrap methods. The method is consistent for a much wider range of stationary processes satisfying very weak dependence conditions. The new approach defines subsamples of ”frequency domain residuals” which are obtained by appropriately rescaling periodograms calculated at the Fourier frequencies of subsamples of the observed time series. These residuals together with a consistent estimator of the spectral density are used to generate subsamples of pseudo periodograms which mimic correctly the weak dependence structure of the periodogram. Aggregating such subsampled pseudo periodograms appropriately, leads to a frequency domain bootstrap approach which can be used to approximate the distribution of the corresponding statistic of interest. We establish consistency of the new frequency domain bootstrap procedure for a general class of stationary time series, ranging clearly beyond the linear process class and for general periodogram-based statistics known as spectral means and ratio statistics. Spectral means and ratio statistics include several of the commonly used statistics in time series analysis, like sample autocovariances or sample autocorrelations, as special cases. The idea underlying our approach is somehow related to that of convolved subsampling which has been recently and independently considered in a different context than ours by Tewes et al. (2017). Furthermore, and for the case of spectral means, we show how the new procedure can be used to benefit from the advantages of classical procedures to bootstrap the periodogram and at the same time to overcome their limitations. In particular, the modification of the standard procedures proposed, uses existing frequency domain methods to replicate the dominant part of the distribution of the statistic of interest and uses the new concept of convolved bootstrapped periodograms of subsamples to correct for those parts of the distribution which are due to the weak dependence of the periodogram ordinates and which cannot be mimicked by classical procedures.

The paper is organized as follows. Section 2 reviews some asymptotic results concerning the class of spectral means and ratio statistics and clarifies the limitations in approximating the distribution of such statistics by frequency domain bootstrap methods which generate independent periodogram ordinates. Section 3 introduces the new frequency domain bootstrap procedure while Section 4 establishes its asymptotic validity for the entire class of spectral means and ratio statistics and for a very wide class of stationary processes. Section 5 discusses the issue of selecting the bootstrap parameters in practice, presents some simulations demonstrating the finite sample performance of the new bootstrap procedure. Finally, all technical proofs are deferred to the Appendix of the paper.

2. Spectral Means and Ratio Statistics

We consider a weakly stationary real-valued stochastic process with mean zero, absolutely summable autocovariance function and spectral density . The periodogram of a sample from this process is defined according to (1.1). While the periodogram is well-known to be an inconsistent estimator of the spectral density , integrated periodograms form an important class of estimators which are consistent under suitable regularity conditions. For some integrable function the integrated periodogram is defined as

which is an estimator for the so-called spectral mean . A further interesting class of statistics is obtained by scaling a spectral mean statistic by the quantity . In particular, this class of statistics, also known as ratio statistics, is defined by

For practical calculations the integral in is commonly replaced by a Riemann sum. This is usually done using the Fourier frequencies based on sample size which are given by , for , with


The approximation of , and respectively of , via Riemann sum is then given by


The construction of is motivated by the fact that integrals over are usually approximated by Riemann sums at frequencies , for . Since the periodogram (and in many applications the function as well) is axis-symmetric, it will be useful to employ a Riemann sum over where for each used frequency the corresponding negative frequency is also used. This is ensured by using with instead of .
Different statistics commonly used in time series analysis belong to the class of spectral means or ratio statistics. We give some examples.

Example 2.1.

  1. The autocovariance of at lag can be estimated by the empirical autocovariance which is an integrated periodogram statistic. This is due to the fact that choosing it follows from straightforward calculations that as well as .

  2. The spectral distribution function evaluated at point is defined as for . The corresponding integrated periodogram estimator is given by

  3. The autocorrelation of at lag can be estimated by the empirical autocorrelation which in view of (i) is a ratio statistic, that is .

We summarize the assumptions imposed on the process and the function :

Assumption 1.

  1. Assumptions on : Let

    be a strictly stationary, real-valued stochastic process with finite eighth moments, mean zero, autocovariance function

    fulfilling and spectral density satisfying . Furthermore, let the fourth order joint cumulants of the process fulfil

    and the eighth order cumulants be absolutely summable, that is

  2. Assumptions on : Let be a square–integrable function of bounded variation.

Notice that the summability conditions imposed on the autocovariance function , imply boundedness and differentiability of , as well as boundedness of the derivative of . Under the conditions of Assumption 1, and some additional weak dependence conditions, it is known that is a consistent estimator for

and that the following central limit theorem holds true:


with , where


and where

is the fourth order cumulant spectral density, cf. Rosenblatt85, Chapter III, Corollary 2.

Remark 2.2.

In the literature the function is sometimes assumed to be even, i.e. for all . In this case the first part

of the limiting variance takes the form

However, we allow for general functions since this restriction to even functions is not necessary.
Approaches to directly estimate the integral involving the fourth order cumulant spectral density and which can potentially be used to estimate the variance

have been proposed by some authors; see Tanuguchi (1982), Keenan (1987) and Chiu (1988). However, the empirical performance of such estimators is unknown. Alternatively, Shao (2009) proposed an approach to construct confidence intervals for

based on self normalization which bypasses the problem of estimating the variance term .
The second summand of the limiting variance simplifies if the process is a linear process, that is, if admits the representation (1.2) for some square-summable sequence of coefficients and some i.i.d. white noise process with finite fourth moments. Denoting and , it then follows by a standard calculation the decomposition . In this case, the second summand of the limiting variance from (2.2) can be written as


Regarding the class of ratio statistics the situation is somehow different. Notice first that


In view of (2.2) and the fact that we then have that


where with


It can be easily verified that . This implies that for the class of linear processes considered in Remark 2.2 and by the same arguments to those used there, we get that

. That is, the variance of the limiting Gaussian distribution (

2.6) simplifies for linear processes to and it is, therefore, not affected by the fourth order structure of the process. Note that this simplification for ratio statistics holds no longer true when considering nonlinear processes.

3. The frequency domain bootstrap procedure

A common way to approximate the distribution of based on a given sample is to use the following multiplicative periodogram bootstrap (MPB) approach: Approximate by the distribution of


where is a consistent (e.g. kernel-type) estimator of based on , and


where the bootstrap random variables

are constructed in the following way, cf. FraHar92: For , let

be i.i.d. with a discrete uniform distribution on the set of rescaled ”frequency domain residuals”

where and . Then, for , set . The construction with ensures that is an even function which preserves an important property of the periodogram . To see the motivation for approach (3.1), notice that is a Riemann approximation for . Moreover, the bootstrap random variables are supposed to mimic the behavior of the peridogram ordinates . Notice that based on the fact that has an asymptotic standard exponential distribution for , an alternative approach would be to generate the ’s as i.i.d. exponentially distributed random variables with parameter . Since the asymptotic behavior of MPB procedure is identical for both approaches used to generate the pseudo innovations , see Dahlhaus and Janas (1996), for simplicity we concentrate in the following on the approach generating the ’s as standard exponential distributed random variables.

Under mild conditions

has a limiting normal distribution with mean zero and variance

as defined in (2.3). The proof is given in Proposition 4.3. For the special case of linear processes this asymptotic result for integrated periodograms was obtained in DahlhausJanas96. Moreover, ShaoWu07 obtain a similar result for smoothed periodogram spectral density estimators and a general class of nonlinear processes. Hence, for a quite general class of stationary processes, the bootstrap approach (3.1) correctly captures the first summand but fails to mimic the second summand of the limiting variance from (2.2). Consequently, the approach asymptotically only works in those special cases where vanishes. For the class of linear processes – where takes the form from (2.5) – this is the case, for example, if the innovations are Gaussian which implies . Another example for linear processes occurs when one is estimating a spectral mean identical to zero, i.e , which is the case for ratio statistics, cf. the discussion in the previous section. But except for these special cases the approach via fails in general, especially if the underlying process is not a linear process driven by i.i.d. innovations.

In the following, we propose an alternative bootstrap quantity, denoted by , that is capable of mimicking the distribution of including the entire limiting variance of as given in (2.2). The modified approach is based on periodograms calculated on subsamples. More precisely, the periodogram of each subsample of length of the original time series is first calculated. These periodograms are then appropriately rescaled leading to a set of subsamples of “frequency domain residuals”. A number of randomly selected subsamples of such residuals is then chosen which multiplied with the spectral density estimator at the appropriate set of Fourier frequencies, leads to the generation of subsamples of pseudo periodograms that correctly imitate the weak dependence structure of the ordinary periodogram. The convolution of these subsampled pseudo periodograms is then used for the construction of a bootstrap quantity which is used to approximate the distribution of of interest. We show in Theorem 4.4 that this new approach is consistent under very mild conditions on the underlying stochastic process. Furthermore, the same approach can be used in a hybrid type bootstrap procedure, which is denoted by , and which is composed from both and and corrects for the drawbacks of the MPB procedure based solely on .

We will now first state the proposed basic bootstrap algorithm and discuss some of its properties in a series of remarks.

Convolved Bootstrapped Periodograms of Subsamples

  1. Let be some consistent spectral density estimator, for instance, the estimator used in (3.2). Choose a block length , assume that with . For , with , let

    where are Fourier frequencies associated with subsample series of length , i.e. , with , cf. (2.1) for the definition of .

  2. Define the rescaled frequency domain residuals of the subsampled periodogram as,

    where .

  3. Generate random variables i.i.d. with a discrete uniform distribution on the set . For , define

    and let

  4. Approximate the distribution of by the distribution of the bootstrap quantity


    Furthermore, approximate the distribution of by that of


Some remarks are in order.

Remark 3.1.
  1. The above approach differs from subsampling since we do not calculate (respectively ) solely on a subsample of the observed time series. Similarly, the pseudo periodograms , , as well as (respectively ), are not calculated on randomly selected subsamples of the observed time series. In fact, our procedure generates new pseudo periodograms of subsamples , , using the spectral density estimator based on the entire time series and the set of appropriately defined subsamples of frequency domain residuals and .

  2. Similar to the MPB, the rescaling in (2) ensures that , i.e., , which avoids an unnecessary bias at the resampling step.

  3. Notice that can be written as , where

    Comparing the above expression with that of the MPB given by

    clarifies that, apart from the different number of observations on which and are based, the essential difference lies in the way the pseudo innovations in the two approaches are generated. In particular, while the ’s are independent the pseudo random variables are not. In fact, due to resampling the entire subsample of frequency domain residuals , in generating the subsampled pseudo periodogram , , the weak dependence structure of the periodogram within the subsamples is preserved.

  4. Note that the rescaling quantity used in (2) is actually itself a spectral density estimator which is based on averaging periodograms calculated over subsamples. Such estimators have been thoroughly investigated by many authors in the literature; Bartlett (1948), (1950), Welch (1967); see also Dahlhaus85. We will make use of some of the results obtained for this estimator later on. This implies that we could also set in (3) of the bootstrap procedure in order to generate the subsamples of pseudo periodograms . However, allowing for the use of any consistent spectral density estimator makes the bootstrap approach much more flexible. Depending on the data sample at hand, either appropriate parametric or nonparametric estimators may be chosen from case to case.

As mentioned in the Introduction the new approach to bootstrap the statistics (respectively ) can also be used in a hybrid type procedure in the case of spectral maens in order to correct for the shortcomings of standard frequency domain bootstrap methods, like for instance of the MPB procedure. In the following we formulate such a hybrid frequency domain bootstrap approach which, as we will see in Theorem 4.5, makes the MPB procedure consistent for the entire class of spectral means and ratio statistics.

A hybrid periodogram bootstrap (spectral means)

  1. Let be generated according to (3.3) – that is, as in the bootstrap procedure generating subsamples of pseudo periodograms – and let be defined as in (3.1), where the i.i.d. random variables , , and are independent.

  2. Approximate the distribution of by the empirical distribution of the rescaled bootstrap quantity

    with the standardization quantity , and with a variance correction factor , where

Remark 3.2.

The idea of the above hybrid procedure is to base bootstrap approach on and to perform a variance correction to achieve consistency for general stationary processes. As argued before, has a limiting normal distribution with mean zero and variance . It is not able to mimic the fourth order cumulant term . As can be seen by inspecting the proof of Proposition 4.3, this is due to the fact that the bootstrap ordinates and are (conditionally) independent whenever , while periodogram ordinates at different frequencies are in general dependent. To be precise, it is well–known that for different fixed frequencies periodogram ordinates are only asymptotically independent but for finite sample size correlated. When using Fourier frequencies instead of fixed frequencies, this effect of interdependence does not vanish asymptotically but shows up in form of summand in the limiting variance. The other summand in turn evolves from the sum of the variances of the periodogram ordinates at different frequencies without taking dependencies into account; and this contribution to the limiting variance is mimicked correctly by .
In approach we therefore use solely to mimic . The factor standardizes and the factor establishes the appropriate variance. We have . Since the random variables and are conditionally independent, is asymptotically normal with mean zero and variance . Hence, the asymptotic variance has to be reduced by to achieve consistency. To be precise, the contribution of to has to be removed since, then, contributes the part and the necessary correction for . This is done in step of the hybrid bootstrap algorithm: Note that by the proof of Theorem 4.4

in probability, as

, so that in probability. On the one hand, this implies asymptotic consistency of the bootstrap approach for general stationary processes. On the other hand, is also the appropriate variance correction for finite sample sizes because represents exactly the contribution of to , as can be seen by the proof of Theorem 4.4 .

Remark 3.3.

The modification of the multiplicative periodogram bootstrap proposed leads to a potential advantage of this procedure compared to the one based solely on convolving bootstrapped periodograms of subsamples. To elaborate, consider for instance the case of spectral means and recall that in the modification proposed, the estimator of the first part of the variance is delivered by the multiplicative periodogram bootstrap while the procedure based on convolved bootstrapped periodograms is only used in order to estimate the second part of the variance. This, however, implies that the hybrid bootstrap estimator of the distribution of the statistic of interest becomes less sensitive with respect to the choice of the subsampling parameter , compared to the procedure based exclusively on convolved bootstraped periodograms. This aspect will be demonstrated in Section 5.

The next remark deals with the question how the above hybrid bootstrap procedure can be implemented in practice.

Remark 3.4.

The bootstrap variances and used in step of the hybrid bootstrap approach are not readily available when implementing the algorithm in practice. Hence, they have to be replaced by estimators. One possibility is via Monte Carlo: Repeat step multiple times to generate replications
of . Then calculate the variance estimator

where . Proceed analogously to replace in and denote the estimated version by . Finally, obtain rescaled bootstrap replications , via

We conclude this section by a modification of the hybrid bootstrap proposal which is appropriate to approximate the distribution of ratio statistics. This modification is necessary due to the normalizing term ; see the definition of .

A hybrid periodogram bootstrap (ratio statistics)

  1. Let






    Finally, let .

  2. Approximate the distribution of by the distribution of , where

    where , , and

Remark 3.5.

Notice that and which makes the centering of and in (1) obsolete.

Remark 3.6.

As for spectral means the distribution of can be evaluated by Monte Carlo. In particular, we may repeat step multiple times to generate replications , , , , and we can then calculate , . Here, , and . The empirical distribution of the replications , , can then be used to estimate the distribution of .

4. Bootstrap Validity

In order to establish consistency for the aforementioned bootstrap approaches, we impose the following assumption on the asymptotic behavior of the subsample block length :

Assumption 2.

  1. For block length and , it holds and , such that and , as .

  2. It holds and , as .

For the bootstrap approaches considered, we assume uniform consistency for the spectral density estimator , that is:

Assumption 3.

The spectral density estimator fulfils

This is a common and rather weak assumption in the spectral analysis of time series because the rate of convergence is not specified.
The upcoming proposition states an asymptotic result for bootstrap approach . DahlhausJanas96 proved this result for the special case of linear processes as given by (1.2). However, since this restriction is not necessary for the bootstrap quantities, we derive the limiting distribution under the aforementioned assumptions for general stationary processes.
The following two preliminary results will be useful for the proof of consistency for bootstrap approach .

Lemma 4.1.

Under the conditions of Assumption 1 and 2 it holds for the Fourier frequencies , :

Lemma 4.2.

Let Assumptions