Estimation and Inference of Time-Varying Auto-Covariance under Complex Trend: A Difference-based Approach

1 Introduction

Our discussion begins with a heteroscedastic nonparametric regression model

Y_{i} = μ (t_{i}) + \sqrt{V (t_{i})} ε_{i}, i = 1, \dots, n,

(1)

where $Y_{i}$ are the observations, $μ$ is an unknown mean function, $t_{i} = i / n$ are the design points, $i = 1, \dots, n, ε_{i}$

are the errors with mean zero and variance

1,

and

V

is the variance function. Historically, it has been assumed that the errors

ε_{i}

are independent. Variance estimation in regression models with the unknown mean has traditionally been a rather important problem. Accurate variance estimation is required for the purpose of, for example, construction of confidence bands for the mean function, testing the goodness of fit of a model, and also in order to choose the amount of smoothing needed to estimate the mean function; see e.g. [29], [13], [15], and [18]. An extensive survey of the difference sequence approach to estimate the variance in the nonparametric regression setting when the variance function is only a constant can be found in [7].

The situation when the variance is not constant is more complicated. One of the first attempts to estimate the variance function in a regression model was made in [20] who proposed the basic idea of kernel smoothing of squared differences of observations. This idea has been further developed in [22]. [2] introduced a class of difference-based local polynomial regression-based estimators of the variance function $V (x)$ and obtained optimal convergence rates for this class of estimators that are uniform over broad functional classes. [31] obtained the minimax rate of convergence for estimators of the variance function in the model (1) and characterized the effect of not knowing the mean function on the estimation of variance function in detail. Similar approach was used to construct a class of difference-based estimators in [3] when the covariate $X \in R^{d}$ for $d > 1.$

All of the above mentioned papers only considered the case where the data are independent. However, difference-based methods have also been used to estimate variance and/or autocovariance in nonparametric regression where the errors are generated by a stationary process. The pioneering approach here was probably that of [21]

who proposed estimators based on the first-order differences to estimate (invertible) linear transformations of the variance-covariance matrix of stationary

m

-dependent errors. Here, by

m

-dependent errors we mean the errors generated by a stationary process whose autocovariance is equal to zero for any lag greater than some

m > 0.

[19] suggested second order differences to estimate the zero frequency of the spectral density of stationary processes with short-range dependence. In the case of autoregressive errors, [16] proposed

\sqrt{n}

-root consistent and, under the assumption of normality of errors, efficient estimators of the autocovariance that are also based on differences of observations. Under certain mixing conditions, [26] proposed estimating the autocovariance function by applying difference-based estimators of the first order to the residuals of a kernel-based fit of the signal. [34] provided an optimal difference-based estimator of the variance for smooth nonparametric regression when the errors are correlated. Finally, the closest to us in spirit is, probably, [30] that proposed a class of difference-based estimators for the auto-covariance in nonparametric regression when the signal is discontinuous and the errors form a stationary

m

-dependent sequence. To the best of our knowledge, the problem of auto-covariance estimation in a nonparametric regression where the errors form a non-stationary sequence while the signal is discontinuous has not been considered before.

The purpose of this article is to estimate and make inference of the time-varying covariance structure of a locally stationary time series when it is contaminated by a complex trend function with both smooth and abrupt changes. Here local stationarity refers to the slowly or smoothly evolving data generating mechanism of a temporal system ([6], [24], [37]). In time series analysis, the estimation and modelling of the auto-covariance structure is of fundamental importance in, for example, the optimal forecasting of the series ([1]), the efficient estimation of time series regression models ([17]) and the inference of time series regression parameters ([1]

). When the trend function is discontinuous, removing the trend from the time series and then estimating the auto-covariances from the residuals is not a good idea since it is very difficult to estimate the trend function near the points of discontinuity accurately. In this case, the aforementioned difference-based methods offer a good alternative. In this paper, we adopt a difference-based local linear regression method for the aforementioned time-varying auto-covariance estimation problem. The method can be viewed as a nonparametric and non-stationary extension to

[30]. It is shown that the uniform convergence rate of auto-covariance function estimation for the difference-based method under complex trend is the same as that of auto-covariance function estimation of a zero-mean time series when the number of points of discontinuity as well as the jump sizes diverge to infinity at a sufficiently slow rate. Therefore, asymptotically, the accuracy of auto-covariance function estimation will not be affected by the complex trend when the difference-based nonparametric method is used.

Making inference of the auto-covariance functions is an important task in practice as practitioners and researchers frequently test whether certain parametric or semi-parametric models are adequate to characterize the time series covariance structure. For instance, one may be interested in testing whether the auto-covariance functions are constant over time so that a weakly stationary time series model is sufficient to forecast the future observations. There is a rich statistical literature on the inference of auto-covariance structure of locally stationary time series, particularly on the testing of weak stationarity of such series. See for instance

[25], [12], [8], [23], [9] and [11]

. To our knowledge, only constant or smoothly time-varying trend were considered in the aforementioned literature of covariance inference. In this paper, simultaneous confidence bands (SCB) with asymptotically correct coverage probabilities are constructed for the time-varying auto-covariance functions when estimated by the difference-based local linear method. The SCB serves as an asymptotically correct tool for various hypothesis testing problems of the auto-covariance structure under discontinuous mean functions. A general way to perform such hypothesis tests is to estimate the auto-covariance functions under the parametric or semi-parametric null hypothesis and then check whether the fitted functions can be fully embedded into the SCB. As the auto-covariance functions can be estimated with faster convergence rates under the parametric or semi-parametric null hypothesis, the aforementioned way to perform the test achieves correct Type-I error rate asymptotically. The tests are of asymptotic power 1 for local alternatives whose uniform distances from the null are of the order greater than that of the width of the SCB, see Theorem 2 in

[35] for instance. We also propose a simulation-assisted bootstrapping method for the practical construction of the SCB.

The paper is organized as follows. In Section 2, we introduce the model formulation and some assumptions on $x_{i}$ and $ε_{i}^{k}$ . Section 3 presents the asymptotic theory for local estimate $β_{k} (\cdot)$ . Practical implementation including a suitable difference lag and tuning parameters selection procedure, estimation of covariance matrices as well as an assisted bootstrapping method are discussed in Section 4. In Section 5, we conduct some simulation experiments on the performance of our SCBs. A real data application is provided in Section 6. The proofs of the main results are deferred to the Appendix.

2 Model formulation

Consider model:

y_{i, n} = μ_{i, n} + x_{i, n}

(2)

where $μ_{i, n} := μ (t_{i})$ is a mean function or signal with unknown change points, $y_{i, n} = y (t_{i})$ and $x_{i, n} = x (t_{i}) := G (\frac{i}{n}, F_{i})$ is a zero-mean locally stationary process with $t_{i} = i / n, i = 1, . . ., n$ . Eq. 2 covers a wide range of nonstationary linear and nonlinear processes, see [37] for more discussion. We shall omit the subscript $n$ in the sequel if no confusion arises. Let $ζ_{i}, i \in Z$

, be independent identically distributed (i.i.d.) random variables, and define

F_{i} = (. . ., ζ_{i - 1}, ζ_{i})

. Then, the process

{x_{i}}

can be written as

x_{i} = G (t_{i}, F_{i}),

where $G (\cdot, \cdot)$ is a measurable function such that $G (t, F_{i})$ is well defined for all $t \in [0, 1]$ . In this paper, we focus on the case that there exists $0 = a_{0} < a_{1} < \dots < a_{d} < a_{d + 1} = 1$ such that

μ (t) = d \sum j = 0 μ_{j} (t) 1_{{a_{j} \leq t < a_{j + 1}}},

where $μ_{j} (t)$ is a Lipschitz continuous function over $[a_{j}, a_{j + 1})$ and $d$ is the total number of change points. Till the end of this paper, we will always assume $d = d_{n} = O (n^{α})$ and the maximal jump size $Δ_{n} = {max}_{1 \leq j \leq d} | μ_{j} (a_{j}) - μ_{j} (a_{j}^{-}) | = O (n^{β})$ with $0 \leq α, β < 1$ .

To estimate the second order structure of the process Eq. 2, we introduce the approach based on the difference sequence of a finite order applied to the observations $y_{i}$ . Assuming that the number of observations is $n + k$ , this difference-based covariance estimation approach would define simple squared differences of the observations, i.e., $ρ_{i}^{k} := ρ_{k} (t_{i - k}) = (y_{i} - y_{i - k})^{2}, i = k + 1, k + 2, . . ., k + n$ . Notice that for any fixed $t$ , ${ξ_{i} (t) := G (t, F_{i})}_{i \in Z}$ is a stationary process. For convenience, let us denote $s_{i} = t_{i - k} = \frac{i - k}{n} .$ Then, $γ_{k} (s_{i})$ is the $k$ th order autocovariance function of the process ${x_{i}}$ at the fixed time $s_{i};$ in other words, $γ_{k} (s_{i}) := C o v (x_{i}, x_{i - k}), k = 0, 1, . . .$ . If $k = 0$ , then $γ_{0} (s_{i})$ turns out to be the variance of $x_{i}$ .

We first introduce some notation that will be used throughout this paper. For any vector

v = (v_{1}, v_{2}, . . ., v_{p}) \in R^{p}

, we let

| v | = (\sum_{i = 1}^{p} v_{i}^{2})^{1 / 2}

. For any random vector

V

, write

V \in L^{p} (p > 0)

∥ V ∥_{p} := E (| V |^{p})^{1 / p} < \infty

. Denote

C^{p} ([0, 1])

as the function space on [0,1] of functions that have continuous first

p

derivatives with integer

p > 0

. Now, we need the following definition and assumptions:

Definition 1 (Physical dependence measure).

Let $(ζ_{j}^{'})_{j \in Z}$ be an i.i.d. copy of $(ζ_{j})_{j \in Z}$ . Then, for any $j \geq 0$ , we denote $F_{j}^{'} = (F_{- 1}, ζ_{0}^{'}, ζ_{1}, . . ., ζ_{j}) .$ The physical dependence measure for a stochastic system $L (t, F_{j})$ is defined as

δ_{q} (L, j) = sup t \in [0, 1] ∥ L (t, F_{j}) - L (t, F_{j}^{'}) ∥_{q} .

(3)

If $j < 0$ , let $δ_{q} (L, j) = 0$ . Thus, $δ_{q} (L, j)$ measures the dependence of the output $L (t, F_{j})$ on the single input $ζ_{0}$ ; see [32] for more details.

Assumption 1.

$δ_{8} (G, l) = O (l^{- 2})$ for $l \geq 1$ .

Assumption 2 (Stochastic Lipschitz continuity).

There exists a constant $C > 0$ , such that $∥ G (t, F_{i}) - G (s, F_{i}) ∥_{4} \leq C | t - s |$ holds for all $t, s \in [0, 1]$ and ${sup}_{t \in [0, 1]} ∥ G (t, F_{i}) ∥_{8} < \infty$ .

creftype 1 shows that the dependence measure of time series ${x_{i}}$ decays at a polynomial rate, thus indicating short-range dependence. creftype 2 means that $G$ changes smoothly over time and ensures local stationarity. Here, we show some examples of the locally stationary linear and nonlinear time series that satisfy these assumptions.

Example 1 (Nonstationary linear processes).

Let $ζ_{i}$ be i.i.d. random variables with $ζ_{i} \in L^{q}, q \geq 1$ ; let $a_{j} (\cdot), j = 0, 1, . . .,$ be $C^{1} ([0, 1])$ functions such that

G (t, F_{i}) = \infty \sum j = 0 a_{j} (t) ζ_{i - j}

(4)

is well defined for all $t \in [0, 1]$ . Clearly by [37, Proposition 2], we know that creftype 1 will be satisfied if ${sup}_{t \in [0, 1]} {| a_{l} (t) |^{2}} = O (l^{- 2})$ . Furthermore, if $\sum_{j = 0}^{\infty} {{sup}_{t \in [0, 1]} | a_{j}^{'} (t) |^{2}} < \infty$ , the stochastic Lipschitz continuity condition in creftype 2 also holds true.

Example 2 (Nonstationary nonlinear processes).

Let $ζ_{i}$ be i.i.d. random variables and consider the nonlinear time series framework

ξ_{i} (t) = R (t, ξ_{i - 1} (t), ζ_{i}),

(5)

where $R$ is a measurable function and $t \in [0, 1]$ . This form has been introduced by [37] and [36]. Suppose that for some $x_{0}$ , we have ${sup}_{t \in [0, 1]} ∥ R (t, x_{0}, ζ_{i}) ∥_{q} < \infty$ for $q > 0$ . Denote

χ := sup t \in [0, 1] L (t), where L (t) = sup x \neq y \frac{∥ R (t, x, ζ_{0}) - R (t, y, ζ_{0}) ∥_{q}}{| x - y |} .

It is known from [37, Theorem 6] that if $χ < 1$ , then Eq. 5 admits a unique locally stationary solution with $ξ_{i} (t) = G (t, F_{i})$ and the physical dependence measure satisfies that $δ_{q} (G, j) \leq C χ^{j}$

, which shows geometric moment contraction. Hence, the temporal dependence with exponentially decay indicates

creftype 1 holds with

q = 8

. Further by [37, Proposition 4], we conclude that creftype 2 holds for

q = 4

sup t \in [0, 1] ∥ M (G (t, F_{0})) ∥_{q} < \infty, where M (x) = sup 0 \leq t < s \leq 1 \frac{∥ R (t, x, ζ_{0}) - R (s, x, ζ_{0}) ∥_{q}}{| t - s |} .

Due to the local stationarity of the process ${x_{i}}$ , we have the following lemma which shows that, under mild assumptions, the auto-covariance of ${x_{i}}$ also exhibits polynomial decay.

Lemma 1.

Suppose Assumptions 1 and 2 hold, then we have $γ_{k} (t) = O (k^{- 2})$ for $k \geq 1$ .

With the above result, we can choose $h$ large enough such that $γ_{k} (t) \approx 0$ for $k \geq h$ . Next we focus on the difference series $ρ_{i}^{k}$ for $k = 1, . . ., h$ and we always assume $h = h_{n} = O (n^{1 / 4} log n)$ . By Eq. 2, we know that

	$ρ_{i}^{k}$	$= (x_{i} - x_{i - k})^{2} + (μ_{i} - μ_{i - k})^{2} + 2 (x_{i} - x_{i - k}) (μ_{i} - μ_{i - k})$
	$:$	$= α_{i}^{k} + λ_{i}^{k} + θ_{i}^{k} .$		(6)

Recall $s_{i} = (i - k) / n$ for $i = k + 1, . . ., . k + n$ and notice that $α_{i}^{k} := α_{k} (s_{i}) = (x_{i} - x_{i - k})^{2}$ is the squared difference of two locally stationary processes. Therefore, it is also a locally stationary process. As a result, we can define

α_{i}^{k} = (x_{i} - x_{i - k})^{2} = β_{k} (s_{i}) + ε_{i}^{k}, k = 1, . . ., h,

(7)

where $β_{k} (\cdot)$ is the unknown trend function and $ε_{i}^{k} := ε_{k} (s_{i})$ is a zero-mean process. Then $ε_{i}^{k}$ can be written as

ε_{i}^{k} = H_{k} (s_{i}, F_{i}),

(8)

where $H_{k}$ is a measurable function similar to $G$ . With Eq. 7, if the trend function is smooth, one can easily obtain the estimator of $β_{k} (\cdot)$ . Now, we introduce the following conditions.

Assumption 3.

For each $k = 0, . . ., h - 1$ , we assume that the $k$ th order autocovariance function $γ_{k} (t) \in C^{3} ([0, 1])$ .

Assumption 4.

The smallest eigenvalue of

σ_{k} (t)

is bounded away from 0 on

[0, 1]

for

k = 1, . . ., h

, where

σ_{k} (t) = {\infty \sum j = - \infty C o v (H_{k} (t, F_{0}), H_{k} (t, F_{j}))}_{k}^{1 / 2},

(9)

and $σ_{k}^{2} (t)$ represents the long-run variance of $ε_{k} (t)$ for each fixed $t \in [0, 1]$ .

Assumption 5.

A kernel $K (\cdot)$ is a symmetric proper density function with the compact support $[- 1, 1]$ .

creftype 3 guarantees that the trend function $β_{k} (\cdot)$ changes smoothly for each $k = 1, . . ., h$ and is three-times continuously differentiable over $[0, 1]$ . creftype 4 prevents the asymptotic multicollinearity of regressors. creftype 5 allows popular kernel functions such as Epanechnikov kernel. Now substituting Eq. 7 to Section 2, we have

ρ_{i}^{k} = β_{k} (s_{i}) + ε_{i}^{k} + λ_{i}^{k} + θ_{i}^{k} .

(10)

Since the length of the series ${ρ_{i}^{k}}_{i = k + 1}^{k + n}$ is $n$ , we reset the subscript with respect to $i$ as ${ρ_{i}^{k}}_{i = 1}^{n}$ and therefore the time point turns out to be $t_{i} = i / n$ for $i = 1, . . ., n$ . Similar notations are used for series ${ε_{i}^{k}}, {λ_{i}^{k}}$ and ${θ_{i}^{k}}$ . By creftype 3 and the Taylor’s expansion on $β_{k} (\cdot)$ , it is natural to estimate $β_{k} (t)$ using the local linear estimator as follows:

({ˆ β}_{k, b} (t), {ˆ β}_{k, b}^{'} (t)) = arg min c_{0}, c_{1} \in R [n \sum i = 1 [ρ_{i}^{k} - c_{0} - c_{1} (t_{i} - t)]^{2} K_{b} (t_{i} - t)],

(11)

where $t_{i} = i / n$ with $i = 1, . . ., n$ and $K_{b} (\cdot) = K (\cdot / b)$ is a kernel function, $b = b_{n}$ is the bandwidth satisfying $b \to 0$ and $n b \to \infty$ . Since Eq. 11 is essentially a weighted least squares estimate, we can write the solution of Eq. 11 as

{ˆ β}_{k, b} (t) = n \sum i = 1 ω_{n}^{b} (t, i) ρ_{i}^{k},

(12)

where $ω_{n}^{b} (t, i) = K_{b} (t_{i} - t) \frac{S_{2}^{b} (t) - (t_{i} - t) S_{1}^{b} (t)}{S_{2}^{b} (t) S_{0}^{b} (t) - [S_{1}^{b} (t)]^{2}}$ with $S_{j}^{b} (t) = \sum_{i = 1}^{n} (t_{i} - t)^{j} K_{b} (t_{i} - t)$ , $j = 0, 1, 2$ . The time domain of $t$ is fixed over $[0, 1]$ and $ω_{n}^{b} (t, i), n (t - b) \leq i \leq n (t + b)$ is the weight given to each observation.

Next, we will establish the following two lemmas that are useful in establishing asymptotic properties of proposed estimators. Their proofs are given in the Appendix.

Lemma 2.

Suppose Assumptions 1-2 hold, then we have $δ_{4} (H_{k}, l) = O (l^{- 2})$ for $0 < l < k$ and $δ_{4} (H_{k}, l) = O (l^{- 2}) + O ((l - k + 1)^{- 2})$ for $l \geq k$ , where $k = 1, . ., h$ .

Lemma 3.

Suppose Assumptions 1-3 hold, then we have $∥ H_{k} (t, F_{i}) - H_{k} (s, F_{i}) ∥_{2} \leq C | t - s |$ and ${sup}_{t \in [0, 1]} ∥ H_{k} (t, F_{i}) ∥_{4} < \infty$ .

3 Main Results

3.1 Asymptotic theory

By creftype 3 and for $l = 0, 1, . . .$ , define

	$Q_{n, l}^{k} (t)$	$= \frac{1}{n b} n \sum i = 1 {(\frac{t_{i} - t}{b})}^{l} K (\frac{t_{i} - t}{b}),$		(13)
	$R_{n, l}^{k} (t)$	$= \frac{1}{n b} n \sum i = 1 ρ_{i}^{k} {(\frac{t_{i} - t}{b})}^{l} K (\frac{t_{i} - t}{b}) .$		(14)

Then Eq. 11 can be expressed as

⎛ ⎝ \begin{matrix} {ˆ β}_{k, b} (t) b {ˆ β}_{k, b}^{'} (t) \end{matrix} ⎞ ⎠ = {(\begin{matrix} Q_{n, 0}^{k} (t) & Q_{n, 1}^{k} (t) Q_{n, 1}^{k} (t) & Q_{n, 2}^{k} (t) \end{matrix})}^{- 1} (\begin{matrix} R_{n, 0}^{k} (t) R_{n, 1}^{k} (t) \end{matrix}) := [Q_{n}^{k} (t)]^{- 1} R_{n}^{k} (t) .

(15)

Let

μ_{l} = \int_{R} x^{l} K (x) d x and ϕ_{l} = \int_{R} x^{l} K^{2} (x) d x, l = 0, 1, . . . .

Now, we will construct SCBs for $β_{k} (\cdot), k = 1, . . ., h$ .

Theorem 1.

Suppose that Assumptions 1-5 hold and further assume that
(1) $σ_{k} (t)$ is Lipschitz continuous on [0,1].
(2) $α + 2 β \leq 2 / 5$ .
(3) $log (n) / (n^{3 / 5 - β} b) + n b^{5} log (n) \to 0$ .
Then, for each $k = 1, . . ., h$ , we have

	$P [\sqrt{\frac{n b}{ϕ_{0}}} sup t \in T ∣ ∣ σ_{k}^{- 1} (t) {{ˆ β}_{k, b} (t) - β_{k} (t)} ∣ ∣ - B_{K} (m^{}) \leq \frac{u}{\sqrt{2 log (m^{})}}]$
	$= exp {- 2 exp (- u)},$

as $n \to \infty$ , where $T = [b, 1 - b], m^{*} = 1 / b$ and

B_{K} (m^{*}) = \sqrt{2 log (m^{*})} + \frac{1}{\sqrt{2 log (m^{*})}} log (\frac{1}{π} \sqrt{\frac{1}{4 ϕ_{0}} \int_{- 1}^{1} | K^{'} (u) |^{2} d u}) .

Let us comment on the conditions listed in Theorem 1. Condition (1) shows the smoothness of $σ_{k} (t)$ . Condition (2) indicates that the change-point number and size can both go to infinity but at a slow rate. The assumption $n b^{5} log (n) \to 0$ in Condition (3) is an undersmoothing requirement that reduces the bias of the estimators to the second order.

Notice that $E (ρ_{i}^{k}) = β_{k} (t_{i}) + E (λ_{i}^{k}) = 2 γ_{0} (t_{i}) - 2 γ_{k} (t_{i}) + {~ Δ}_{i}^{2}$ , where ${~ Δ}_{i}^{2} = O (1 / n^{2})$ when there is no change point between observations $y_{i}$ and $y_{i - k}$ , ${~ Δ}_{i}^{2} \leq Δ_{n}^{2} = O (n^{2 β})$ when there exists at least a change point on $μ (\cdot)$ . However, the estimate of $λ_{i}^{k}$ can be viewed as a negligible term (see Eq. 21 in the proof of Theorem 2). With the previous discussion in mind, we can define

	${ˆ γ}_{0} (t) = \frac{1}{2} {ˆ β}_{h, b_{h}} (t),$
	${ˆ γ}_{k} (t) = \frac{1}{2} [{ˆ β}_{h, b_{k}} (t) - {ˆ β}_{k, b_{k}} (t)], k = 1, . . ., h - 1,$

where $b_{h}$ and $b_{k}$ are the bandwidths for estimators ${ˆ γ}_{0} (t)$ and ${ˆ γ}_{k} (t)$ , respectively. Making it easy to distinguish, here we use the different notations for the bandwidths which will be selected by some criterion (see Section 4.4). Notice that we require the same bandwidth $(b_{k})$ to compute the estimator of $γ_{k} (t)$ . With the above results, the SCB for $γ_{0} (\cdot)$ is straightforward.

Corollary 1.

With the conditions in Theorem 1, we have


	$= exp {- 2 exp (- u)} .$

Furthermore, to facilitate the SCB for $γ_{k} (\cdot), k = 1, . . ., h - 1$ , we will consider a linear combination of $β_{k} (\cdot)$ . First, define ${˜ H}_{k} = (H_{h} (t, F_{i}), H_{k} (t, F_{i}))^{⊤}$ and a $2$ by $2$ matrix

Σ_{k}^{2} (t) = \infty \sum j = - \infty C o v ({˜ H}_{k} (t, F_{0}) {˜ H}_{k} (t, F_{j})) .

(16)

We also denote ${˜ β}_{k} (t) = (β_{h} (t), β_{k} (t))^{⊤}$ as a two-dimensional vector, $C = (1, - 1)^{⊤}$ and $β_{C, k} (t) = C^{⊤} {˜ β}_{k} (t) .$ The natural estimators for ${˜ β}_{k} (t)$ and $β_{C, k} (t)$ are ${ˆ β}_{k} (t) = ({ˆ β}_{h, b_{k}}, {ˆ β}_{k, b_{k}})^{⊤}$ and ${ˆ β}_{C, k} (t) = C^{⊤} {ˆ β}_{k} (t) = {ˆ β}_{h, b_{k}} - {ˆ β}_{k, b_{k}}$ , respectively. Furthermore, let $σ_{C, k}^{2} (t) = C^{⊤} Σ_{k}^{2} (t) C,$ similar to Theorem 3 in [38]. At this point, we can obtain the following result.

Corollary 2.

Suppose that the smallest eigenvalue of $σ_{C, k} (t)$ is bounded away from 0 on $[0, 1]$ for $k = 1, . . ., h - 1.$ Moreover, we assume that all of the conditions of Theorem 1 are valid. Then, we have (i)

	$P [\sqrt{\frac{n b_{k}}{ϕ_{0}}} sup t \in T ∣ ∣ σ_{C, k}^{- 1} (t) {{ˆ β}_{C, k} (t) - β_{C, k} (t)} ∣ ∣ - B_{K} (m^{}) \leq \frac{u}{\sqrt{2 log (m^{})}}]$
	$= exp {- 2 exp (- u)},$

as $n \to \infty$ . (ii) Furthermore, one can easily deduce the SCB for $γ_{k} (\cdot)$ , $k = 1, . . ., h - 1$ ,


	$= exp {- 2 exp (- u)} .$

Remark 1.

It is noteworthy to mention that for estimating $β_{C, k} (t)$ , we use the same bandwidth $b_{k}$ ; therefore, the entire estimator ${ˆ β}_{C, k} (t) = {ˆ β}_{h, b_{k}} (t) - {ˆ β}_{k, b_{k}} (t)$ depends on only a single tuning parameter (bandwidth $b_{k}$ ). This enables us to achieve the conclusion of Corollary 2(i) based on the result of Theorem 1. As a result, Corollary 2(ii) also holds true due to this fact.

After constructing SCBs for the second-order structure $γ_{k} (\cdot)$ , the following theorem states that ${ˆ γ}_{k} (t)$ are consistent estimators for $γ_{k} (t)$ uniformly in $t$ for all $k = 0, . . ., h - 1$ .

Theorem 2.

Under Assumptions 1-5 and suppose conditions

α + 2 β \leq \frac{2}{5}, \frac{log (n)}{n^{3 / 5 - β} b} + n b^{5} log (n) \to 0

hold true. Then, we have

sup t \in T ∣ ∣ {ˆ β}_{k, b} (t) - β_{k} (t) ∣ ∣ = O_{P} (χ_{n}), k = 1, . . ., h,

where $χ_{n} = b^{2} + \frac{log (n)}{n^{3 / 5 - β} b} + \sqrt{\frac{log (n)}{n^{1 - 2 β} b}} + \sqrt{\frac{1}{n^{1 - α - 4 β} b}}$ .

This theorem implies the uniform consistency of ${ˆ β}_{k, b} (\cdot) .$ Additionally, due to the relationship between $β_{k}$ and $γ_{k},$ we can also easily obtain the following consistency result for ${ˆ γ}_{k} (\cdot) .$

Corollary 3.

With the conditions in Theorem 2, we have

sup t \in T | {ˆ γ}_{k} (t) - γ_{k} (t) | = O_{P} (χ_{n}), k = 0, . . ., h - 1.

4 Practical implementation

4.1 Selection of the difference lag

Note that for any fixed time $t$ , ${ˆ β}_{k, b_{k}} (t) = 2 {ˆ γ}_{0} (t) - 2 {ˆ γ}_{k} (t)$ and recall that $γ_{k} (t) \approx 0$ when $k \geq h$ , where $h$ is a large value that has been chosen in advance. Hence, we know that if $k \geq h$ , ${ˆ β}_{k, b_{k}} (t) \approx 2 {ˆ γ}_{0} (t)$ is practically invariant with respect to $t$ as $k$ increases. This fact suggests the following bandwidth selection procedure.

First, for any fixed $t$ , we choose a large enough value $h_{0} (t)$ and select $k = h_{0} (t) .$ Next, we calculate ${ˆ β}_{k, b_{k}} (t)$ . Then, by successively decreasing the value of $k$ and considering $k = h_{0} (t) - 1, h_{0} (t) - 2, . . .,$ we calculate the corresponding quantities ${ˆ β}_{k, b_{k}} (t)$ until ${ˆ β}_{k, b_{k}} (t)$ shows an abrupt change. At this point, the optimal difference lag for time $t$ can be selected as the current $k$ plus $1.$ Intuitively, we can interpret this through the scatterplot of $(k, {ˆ β}_{k, b_{k}} (t))$ . When the slope of the function ${ˆ β}_{k, b_{k}} (t)$ shows an obvious change, then we can choose $h^{*} (t) = k + 1$ . Following the above procedure for each time point $t_{i} = i / n, i = 1, . . ., n$ , we finally choose the optimal lag as $h = \sum_{i = 1}^{n} h^{*} (t_{i}) / n$ .

4.2 Covariance matrix estimation

To apply Corollaries 1 and 2 (ii), we need to estimate the long-run variance $Σ_{k}^{2} (\cdot)$ in Eq. 16 first. This problem is complicated but has been extensively studied by many researchers. Here we adopt the technique considered by [38].

Let $Q_{i}^{k} = \sum_{j = - m}^{m} {~ ε}_{i + j}, i = 1, . . ., n$ , where ${~ ε}_{i} = (ε_{i}^{h}, ε_{i}^{k})^{⊤}$ for $i = 1, . . ., n$ . Notice that $E ({~ ε}_{i}) = 0$ and denote $N_{i}^{k} := Q_{i}^{k} Q_{i}^{k ⊤} / (2 m + 1)$ . In the locally stationary case, we can make use of the fact that a block of ${{~ ε}_{i}}$ is approximately stationary when its length is small compared with $n$ . Hence, $E (N_{i}^{k}) \approx Σ_{k}^{2} (t_{i})$ as $m \to \infty$ and $m / n \to 0$ . Let $τ = τ_{n}$ be the bandwidth and define the covariance matrix estimator as

{˜ Σ}_{k}^{2} (t) = n \sum i = 1 {~ ω}_{n}^{τ} (t, i) N_{i}^{k}, {~ ω}_{n}^{τ} (t, i) = \frac{K_{τ} (t_{i} - t)}{\sum_{k = 1}^{n} K_{τ} (t_{k} - t)},

with $τ$ being the bandwidth. Therefore, the estimate ${˜ Σ}_{k}^{2} (t)$ is guaranteed to be positive semidefinite. The following theorems provide consistency of our covariance matrix estimate.

Theorem 3.

Assume that $Σ_{k}^{2} (t) \in C^{2} [0, 1], δ_{4} ({˜ H}_{k}, j) = O ({j log (j)}^{- 2}), m = m_{n} \to \infty, m = O (n^{1 / 3}), τ \to 0$ and $n τ \to \infty$ . Then, $(i)$ for each $k, k = 1, . . ., h - 1$ and any fixed $t \in (0, 1)$ ,

∥ ∥ {˜ Σ}_{k}^{2} (t) - Σ_{k}^{2} (t) ∥ ∥ = O (\sqrt{\frac{m}{n τ}} + \frac{1}{m} + τ^{2}),

$(i i)$ for $I = [τ, 1 - τ]$ ,

∥ ∥ ∥ sup t \in I ∣ ∣ {˜ Σ}_{k}^{2} (t) - Σ_{k}^{2} (t) ∣ ∣ ∥ ∥ ∥ = O (\sqrt{\frac{m}{n τ^{2}}} + \frac{1}{m} + τ^{2}) .

In practice, the errors ${~ ε}_{j}$ cannot be observed, thus we use ${ˆ Σ}_{k}^{2} (t) = \sum_{i = 1}^{n} {˜ ω}_{n}^{τ} (t, i) {ˆ N}_{i}^{k}$ , where ${ˆ N}_{i}^{k}$ is defined as $N_{i}^{k}$ with ${~ ε}_{j}$ therein replaced by its estimator ${^ε}_{j}$ .

Theorem 4.

Assume that conditions of Theorem 2 and conditions of Theorem 3 hold. Denote $ν_{n} = \sqrt{m log (n)} χ_{n}$ , where $χ_{n}$ is defined in Theorem 2 and further assume $ν_{n} \to 0$ . Then

sup t \in I ∣ ∣ {ˆ Σ}_{k}^{2} (t) - Σ_{k}^{2} (t) ∣ ∣ = O_{P} (ν_{n} + \sqrt{\frac{m}{n τ^{2}}} + \frac{}{1} m + τ^{2}) .

Note that $σ_{h}^{2} (t)$ is the first diagonal element of $Σ_{k}^{2} (t)$ and $σ_{C, k}^{2} (t) = C^{⊤} Σ_{k}^{2} (t) C$ . Thus, the covariance estimates in Corollaries 1 and 2 (ii) can be easily calculated via plugging in the long-run covariance matrix estimate ${ˆ Σ}_{k}^{2} (t)$ .

4.3 Simulation assisted bootstrapping method

Now we aim to apply Corollary 1 and Corollary 2 (ii) to construct the SCBs. Let ${ˆ γ}_{0}^{''} (t)$ and ${ˆ γ}_{k}^{''} (t)$ be uniformly consistent estimators of $γ_{0}^{''} (t)$ and $γ_{k}^{''} (t)$ for $k = 1, . . ., h - 1$ , respectively. Then the corresponding $(1 - α)$ th SCB with $α \in (0, 1)$ for $γ_{0} (t)$ and $γ_{k} (t)$ are

	$⎡ ⎣ {ˆ γ}_{0} (t) \pm {^σ}_{h} (t) \sqrt{\frac{ϕ_{0}}{4 n b}} (B_{K} (m^{}) - \frac{log [log (1 - α)^{- 1 / 2}]}{\sqrt{2 log (m^{})}}) ⎤ ⎦,$
	$⎡ ⎣ {ˆ γ}_{k} (t) \pm {^σ}_{C, k} (t) \sqrt{\frac{ϕ_{0}}{4 n b}} (B_{K} (m^{}) - \frac{log [log (1 - α)^{- 1 / 2}]}{\sqrt{2 log (m^{})}}) ⎤ ⎦, k = 1, . . ., h - 1.$

Due to the slow rate of convergence to Gumbel distribution, in practice, the UCB from Corollaries 1 and 2 (ii) may not have good finite-sample performances. To circumvent this problem, we shall adopt a simulation assisted bootstrapping approach.

Proposition 1.

Suppose conditions in Theorem 1 hold and also assume that $σ_{k} (t)$ is Lipschitz continuous for $k = 1, . . ., h$

. Then, on a richer probability space, there are i.i.d. standard normal distributed random variables

u_{i}

such that

	$sup t \in T \| {ˆ γ}_{0} (t) - γ_{0} (t) - Z_{0} (t) \| = O_{P} (ψ_{n}),$
	$sup t \in T \| {ˆ γ}_{k} (t)$

Estimation and Inference of Time-Varying Auto-Covariance under Complex Trend: A Difference-based Approach

Authors

Modelling Time-Varying First and Second-Order Structure of Time Series via Wavelets and Differencing

Simultaneous inference for time-varying models

Simultaneous Confidence Band for Stationary Covariance Function of Dense Functional Data

Robust Concordance Rate for A Four-Quadrant Plot

Locally adaptive factor processes for multivariate time series

Simultaneous Sieve Inference for Time-Inhomogeneous Nonlinear Time Series Regression

On the Underspread/Overspread Classification of Random Processes

1 Introduction

2 Model formulation

Definition 1 (Physical dependence measure).

Assumption 1.

Assumption 2 (Stochastic Lipschitz continuity).

Example 1 (Nonstationary linear processes).

Example 2 (Nonstationary nonlinear processes).

Lemma 1.

Assumption 3.

Assumption 4.

Assumption 5.

Lemma 2.

Lemma 3.

3 Main Results

3.1 Asymptotic theory

Theorem 1.

Corollary 1.

Corollary 2.

Remark 1.

Theorem 2.

Corollary 3.

4 Practical implementation

4.1 Selection of the difference lag

4.2 Covariance matrix estimation

Theorem 3.

Theorem 4.

4.3 Simulation assisted bootstrapping method

Proposition 1.

Estimation and Inference of Time-Varying Auto-Covariance under Complex Trend: A Difference-based Approach

Authors

Related Research

Modelling Time-Varying First and Second-Order Structure of Time Series via Wavelets and Differencing

Simultaneous inference for time-varying models

Simultaneous Confidence Band for Stationary Covariance Function of Dense Functional Data

Robust Concordance Rate for A Four-Quadrant Plot

Locally adaptive factor processes for multivariate time series

Simultaneous Sieve Inference for Time-Inhomogeneous Nonlinear Time Series Regression

On the Underspread/Overspread Classification of Random Processes

This week in AI

1 Introduction

2 Model formulation

Definition 1 (Physical dependence measure).

Assumption 1.

Assumption 2 (Stochastic Lipschitz continuity).

Example 1 (Nonstationary linear processes).

Example 2 (Nonstationary nonlinear processes).

Lemma 1.

Assumption 3.

Assumption 4.

Assumption 5.

Lemma 2.

Lemma 3.

3 Main Results

3.1 Asymptotic theory

Theorem 1.

Corollary 1.

Corollary 2.

Remark 1.

Theorem 2.

Corollary 3.

4 Practical implementation

4.1 Selection of the difference lag

4.2 Covariance matrix estimation

Theorem 3.

Theorem 4.

4.3 Simulation assisted bootstrapping method

Proposition 1.