Strongly consistent autoregressive predictors in abstract Banach spaces

1 Introduction

In the last few decades, there has been a growing interest in the statistical analysis of high-dimensional data from the Functional Data Analysis (FDA) perspective. The book by Ramsay and Silverman

RamsaySilverman05 provides an overview of FDA techniques adapted from the multivariate data context or specifically formulated for the FDA framework. The monograph by Hsing and Eubank HsingEubank15 introduces functional analytical tools useful for the estimation of random elements in function spaces. The book by Horváth and Kokoszka HorvathandKokoszka is mainly concerned with inference based on second order statistics; a central topic in this book is the analysis of functional data exhibiting dependent structures in time and space. The methodological survey paper by Cuevas Cuevas14 discusses central topics in FDA. Recent advances in the statistical analysis of high-dimensional data from the parametric, semiparametric and nonparametric FDA viewpoints are collected in the JMVA Special Issue by Goia and Vieu GoiaVieu16 .

Linear time series models traditionally arise for processing temporal linear correlated data. In the FDA context, Bosq’s monograph Bosq00

introduces linear functional time series theory. The RKHS generated by the autocovariance operator plays a crucial role in the estimation approach presented therein. In particular, the eigenvectors of the autocovariance operator are considered for projection; see also

Alvarez17 . Its empirical version is computed when they are unknown. The resulting plug-in predictor is obtained as a linear functional of the observations, based on the empirical approximation of the autocorrelation operator. This approach exploits the Hilbert space structure; its extension to the metric space context, particularly in the Banach space context, relies on a relationship (continuous embeddings) between the Banach space norm and the RKHS norm induced by the extended autocovariance operator. This is in contrast with the nonparametric regression approach, where semi-metric spaces are usually considered; see, e.g., FerratyKeilegom12 , where asymptotic normality is derived, in the regression model with functional response and covariates. In particular, a linear combination of the observed response values is considered, in the nonparametric local-weighting-based approach. Here, the weights are defined from an isotropic kernel depending on the metric or semi-metric of the space in which the regressors take their values; see, e.g., Ferraty06 , and, in particular, Ferraty02 in the functional time series framework. However, the more flexible nonparametric approach also presents some computational drawbacks, requiring the resolution of several selection problems. For instance, a choice of smoothing parameter and the kernel involved in the definition of the weights should be performed. Real-valued covariates were incorporated in the semiparametric kernel-based proposal by Aneiros-Pérez and Vieu Aneiros08 , which involves an extension to the functional partial linear time series framework; see also AneirosVieu06

on semi-functional partial linear regression. Goia and Vieu

GoiaVieu15 also adopted a semiparametric approach in their formulation of a two-term Partitioned Functional Single Index Model. Geenens Geenens11 exploited the alternative provided by semi-metrics to avoid the so-called curse of dimensionality.

In a parametric linear framework, functional time series models in Banach spaces were introduced in MasPumo10 . Strong mixing conditions and the absolute regularity of Banach-valued autoregressive processes were studied in Allam11 . Empirical estimators for Banach-valued autoregressive processes were discussed in Bosq02 where, under some regularity conditions, and for the case of orthogonal innovations, the empirical mean was shown to be asymptotically optimal, with respect to almost sure convergence and convergence of order $2$ . The empirical autocovariance operator was also interpreted as a sample mean of an autoregressive process in a suitable space of linear operators. The extension of these results to the case of weakly dependent innovations was obtained in Dehling05 . A strongly consistent sieve estimator of the autocorrelation operator of a Banach-valued autoregressive process was considered in RachediMourid03 . Limit theorems for a seasonality estimator were given in Mourid02 in the case of Banach autoregressive perturbations. Confidence regions were also obtained for the seasonality function in the Banach space of continuous functions. An approximation of Parzen’s optimal predictor in the RKHS framework was used in MokhtariMouri

to predict temporal stochastic processes in Banach spaces. The existence and uniqueness of an almost surely strictly periodically correlated solution to the first order autoregressive model in Banach spaces was derived in

Parvardeha . Under some regularity conditions, limit results were obtained for AR

D

(1) processes in Hajj11 , where

D [0, 1]

denotes the Skorokhod space of right-continuous functions on

[0, 1]

having a left limit at all

t \in [0, 1]

. Conditions for the existence of strictly stationary solutions of ARMA equations in Banach spaces with independent and identically distributed noise innovations can be found in Spangenberg13 .

In deriving strong consistency results for ARB $(1)$ component-wise estimators and predictors, Bosq Bosq00 restricted his attention to the case of the Banach space $C [0, 1]$ of continuous functions on $[0, 1]$ with the sup norm. Labbas and Mourid Labbas02 considered an ARB $(1)$ context, for an arbitrary real separable Banach space $B$ , under the construction of a Hilbert space $˜ H$ , where $B$ is continuously embedded as in Kuelbs’ lemma Kuelbs70 . Assuming the existence of a continuous extension to $˜ H$ of the autocorrelation operator $ρ \in L (B),$ they proved the strong consistency of the formulated component-wise estimator of $ρ$ and of its associated plug-in predictor in the norms of $L (˜ H)$ and $˜ H$ , respectively.

The linear time series framework in Banach spaces studied here is motivated by the statistical analysis of temporal correlated functional data in nuclear spaces, arising notably in the observation of the solution to stochastic differential or fractional pseudodifferential equations; see, e.g., Anh16a ; Anh16b . The scales of Banach spaces constituted by fractional Sobolev and Besov spaces play a central role in the context of nuclear spaces. Continuous (nuclear) embeddings usually connect the elements of these scales; see, e.g., Triebel83 . In this paper a Rigged-Hilbert-Space structure is defined, involving the separable Hilbert space $˜ H$ appearing in the construction of Kuelbs’ lemma Kuelbs70 . A key assumption here is the existence of a continuous (Hilbert–Schmidt class) embedding introducing the RKHS associated with the extended autocovariance operator of the ARB $(1)$ process, into the Hilbert space generating the Gelfand triple, equipped with a finer topology than the $B$ -topology. Under this scenario, strong consistency results are derived in the space $L (B)$ of bounded linear operators on $B$ , considering an abstract separable Banach space framework.

This paper is structured as follows. Background material and notation are first given in Section 2. Section 3 states the basic assumptions and key lemmas which are then proved in Section 4. This paper’s main strong consistency result is derived in Section 5, and examples are presented in Section 6. Closing comments are in Section 7. The results are illustrated numerically in an Online Supplement under the scenario described in Section 6.

2 Preliminaries

Let $(B, {∥ \cdot ∥}_{B})$ be a real separable Banach space equipped with the norm ${∥ \cdot ∥}_{B}$ and let $L_{B}^{2} (Ω, A, P)$ be the space of zero mean $B$

-valued random variables

X

such that

\sqrt{\int_{B} ∥ X ∥_{B}^{2} d P} < \infty .

Let $X = {X_{n} : n \in Z}$ be a zero mean $B$

-valued stochastic process on the probability space

(Ω, A, P)

such that, for all

n \in Z

X_{n} = ρ (X_{n - 1}) + ε_{n},

(1)

where $ρ \in L (B)$ denotes the autocorrelation operator of $X$ ; see Bosq00 . In Eq. (1), the $B$ -valued innovation process $ε = {ε_{n} : n \in Z}$ on $(Ω, A, P)$

is assumed to be a strong white noise uncorrelated with the random initial condition. Thus

ε

is a zero mean Banach-valued stationary process with iid components and

σ_{ε}^{2} = E {{∥ ε_{n} ∥}_{n}^{2}} < \infty

for all

n \in Z

Assume that there exists an integer $j_{0} \in {1, 2, \dots}$ such that

∥ ρ^{j_{0}} ∥_{L (B)} < 1.

(2)

Then Eq. (1) admits a unique strictly stationary solution with $σ_{X}^{2} = E {{∥ X_{n} ∥}_{n}^{2}} < \infty,$ i.e., belonging to $L_{B}^{2} (Ω, A, P),$ given by $X_{n} = \sum_{j \in Z}^{\infty} ρ^{j} (ε_{n - j}),$ for each $n \in Z$ ; see Bosq00 . Under (2), the autocovariance operator $C$ of an ARB $(1)$ process $X$ is defined from the autocovariance operator of $X_{0} \in L_{B}^{2} (Ω, A, P)$ as $C (x^{⋆}) = E {x^{⋆} (X_{0}) X_{0}}$ for all $x^{⋆} \in B^{⋆}$ , and the cross-covariance operator $D$ is given by $D (x^{⋆}) = E {x^{⋆} (X_{0}) X_{1}}$ for all $x^{⋆} \in B^{⋆}$ . Since $C$ is assumed to be a nuclear operator, then as per Eq. (6.24) on p. 156 of Bosq00 , there exists a sequence ${x_{j} : j \geq 1} \subset B$ such that, for every $x^{⋆} \in B^{⋆}$ ,

C (x^{⋆}) = \infty \sum j = 1 x^{⋆} (x_{j}) x_{j}, \infty \sum j = 1 ∥ x_{j} ∥_{B}^{2} < \infty .

If $D$ is also assumed to be a nuclear operator, then, as per Eq. (6.23) on p. 156 of Bosq00 , there exist sequences ${y_{j} : j \geq 1} \subset B$ and ${x_{j}^{⋆ ⋆} : j \geq 1} \subset B^{⋆ ⋆}$ such that, for every $x^{⋆} \in B^{⋆},$

D (x^{⋆}) = \infty \sum j = 1 x_{j}^{⋆ ⋆} (x^{⋆}) y_{j}, \infty \sum j = 1 ∥ x_{j}^{⋆ ⋆} ∥ \times ∥ y_{j} ∥ < \infty .

From Eqs. (6.45) and (6.58) on pp. 164–168 of Bosq00 , empirical estimators of $C$ and $D$ are respectively given, for all $x^{⋆} \in B^{⋆}$ and any integer $n \geq 2$ , by

C_{n} (x^{⋆}) = \frac{1}{n} n - 1 \sum i = 0 x^{⋆} (X_{i}) (X_{i}), D_{n} (x^{⋆}) = \frac{1}{n - 1} n - 2 \sum i = 0 x^{⋆} (X_{i}) (X_{i + 1}) .

Lemma 2.1 in Kuelbs70 , recalled just below, plays a key role in our approach.

Lemma 1.

If $B$ is a real separable Banach space with norm ${∥ \cdot ∥}_{B},$ then there exists an inner product $⟨ \cdot, \cdot ⟩_{˜ H}$ on $B$ such that the norm ${∥ \cdot ∥}_{˜ H},$ generated by $⟨ \cdot, \cdot ⟩_{˜ H},$ is weaker than ${∥ \cdot ∥}_{B} .$ The completion of $B$ under the norm ${∥ \cdot ∥}_{˜ H}$ defines the Hilbert space $˜ H,$ where $B$ is continuously embedded.

Denote by ${x_{n} : n \geq 1} \subset B$ a dense sequence in $B$ and by ${F_{n} : n \geq 1} \subset B^{⋆}$ a sequence of bounded linear functionals on $B$ satisfying

F_{n} (x_{n}) = {∥ x_{n} ∥}_{B}, ∥ F_{n} ∥ = 1,

(3)

such that, for all $x \in B$ ,

{∥ x ∥}_{B} = sup n \geq 1 | F_{n} (x) | .

(4)

The inner product $⟨ \cdot, \cdot ⟩_{˜ H},$ and associated norm, in Lemma 1, is defined, for all $x, y \in ˜ H$ , by

⟨ x, y ⟩_{˜ H} = \infty \sum n = 1 t_{n} F_{n} (x) F_{n} (y),

while for all $x \in B$ ,

{∥ x ∥}_{˜ H}^{2} = \infty \sum n = 1 t_{n} {F_{n} (x)}_{n}^{2} \leq {∥ x ∥}_{B}^{2},

(5)

where ${t_{n} : n \geq 1}$ is a sequence of positive numbers summing up to $1$ .

3 Main assumptions and preliminary results

In view of Lemma 1, for every $n \in Z,$ $X_{n} \in B ↪ ˜ H$ satisfies almost surely, for all $n \in Z$ ,

X_{n} = ˜ H \infty \sum j = 1 ⟨ X_{n}, v_{j} ⟩_{˜ H} v_{j},

where ${v_{j} : j \geq 1}$ is any orthonormal basis of $˜ H .$ The trace auto-covariance operator

C = E {(\infty \sum j = 1 ⟨ X_{n}, v_{j} ⟩_{˜ H} v_{j}) \otimes (\infty \sum j = 1 ⟨ X_{n}, v_{j} ⟩_{˜ H} v_{j})}

of the extended ARB $(1)$ process is a trace operator on $˜ H$

admitting a diagonal spectral representation in terms of its eigenvalues

{C_{j} : j \geq 1}

and eigenvectors

{ϕ_{j} : j \geq 1}

that provide an orthonormal system in

˜ H

. In what follows, the following identities in

˜ H

will be considered, for the extended version of ARB

(1)

process

X

. For arbitrary

f, h \in ˜ H,

	$C (f) = ˜ H \infty \sum j = 1 C_{j} ⟨ f, ϕ_{j} ⟩_{˜ H} ϕ_{j},$		(6)
	$D (h) = ˜ H \infty \sum j = 1 \infty \sum k = 1 ⟨ D (ϕ_{j}), ϕ_{k} ⟩_{˜ H} ⟨ h, ϕ_{j} ⟩_{˜ H} ϕ_{k},$
	$C_{n} (f) = ˜ H a . s . n \sum j = 1 C_{n, j} ⟨ f, ϕ_{n, j} ⟩_{˜ H} ϕ_{n, j},$		(7)
	$C_{n, j} = a . s . \frac{1}{n} n - 1 \sum i = 0 X_{i, n, j}^{2}, X_{i, n, j} = ⟨ X_{i}, ϕ_{n, j} ⟩_{˜ H}, C_{n} (ϕ_{n, j}) = ˜ H a . s . C_{n, j} ϕ_{n, j},$
	$D_{n} (h) = ˜ H a . s . \infty \sum j = 1 \infty \sum k = 1 ⟨ D_{n} (ϕ_{n, j}), ϕ_{n, k} ⟩_{˜ H} ⟨ h, ϕ_{n, j} ⟩_{˜ H} ϕ_{n, k},$		(8)

where, for arbitrary integer $n \geq 2,$ ${ϕ_{n, j} : j \geq 1}$ is a complete orthonormal system in $˜ H,$ and $C_{n, 1} \geq \dots \geq C_{n, n} \geq 0 = C_{n, n + 1} = C_{n, n + 2} = \dots$

The following assumption plays a crucial role in the derivation of the main results in this paper.

Assumption A1. $∥ X_{0} ∥_{B}$

is almost surely bounded, and the eigenspace

V_{j}

associated with

C_{j} > 0

in (6) is one-dimensional for every integer

j \geq 1

Under Assumption A1, we can define the following quantities:

a_{1} = 2 \sqrt{2} \frac{1}{C_{1} - C_{2}}, a_{j} = 2 \sqrt{2} max (\frac{1}{C_{j - 1} - C_{j}}, \frac{1}{C_{j} - C_{j + 1}}), j \geq 2.

(9)

Remark 1.

This assumption can be relaxed to consider multidimensional eigenspaces by redefining the quantities $a_{1}, a_{2}, \dots$ as the quantities $c_{1}, c_{2}, \dots$ given in Lemma 4.4 of Bosq00 .

Assumption A2. Let $k_{n}$ be such that $C_{n, k_{n}} > 0$ a.s., and both $k_{n} \to \infty$ and $k_{n} / n \to 0$ as $n \to \infty$ .

Remark 2.

Consider

Λ_{k_{n}} = sup j \in {1, \dots, k_{n}} (C_{j} - C_{j + 1})^{- 1} .

(10)

Then for sufficiently large $n$ , we have

k_{n} < C_{k_{n}}^{- 1} < \frac{1}{C_{k_{n}} - C_{k_{n} + 1}} < a_{k_{n}} < Λ_{k_{n}} < k_{n} \sum j = 1 a_{j} .

Assumption A3. As $k \to \infty$ ,

sup x \in B, ∥ x ∥_{B} \leq 1 {∥ ∥ ∥ ∥ ρ (x) - k \sum j = 1 ⟨ ρ (x), ϕ_{j} ⟩_{˜ H} ϕ_{j} ∥ ∥ ∥ ∥}_{B} \to 0.

Assumption A4. The constants ${C_{j} : j \geq 1}$ are such that the inclusion of $H (X)$ into ${˜ H}^{⋆}$ is continuous, i.e., $H (X) ↪ {˜ H}^{⋆},$ where $↪$ denotes the continuous embedding.

Let us consider the closed subspace $H$ of $B$ with the norm induced by the inner product $⟨ \cdot, \cdot ⟩_{H}$ defined as follows:

H = {x \in B : \infty \sum n = 1 {F_{n} (x)}_{n}^{2} < \infty}, \forall_{f, g \in H} ⟨ f, g ⟩_{H} = \infty \sum n = 1 F_{n} (f) F_{n} (g) .

(11)

Then $H$ is continuously embedded into $B$ and the following remark provides the isometric isomorphism established by the Riesz Representation Theorem between the spaces $˜ H$ and its dual ${˜ H}^{⋆} .$

Remark 3.

Let $f^{⋆}, g^{⋆} \in {˜ H}^{⋆},$ and $f, g \in ˜ H,$ such that, for every integer $n \geq 1$ , consider $F_{n} (f^{⋆}) = \sqrt{t_{n}} F_{n} (˜ f),$ $F_{n} (g^{⋆}) = \sqrt{t_{n}} F_{n} (˜ g),$ and $F_{n} (˜ f) = \sqrt{t_{n}} F_{n} (f),$ $F_{n} (˜ g) = \sqrt{t_{n}} F_{n} (g)$ for certain $˜ f, ˜ g \in H .$ Then the following identities hold:

⟨ f^{⋆}, g^{⋆} ⟩_{{˜ H}^{⋆}} = \infty \sum n = 1 \frac{1}{t_{n}} F_{n} (f^{⋆}) F_{n} (g^{⋆}) = \infty \sum n = 1 \frac{1}{t_{n}} \sqrt{t_{n}} \sqrt{t_{n}} F_{n} (˜ f) F_{n} (˜ g) = ⟨ ˜ f, ˜ g ⟩_{H} = \infty \sum n = 1 t_{n} F_{n} (f) F_{n} (g) = ⟨ f, g ⟩_{˜ H} .

Lemma 2.

Under Assumption A4, the following continuous embeddings hold:

H (X) ↪ {˜ H}^{⋆} ↪ B^{⋆} ↪ H ↪ B ↪ ˜ H ↪ {H (X)}^{⋆},

(12)

where

	$˜ H$	$= {x \in B : \infty \sum n = 1 t_{n} {F_{n} (x)}_{n}^{2} < \infty}, \forall_{f, g \in ˜ H} ⟨ f, g ⟩_{˜ H} = \infty \sum n = 1 t_{n} F_{n} (f) F_{n} (g),$
	$H$	$= {x \in B : \infty \sum n = 1 {F_{n} (x)}_{n}^{2} < \infty}, \forall_{f, g \in H} ⟨ f, g ⟩_{H} = \infty \sum n = 1 F_{n} (f) F_{n} (g),$
	${˜ H}^{⋆}$	$= {x \in B : \infty \sum n = 1 \frac{1}{t_{n}} {F_{n} (x)}_{n}^{2} < \infty}, \forall_{f, g \in {˜ H}^{⋆}} ⟨ f, g ⟩_{{˜ H}^{⋆}} = \infty \sum n = 1 F_{n} (f) F_{n} (g) / t_{n},$
	$H (X)$	$= {x \in ˜ H; ⟨ C^{- 1} (x), x ⟩_{˜ H} < \infty}, \forall_{f, g \in C^{1 / 2} (˜ H)} ⟨ f, g ⟩_{H (X)} = ⟨ C^{- 1} (f), g ⟩_{˜ H},$
	${H (X)}^{⋆}$	$= {x \in ˜ H; ⟨ C (x), x ⟩_{˜ H} < \infty}, \forall_{f, g \in C^{- 1 / 2} (˜ H)} ⟨ f, g ⟩_{{H (X)}^{⋆}} = ⟨ C (f), g ⟩_{˜ H} .$

Proof. Let us consider the following inequalities, valid for all $x \in B$ such that $x \in {˜ H}^{⋆}$ :

	$∥ x ∥_{˜ H} = \sqrt{\infty \sum n = 1 t_{n} {F_{n} (x)}_{n}^{2}} \leq ∥ x ∥_{B} = sup n \geq 1 \| F_{n} (x) \|,$
	$∥ x ∥_{B} = sup n \geq 1 \| F_{n} (x) \| \leq \sqrt{\infty \sum n = 1 {F_{n} (x)}_{n}^{2}} = ∥ x ∥_{H} \leq \infty \sum n = 1 \| F_{n} (x) \| = ∥ x ∥_{B^{⋆}},$
	$∥ x ∥_{B^{⋆}} = \infty \sum n = 1 \| F_{n} (x) \| \leq \sqrt{\infty \sum n = 1 \frac{1}{t_{n}} {F_{n} (x)}_{n}^{2}} = ∥ x ∥_{{˜ H}^{⋆}} .$		(13)

Under Assumption A4 (see also Remark 3), for every $f \in C^{1 / 2} (˜ H) = H (X)$ , we have

∥ f ∥_{H (X)} = \sqrt{⟨ C^{- 1} (f), f ⟩_{˜ H}} \geq ∥ f ∥_{{˜ H}^{⋆}} = \sqrt{\infty \sum n = 1 \frac{1}{t_{n}} {F_{n} (x)}_{n}^{2}} .

(14)

From Eqs. (13) and (14), the inclusions in (12) are continuous. Thus the proof is complete. $□$

It is well known that the set ${ϕ_{j} : j \geq 1}$ is also an orthogonal system in $H (X) .$ Futhermore, under Assumption A4, from Lemma 2, ${ϕ_{j} : j \geq 1} \subset H (X) ↪ {˜ H}^{⋆} ↪ B^{⋆} ↪ H$ . Therefore, from (11), for every $j \geq 1,$

∥ ϕ_{j} ∥_{H}^{2} = \infty \sum m = 1 {F_{m} (ϕ_{j})}^{2} < \infty .

(15)

The following assumption is now considered on the norm (15).

Assumption A5. The continuous embedding $i_{H (X), H} : H (X) ↪ H$ belongs to the Hilbert–Schmidt class, i.e., $\sum_{j = 1}^{\infty} ∥ ϕ_{j} ∥_{H}^{2} < \infty$ . Let ${F_{m} : m \geq 1}$ be defined as in Lemma 1. Assumption A5 leads to

\infty \sum j = 1 ⟨ i_{H (X), H} (ϕ_{j}), i_{H (X), H} (ϕ_{j}) ⟩_{H} = \infty \sum j = 1 \infty \sum m = 1 {F_{m} (ϕ_{j})}^{2} = \infty \sum m = 1 N_{m} < \infty,

(16)

where, in particular, from (16),

N_{m} = \infty \sum j = 1 {F_{m} (ϕ_{j})}^{2} < \infty, sup m \geq 1 N_{m} = N < \infty, V = sup j \geq 1 ∥ ϕ_{j} ∥_{B} \leq \infty \sum j = 1 \infty \sum m = 1 {F_{m} (ϕ_{j})}^{2} < \infty .

(17)

The following preliminary results are deduced from Theorem 4.1 (pp. 98–99), Corollary 4.1 (pp. 100–101), and Theorem 4.8 (pp. 116–117) in Bosq00 .

Lemma 3.

Under Assumption A1, the following identities hold, for any standard AR $˜ H (1)$ process, e.g., the extension to $˜ H$ of ARB $(1)$ process $X$ satisfying Eq. (1), as $n \to \infty$ :

{∥ C_{n} - C ∥}_{S (˜ H)} = O [{ln (n) / n}^{1 / 2}] a . s ., {∥ D_{n} - D ∥}_{S (˜ H)} = O [{ln (n) / n}^{1 / 2}] a . s .

(18)

Here, $\to_{a . s .}$ denotes almost surely convergence, and ${∥ \cdot ∥}_{S (˜ H)}$ is the norm in the Hilbert space $S (˜ H)$ of Hilbert–Schmidt operators on $˜ H$ , i.e., the subspace of compact operators $A$ such that $\sum_{j = 1}^{\infty} ⟨ A^{⋆} A (φ_{j}), φ_{j} ⟩_{˜ H} < \infty,$ for any orthonormal basis ${φ_{j} : j \geq 1}$ of $˜ H .$

Lemma 4.

Under Assumption A1, let ${C_{j} : j \geq 1}$ and ${C_{n, j} : j \geq 1}$ in (6) and (7), respectively. Then, as $n \to \infty$ , ${n / ln (n)}^{1 / 2} {sup}_{j \geq 1} | C_{n, j} - C_{j} | \to_{a . s .} 0.$

The following lemma is Corollary 4.3 on p. 107 of Bosq00 .

Lemma 5.

Under Assumption A1, consider $Λ_{k_{n}}$ in (10) satisfying $Λ_{k_{n}} = o [{n / ln (n)}^{1 / 2}],$ as $n \to \infty .$ Then, as $n \to \infty$ ,

sup j \in {1, \dots, k_{n}} ∥ ϕ_{n, j}^{'} - ϕ_{n, j} ∥_{˜ H} \to_{a . s .} 0,

where, for every integer $j \geq 1$ and $n \geq 2$ , $ϕ_{n, j}^{'} = s g n ⟨ ϕ_{n, j}, ϕ_{j} ⟩_{˜ H} ϕ_{j},$ $s g n ⟨ ϕ_{n, j}, ϕ_{j} ⟩_{˜ H} = 1_{⟨ ϕ_{n, j}, ϕ_{j} ⟩_{˜ H} \geq 0} - 1_{⟨ ϕ_{n, j}, ϕ_{j} ⟩_{˜ H} < 0},$ with $1$ denoting an indicator function.

An upper bound for ${∥ c ∥}_{B \times B} = ∥ \sum_{j = 1}^{\infty} C_{j} ϕ_{j} \otimes ϕ_{j} ∥_{B \times B}$ is obtained next.

Lemma 6.

Under Assumption A5, the following inequality holds:

{∥ c ∥}_{B \times B} = sup n, m \geq 1 | C (F_{n}) (F_{m}) | \leq N {∥ C ∥}_{L (˜ H)},

where $N$ has been introduced in (17), $L (˜ H)$ denotes the space of bounded linear operators on $˜ H,$ and ${∥ \cdot ∥}_{L (˜ H)}$ the usual uniform norm on such a space.

Let us consider the following notation:

c = ˜ H \otimes ˜ H \infty \sum j = 1 C_{j} ϕ_{n, j}^{'} \otimes ϕ_{n, j}^{'} = ˜ H \otimes ˜ H \infty \sum j = 1 C_{j} ϕ_{j} \otimes ϕ_{j}, c_{n} = ˜ H \otimes ˜ H \infty \sum j = 1 C_{n, j} ϕ_{n, j} \otimes ϕ_{n, j},

c - c_{n} = ˜ H \otimes ˜ H \infty \sum j = 1 C_{j} ϕ_{n, j}^{'} \otimes ϕ_{n, j}^{'} - \infty \sum j = 1 C_{n, j} ϕ_{n, j} \otimes ϕ_{n, j} .

(19)

Remark 4.

From Lemma 3, for sufficiently large $n$ , there exist positive constants $K_{1}$ and $K_{2}$ such that, for all $φ \in ˜ H$ ,

K_{1} ⟨ C (φ), φ ⟩_{˜ H} \leq ⟨ C_{n} (φ), φ ⟩_{˜ H} \leq K_{2} ⟨ C (φ), φ ⟩_{˜ H} .

In particular, for every $x \in H (X) = C^{1 / 2} (˜ H),$ considering $n$ sufficiently large, we find

\frac{1}{K_{1}} ⟨ C^{- 1} (x), x ⟩_{˜ H} \geq ⟨ C_{n}^{- 1} (x), x ⟩_{˜ H} \geq \frac{1}{K_{2}} ⟨ C^{- 1} (x), x ⟩_{˜ H} \Leftrightarrow \frac{1}{K_{1}} ∥ x ∥_{H (X)}^{2} \geq ⟨ C_{n}^{- 1} (x), x ⟩_{˜ H} \geq \frac{}{1} K_{2} ∥ x ∥_{H (X)}^{2} .

(20)

Eq. (20) means that, for sufficiently large $n$ , the norm of the RKHS $H (X)$ of $X$ is equivalent to the norm of the RKHS generated by $C_{n},$ with spectral kernel $c_{n}$ given in (19).

Lemma 7.

Under Assumptions A1–A5, consider $Λ_{k_{n}}$ in (10) satisfying

\sqrt{k_{n}} Λ_{k_{n}} = o {\sqrt{n / ln (n)}}

(21)

as $n \to \infty$ , where $k_{n}$ has been introduced in Assumption A2. The following almost sure inequality then holds:

{∥ c - c_{n} ∥}_{B \times B} \leq max (N, \sqrt{N}) [∥ C - C_{n} ∥_{L (˜ H)} + 2 max (\sqrt{∥ C ∥_{L (˜ H)}}, \sqrt{∥ C_{n} ∥_{L (˜ H)}}) [sup ℓ \geq 1 sup m \geq 1 | F_{ℓ} (ϕ_{n, m}^{'}) |] \times \sqrt{k_{n} 8 Λ_{k_{n}}^{2} ∥ C_{n} - C ∥_{L (˜ H)}^{2} + \infty \sum m = k_{n} + 1 ∥ ϕ_{n, m} - ϕ_{n, m}^{'} ∥_{˜ H}^{2}} ⎤ ⎦ .

Therefore, ${∥ c - c_{n} ∥}_{B \times B} \to_{a . s .} 0$ as $n \to \infty .$

Lemma 8.

For a standard ARB $(1)$ process satisfying Eq. (1), under Assumptions A1–A5, consider $Λ_{k_{n}}$ in (10) such that

C_{k_{n}}^{- 1} \sqrt{k_{n}} Λ_{k_{n}} = o {\sqrt{n / ln (n)}}

(22)

as $n \to \infty$ , where $k_{n}$ has been introduced in Assumption A2. The following inequality then is established

sup j \in {1, \dots, k_{n}} {∥ ∥ ϕ_{n, j} - ϕ_{n, j}^{'} ∥ ∥}_{B} \leq \frac{2}{C_{k_{n}}} [max (N, \sqrt{N}) [∥ C - C_{n} ∥_{L (˜ H)} + 2 max (\sqrt{∥ C ∥_{L (˜ H)}}, \sqrt{∥ C_{n} ∥_{L (˜ H)}}) {sup ℓ \geq 1 sup m \geq 1 | F_{ℓ} (ϕ_{n, m}^{'}) |} \times \sqrt{k_{n} 8 Λ_{k_{n}}^{2} ∥ C_{n} - C ∥_{L (˜ H)}^{2} + \infty \sum m = k_{n} + 1 ∥ ϕ_{n, m} - ϕ_{n, m}^{'} ∥_{˜ H}^{2}}] + sup j \in {1, \dots, k_{n}} ∥ ϕ_{n, j} - ϕ_{n, j}^{'} ∥_{˜ H} N ∥ C ∥_{S (˜ H)} + V ∥ C - C_{n} ∥_{S (˜ H)}] {a.s.}

(23)

Therefore, under (22), ${sup}_{j \in {1, \dots, k_{n}}} ∥ ϕ_{n, j} - ϕ_{n, j}^{'} ∥_{B} \to_{a . s .} 0,$ as $n \to \infty .$

Lemma 9.

Under Assumption A3, if $\sum_{j = 1}^{k_{n}} ∥ ϕ_{n, j} - ϕ_{n, j}^{'} ∥_{B} \to_{a . s .} 0$ as $n \to \infty,$ then also

sup x \in B, ∥ x ∥_{B} \leq 1 {∥ ∥ ∥ ∥ ρ (x) - k_{n} \sum j = 1 ⟨ ρ (x), ϕ_{n, j} ⟩_{˜ H} ϕ_{n, j} ∥ ∥ ∥ ∥}_{B} \to_{a . s .} 0.

(24)

Remark 5.

Under the conditions of Lemma 8, Eq. (24) holds as soon as

C_{k_{n}}^{- 1} k_{n}^{3 / 2} Λ_{k_{n}} = o {\sqrt{n / ln (n)}} .

Let us know consider the projection operators defined, for all $x \in B \subset ˜ H$ , by

{˜ Π}^{k_{n}} (x) = k_{n} \sum j = 1 ⟨ x, ϕ_{n, j} ⟩_{˜ H} ϕ_{n, j}, Π^{k_{n}} (x) = k_{n} \sum j = 1 ⟨ x, ϕ_{n, j}^{'} ⟩_{˜ H} ϕ_{n, j}^{'} .

(25)

Remark 6.

Under the conditions of Remark 5, let

{˜ Π}^{k_{n}} ρ {˜ Π}^{k_{n}} = k_{n} \sum j = 1 k_{n} \sum p = 1 ⟨ ρ (ϕ_{n, j}), ϕ_{n, p} ⟩_{˜ H} ϕ_{n, j} \otimes ϕ_{n, p} .

Then, as $n \to \infty$ ,

sup x \in B, ∥ x ∥_{B} \leq 1 {∥ ∥ ∥ ∥ ρ (x) - k_{n} \sum j = 1 k_{n} \sum p = 1 ⟨ x, ϕ_{n, j} ⟩_{˜ H} ⟨ ρ (ϕ_{n, j}), ϕ_{n, p} ⟩_{˜ H} ϕ_{n, p} ∥ ∥ ∥ ∥}_{B} \to_{a . s .} 0.

4 Proofs of the lemmas

4.1 Proof of Lemma 6

Applying the Cauchy–Schwarz inequality, we have, for all integers $k, ℓ \geq 1$ ,

	$\| C (F_{k}, F_{ℓ}) \|$	$= ∣ ∣ ∣ ∣ \infty \sum j = 1 C_{j} F_{k} (ϕ_{j}) F_{ℓ} (ϕ_{j}) ∣ ∣ ∣ ∣ \leq \sqrt{\infty \sum j = 1 C_{j} {F_{k} (ϕ_{j})}^{2} \infty \sum p = 1 C_{p} {F_{ℓ} (ϕ_{p})}^{2}}$
		$\leq sup j \geq 1 \| C_{j} \| \sqrt{\infty \sum j = 1 {F_{k} (ϕ_{j})}^{2} \infty \sum p = 1 {F_{ℓ} (ϕ_{p})}^{2}} = sup j \geq 1 \| C_{j} \| \sqrt{N_{k} N_{ℓ}},$		(26)

where ${F_{n} : n \geq 1}$ has been introduced in Eq. (3), and satisfies (4)–(5). Under Assumption A5, from Eq. (17),

∥ c ∥_{B \times B} = sup k, ℓ \geq 1 | C (F_{k}, F_{ℓ}) | \leq sup k, ℓ \geq 1 sup j \geq 1 | C_{j} | \sqrt{N_{k} N_{ℓ}} = N sup j \geq 1 | C_{j} | = N ∥ C ∥_{L (˜ H)} .

This completes the proof. $□$

4.2 Proof of Lemma 7

First observe that

| C - C_{n} (F_{k}) (F_{ℓ}

	$∥ x ∥_{˜ H} = \sqrt{\infty \sum n = 1 t_{n} {F_{n} (x)}_{n}^{2}} \leq ∥ x ∥_{B} = sup n \geq 1 \| F_{n} (x) \|,$
	$∥ x ∥_{B} = sup n \geq 1 \| F_{n} (x) \| \leq \sqrt{\infty \sum n = 1 {F_{n} (x)}_{n}^{2}} = ∥ x ∥_{H} \leq \infty \sum n = 1 \| F_{n} (x) \| = ∥ x ∥_{B^{⋆}},$
	$∥ x ∥_{B^{⋆}} = \infty \sum n = 1 \| F_{n} (x) \| \leq \sqrt{\infty \sum n = 1 \frac{1}{t_{n}} {F_{n} (x)}_{n}^{2}} = ∥ x ∥_{{˜ H}^{⋆}} .$		(13)

Strongly consistent autoregressive predictors in abstract Banach spaces

Authors

Strong-consistent autoregressive predictors in abstract Banach spaces

A note on strong-consistency of componentwise ARH(1) predictors

An optimal linear filter for estimation of random functions in Hilbert space

Minimal Stinespring Representations of Operator Valued Multilinear Maps

Parseval Proximal Neural Networks

Nonparametric regression estimation for quasi-associated Hilbertian processes

Hilbert Space Embeddings of Predictive State Representations

1 Introduction

2 Preliminaries

Lemma 1.

3 Main assumptions and preliminary results

Remark 1.

Remark 2.

Remark 3.

Lemma 2.

Lemma 3.

Lemma 4.

Lemma 5.

Lemma 6.

Remark 4.

Lemma 7.

Lemma 8.

Lemma 9.

Remark 5.

Remark 6.

4 Proofs of the lemmas

4.1 Proof of Lemma 6

4.2 Proof of Lemma 7

Strongly consistent autoregressive predictors in abstract Banach spaces

Authors

Related Research

Strong-consistent autoregressive predictors in abstract Banach spaces

A note on strong-consistency of componentwise ARH(1) predictors

An optimal linear filter for estimation of random functions in Hilbert space

Minimal Stinespring Representations of Operator Valued Multilinear Maps

Parseval Proximal Neural Networks

Nonparametric regression estimation for quasi-associated Hilbertian processes

Hilbert Space Embeddings of Predictive State Representations

This week in AI

1 Introduction

2 Preliminaries

Lemma 1.

3 Main assumptions and preliminary results

Remark 1.

Remark 2.

Remark 3.

Lemma 2.

Lemma 3.

Lemma 4.

Lemma 5.

Lemma 6.

Remark 4.

Lemma 7.

Lemma 8.

Lemma 9.

Remark 5.

Remark 6.

4 Proofs of the lemmas

4.1 Proof of Lemma 6

4.2 Proof of Lemma 7