Identification and estimation of characteristics in stochastic linear systems are usually based on “black-box” models. An input may be single or multiple, perfect or noisy, as well as an output may be. The areas of application of these models are quite different, varying from civil engineering (Engberg and Larsen 1995, Chapter 9), (Ren, Zhao and Harik 2004), (Spiridonakos and Chatzi 2014), communications and networks (Demir and Sangiovanni-Vintcentelli 1998, Sections 2.3–2.5), signal and image processing (Camps-Valls, Rojo-Álvarez, and Martínez-Ramón 2007), (Ogunfumni 2007, Chapters 3–5), (Gao et al. 2014), (Prabhu 2014, Chapters 7–9), system identification and control (Söderström and Stoica 1988, Chapters 5–9), (Sjöberg et al. 1995), (Chen, Ohlsson and Ljung 2012), applications to biology (Rost, Geske and Baake 2006) and finance (Hatemi-J 2012, 2014). Either in parametric of non-parametric framework, the above mentioned models use the feature of any linear system to be uniquely identified by means of the impulse response function (IRF).
Our paper deals with a single-input double-output (SIDO) channels model described by a linear time-invariant (LTI) system whose IRF has two real-valued components (kernels) . System’s input is supposed to be a standard Wiener process on , whereas the outputs are defined as follows:
In this paper, we assume that is unknown while is known (controlled), and our aim is to estimate after observations of the outputs and . For a detailed survey of deterministic and statistical approaches to the mentioned problem including correlogram and periodogram methods, we refer the reader to (Blazhievska and Zaiats 2018). Our main idea is based on the fact that can be obtained by cross-correlating the and given that is “close” to Dirac’s delta:
This leads us to carrying out the following steps:
to introduce the cross-correlogram between the outputs and to take it as an estimator of ;
to approximate the Dirac delta by a family of -like -integrable kernels;
to use asymptotic results about sample cross-corellograms of Gaussian processes;
to study estimator’s properties in functional spaces (particularly, in a space of continuous functions).
Roughly speaking, is estimated by the empiric integral-type sample cross-correlogram
The kernel is controlled in the following way: it involves a parameter , and we assume that tends to the Dirac delta as . Since the output is expressed in terms of , the estimator depends on two parameters: the averaging length required to make asymptotically unbiased, and whose role consists in scaling. We are interested in establishing a relation between and which would provide asymptotic normality of in the Banach space of continuous functions; we would also like to construct confidence intervals for in this space. Our basic assumption will be complemented with some extras related to sample behaviour of Gaussian processes and bilinear forms of Gaussian processes.
A similar problem was considered in (Li 2005) where the centred estimator was proved to be asymptotically normal. A recent paper (Blazhievska and Zaiats 2018) has removed extra assumptions on reducing them to the one and only condition of . An extension to SIDO-systems has been new. An important feature is that the -integrability is, in certain sense, a necessary condition, admitting non-stable system’s channels. Recall that stability of an LTI system is related to the fact that . If were involved only, it would enable to apply asymptotic properties of Delta matroid integrals generated by functionals based on processes with independent increments, see (Anh, Leonenko, and Sakhno 2007) and (Avram, Leonenko, and Sakhno 2010, 2015).
Our study of weak convergence of a centred estimator in Banach spaces of continuous functions leads to a situation where two types of processes appear (Blazhievska and Zaiats 2018): a limiting non-stationary Gaussian process and a pre-limiting process which is stationary and square-Gaussian, belonging to an Orlicz space. Almost sure (a. s.) continuity is proved by means of the entropy approach related to these classes of processes, see (Dudley 1973), (Fernique 1975), (Lifshits 1995, Chapter 14), (Buldygin and Kozachenko 2000, Chapter 4) and (Kozachenko et al. 2017, Chapter 6), while the classical Prokhorov theorem is employed in proving a CLT. Our proof is based on comparison principles for Gaussian processes, similar to what was done in (Buldygin 1983) and extended in (Buldygin and Zaiats 1991, 1995) and (Buldygin and Kharazishvili 2000, Chapter 12). A recent paper (Nourdin, Peccati, and Viens 2014) may open new horizons since it departs from the Gaussian context.
Similar problems for single-input single-output (SISO) systems were considered in (Buldygin, Utzet and Zaiats 2004), (Buldygin and Blazhievska 2009), (Blazhievska 2011), (Kozachenko and Rozora 2015) and (Rozora 2018). For a survey of results on estimation of the correlation function in Gaussian processes which was the parent problem for our setting, see (Buldygin and Kozachenko 2000, Chapter 6).
The following two circumstances motivate a special role of the functional space we choose:
it is important to characterise (in terms of the covariance function) continuous Gaussian processes since these processes are mathematical models of numerous real-life phenomena;
the space of real-valued continuous functions is universal in the class of all separable metric spaces.
Assumptions providing that sample paths of different classes of stochastic processes are a. s. continuous have widely been discussed in the literature. Gaussian processes constitute a theoretically important case. This is why we only state those results that are relevant in the settings. Our study deals with the entropy method. However, in the last twenty years, great progress has been made using majorising measures beginning with the celebrated Talagrand’s paper; see (Lifshits 1995, Chapter 16) and (Ledoux and Talagrand 2013, Chapter 11). Note that the entropy method leads to sufficient conditions whereas majorising measures provide necessary and sufficient ones for Gaussian processes to be a. s. continuous.
This paper is organised as follows: Section 2 contains definitions and preliminary information. Section 3 shows when a centred estimator becomes asymptotically Gaussian in spaces of continuous functions. Confidence intervals for the uniform norm of the pre-limiting and limiting processes are constructed in Section 4. Some illustrative examples are given in Section 5. Wiener shot noise processes appearing in our framework were simulated in Wolfram Mathematica; they are represented on Figures 2–3. Section 6 contains concluding remarks.
The symbols , and are used to mark the end of a proof, a remark, or an example, respectively.
2 Definitions and preliminaries
We introduce the following notations to be used throughout all paper:
is the Banach space of Lebesgue -integrable complex-valued functions on with the norm ;
is the Banach space of complex-valued essentially bounded functions on with the sup-norm ;
is the Fourier-Plancherel transform of a given function , that is
implying that and (e.g., (Kolmogorov and Fomin 1968, 439));
, , is the (separable) Banach space of real-valued continuous functions on with the sup-norm ;
is the Orlicz space of random variables generated by the-function , and defined on a probability space ; (e.g., (Buldygin and Kozachenko 2000, Remark 2.3.1)).
Our model and main assumptions.
We consider the following SIDO LTI system with IRF having two real-valued components (or kernels):
where is an unknown function while is a known function dependent on a parameter . In the above figure, we shadow the channel where the unknown IRF appears. Within the paper, we always suppose that the following assumptions hold:
the family satisfies:
(1a) (1b) (1c) (1d)
In engineering, the Fourier transform of an IRF is often called the frequency transfer function (FTF). Since the basic assumption is, the FTF should be interpreted in the Fourier-Plancherel sense in our framework.
is a real-valued even function;
if is a non-negative function, then
if , then is the usual Fourier transform of
the family approximates, in a way, the Dirac delta.
Summarising all these statements, the family covers -like families for scaled Fourier transforms of classic window functions (Prabhu 2014, Chapter 3). Observe also that the proposed type of -scales for the kernels may lead us to continuous wavelets (Najmi 2012, Chapters 4–6).
Input and output processes.
The input to the system is a standard Wiener process on . Both outputs are supposed to be observed:
the integrals in (2) are interpreted as mean-square Riemann integrals. Since and are -integrable and since our system is LTI, the outputs and are jointly Gaussian, stationary, zero-mean processes having spectral densities and respectively (e.g., (Lindgren 2006, Theorem 4.8)).
We also suppose that the process and all processes , are separable and a. s. sample continuous on . This assumption is natural since we deal with Gaussian stationary processes whose correlation functions are continuous; see Belyaev’s alternative in (e.g., (Lifshits 1995, Theorem 7.3)).
We use the following integral-type sample cross-correlogram as an estimator of :
Here, is the constant appearing in (1d), is the length of the interval where the averaging is performed. Since and are a. s. sample continuous processes, the integral in (3) may be interpreted in the mean-square Riemann sense. For any , it defines an a. s. sample continuous function; one has meaning that is biased as an estimator of .
The function is non-random and continuous on as a normalised (by 1/c) joint covariance function of the processes and , both of which are mean-square continuous.
The fact that the estimator is biased and that it depends on two parameters, and , enables us to study the role of these parameters in obtaining nice statistical properties of the estimator.
The object of study.
The process may be interpreted as a normalised (by ) mean-square limit of integral Riemann sums of the type
The latter allows us to claim that is a centred square-Gaussian process and hence it is an -process. For further details on Orlicz spaces and pre-ordering of embedded norms, see (Buldygin and Kozachenko 2000, Chapters 1, 4–6).
We give a brief account of the facts we need for our future purposes. It is clear that process (4) has zero mean; for any , the covariance function of is as follows:
where denotes the well-known Fejér kernel, that is,
In particular, for any the limit of (2) taken as and has the form
The function on the right-hand side of (6) will be denoted by , . Let , be a measurable separable real-valued Gaussian process with zero mean whose covariance function is :
All finite-dimensional distributions of the process converge, as and , to those of the process (Blazhievska and Zaiats 2018, Theorem 3.2). It is clear that is non-stationary.
Further steps to be made.
In this paper, we prove that is asymptotically normal in . This question is raised in a quite natural way since, for any and , the process is a. s. sample continuous on . Indeed, the process is a. s. sample continuous on , and Remark 2.3 holds. Since converges weakly to in , we can obtain information on asymptotic behaviour of the uniform deviation of the estimator from its mean when runs through . Moreover, since the mean-square error of the (non-stationary) process is majorised by that of which is stationary, Gaussian comparison inequalities can be applied to these Gaussian processes in order to find simple bounds on the tails of the uniform norm of .
3 Asymptotic normality of the process in
We recall some useful tools related to Gaussian stochastic processes (Buldygin and Kozachenko 2000, Chapter 4). Let be a parametric set. A function is called a pseudometric on if it satisfies all axioms of a metric, with the exception that the set may be wider than the diagonal . We write for the minimum of the number of closed -balls of radius whose centres lie in S and which cover (-covering). If there is no finite -covering of , then we write . The function is called the metric massiveness (-covering number) of with respect to . Further, let be a standard notation for the metric entropy of with respect to . For any , the inequality is always interpreted in the sense that for some (and hence for all) we have . Consider the function
Since , it is well-defined and generates the following two pseudometrics: and . Note that if for belonging to a set of a positive Lebesgue measure, then and are metrics. For all , put . The pseudometrics and depend on only. Then for any and
Our choice of the function is not accidental and is motivated by the following fact:
In other words, the pseudometrics and are related to the mean-square error of the stationary Gaussian output .
The next theorem gives sufficient conditions for and to be a. s. continuous and shows when converges weakly to in as and . This weak convergence will be denoted by . In the sequel, we use the notation , for the uniform metric.
For any , assume that the following inequality holds:
Then we have: (I) a. s.; (II) a. s. for any fixed and (III) as and In particular, for any
Assumption (9) is satisfied (Buldygin, Utzet, and Zaiats 2004) if the following condition holds for some :
The proof of Theorem 3.1 requires auxiliary results. First of all, consider the link between the pseudometrics and . Since, for all ,
For any and any the following inequality holds:
Moreover, the pseudometric is continuous with respect to the pseudometic .
Apply the Cauchy-Schwarz inequality, the Young inequality for convolutions (e.g., (Edwards 1965, 655)), and the fact that , to (2). Then
implying (13). Note that the inequality is uniform in and . By the Lebesgue dominated convergence, as . Therefore is continuous with respect to . It is clear that convergence to zero is equivalent both for and . This proves Lemma 3.1. ∎
Proof of Theorem 3.1.
By the Dudley theorem on continuity of Gaussian processes (Dudley 1973), Statement (I) in Theorem 3.1 holds if, for any ,
where . By the Cauchy-Schwarz inequality applied to (6), we obtain
The Lebesgue dominated convergence implies that as . Therefore is a mean-square continuous process. Formula (14) holds if . The proof of Statement (I) in Theorem 3.1 becomes complete by application of (9).
The next step is based on the fact that is a square-Gaussian process. By Theorem 3.5.4 in (Buldygin and Kozachenko 2000), Statement (II) holds if, for any
The last step of the proof is based on application of relevant results on square-Gaussian processes. We will use Lemma 4.2.1 in (Buldygin and Kozachenko 2000) whose adapted version is as follows:
Let be a family of a. s. sample continuous random processes. For any , assume that the following conditions hold
the pseudometric is continuous with respect to ;
Then, for any we have
Let us check whether the assumptions of Lemma 3.2 hold. Since is a square-Gaussian zero-mean process, we have, by Theorem 6.2.2 in (Buldygin and Kozachenko 2000), for any and :
The Lebesgue dominated convergence implies that the pseudometric is continuous with respect to the metric . By Lemma 3.1, the pseudometric
Theorem 3.2 in (Blazhievska and Zaiats 2018) combined with (20) implies that the Prohorov theorem on weak convergence of stochastic processes in holds (e.g., (Billingsley 2013, 59)). This proves Statement (III) in Theorem 3.1. In this way, the proof of Theorem 3.1 is complete. ∎
4 Confidence intervals for the uniform norms of and
Theorem 3.1 states a CLT in for the centred estimator . In particular, for any
This equality enables us to construct confidence intervals for and which are accurate enough for large and . Observe that each of the parameters and tends to infinity on its own; a link between them will appear latter on, when we look at the error term. We use two different techniques for studying tails of the uniform norm for the underlying processes:
the pre-limiting process requires a theory related to square-Gaussian processes (or -processes), see (Buldygin and Kozachenko 2000, Chapters 4–6) and (Kozachenko and Rozora 2015);
the limiting process is treated by means of comparison principles related to Gaussian processes, see (Buldygin 1983) and (Buldygin and Kharazishvili 2000, Chapter 12).
Note that each of these techniques requires the mentioned zero-mean processes to be separable and to satisfy certain majorisation between their entropy characteristics or their correlation functions.
4.1 Bounds on large deviations of the supremum of
Since the distribution of a Gaussian process is completely determined by the covariance function of this process, it is natural to anticipate that a majorisation of covariances would lead to a certain subordination between the processes. The well-known Anderson and Slepian inequalities, see Theorem 3 and Lemma 3 respectively in (Buldygin and Kharazishvili 2000, Chapter 12), turn out to be technical tools for obtaining useful results on comparison of Gaussian processes. Let us focus on them in more detail.
Recall that and are separable, Gaussian and have zero means. We will compare the limiting process to a new process obtained by adding an independent random variable to the output scaled by . This idea enables us to use Slepian’s and Anderson’s inequalities simultaneously.
Since formula (21) gives majorisation of in terms of the output only, where neither of the parameters and is involved, the confidence intervals for constructed by comparison with will be free of these parameters.
Let us give an adapted version of Theorem 4 in (Buldygin and Kharazishvili 2000, Chapter 12). The proof is dropped; it follows the lines of the source theorem.
Assume that is an -distributed random variable independent of and let
Then, for any , we have
We will specify the structure of the function below. Since
and since for any , one has
Note that all these calculations are only based on the assumption that is -integrable.
Inequality (22) in Theorem 4.1 may be rewritten in another way. The following statements can be extracted from (Buldygin and Kharazishvili 2000, Chapter 12); Corollary 4.1 is contained in the proof of Theorem 3, and Corollary 4.2 is adapted from Lemma 4. We drop explicit proofs of these corollaries.
For any , we have
Here, is any number belonging to and is as defined in (23).
For any , we have