On the Underspread/Overspread Classification of Random Processes

by   Werner Kozek, et al.
NYU college
Universität Wien

We study the impact of the recently introduced underspread/overspread classificationon the spectra of processes with square-integrable covariance functions. We briefly review the most prominent definitions of a time-varying power spectrum and point out their limited applicability for general nonstationary processes. The time-frequency-parametrized approximation of the nonstationary Wiener filter provides an excellent example for the main conclusion: It is the class of underspread processeswhere a time--varying power spectrum can be used in the same manner as the time--invariant power spectrum of stationary processes.



page 1

page 2

page 3

page 4


Continuous-time locally stationary time series models

We adapt the classical definition of locally stationary processes in dis...

Fraudulent White Noise: Flat power spectra belie arbitrarily complex processes

Power spectral densities are a common, convenient, and powerful way to a...

PLSO: A generative framework for decomposing nonstationary timeseries into piecewise stationary oscillatory components

To capture the slowly time-varying spectral content of real-world time s...

Isomorphism Problem Revisited: Information Spectrum Approach

The isomorphism problem in the ergodic theory is revisited from the pers...

Adaptive Frequency Band Analysis for Functional Time Series

The frequency-domain properties of nonstationary functional time series ...

Spectral Inference under Complex Temporal Dynamics

We develop unified theory and methodology for the inference of evolution...

Estimation and Inference of Time-Varying Auto-Covariance under Complex Trend: A Difference-based Approach

We propose a difference-based nonparametric methodology for the estimati...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Time-frequency distribution are widely used to search for hidden structure in the signal. When the signal consists of a small number of slowly varying sinusoids, the Wentzel-Kramer-Brillioun representation reduces the signal to curves in the time-frequency plane [16]. We consider the case of nonstationary stochastic processes with underlying time-frequency structure in the correlation operator.

The evolutionary spectrum is one common representation of nonstationary processes. In [15], we propose estimating the evolutionary spectrum by smoothing the log-spectrogram using a data-adaptive kernel smoother in the time-frequency plane. The evolutionary spectrum has two advantages: it is always positive and it converges to the spectrum as the ratio of the characteristic time scale to the sampling rate becomes large. Its disadvantages are its lack of uniqueness and its relatively poor time frequency resolutions.

We consider a different class of representations of nonstationary processes: quadratic Cohen’s class spectra. These representations correspond to the expected value of Cohen’s class time-frequency representations. An important member of this class is the Wigner-Ville spectrum. This class of spectral representations possess useful operator properties and a reproducing kernel Hilbert space structure.

In this article, we consider the estimation problem: how to estimate the Cohen’s class spectra. This same problem has been considered by Sayeed and Jones as well. In [19], a complete knowledge of the correlation operator is assumed. This assumption is appropriate for the signal classificiation problem of recognizing one or more specific signals. Our approach assumes much weaker a priori knowledge. We assume only that the signal is underspread which corresponds to being double band limited in the ambiguity plane.

In Section 2, we review time-frequency representations of deterministic signals. Section 3 presents the analogous theory for time varying spectra. Section 4 defines and motivates underspread processes.

Section 5 analyzes the bias and variance of a special class of quadratic estimators of Cohen’s class spectra. Section 6 determines the minimum variance unbiased estimator of an underspread process. Section 7 describes a related estimation using multiple windows. Section 8 presents a biomedical example.

2 Quadratic Time-Frequency Distributions

Every real–valued quadratic time–frequency (TF) shift–invariant (QTFI) representation of a signal can be represented as a quadratic form [1]. Comprehensive reviews of Cohen’s class are given in [2, 4]. We now cast Cohen’s class of time-frequency representations in an operator theoretic framework. Let be a self–adjoint Hilbert-Schmidt (H-S) “prototype” operator. We define the quadratic time-frequency shift invariant (QTFI) distribution


where is a TF shifted version of the self–adjoint prototype operator . The choice of the kernel, , determines a particular representation in Cohen’s class. The TF–shifting of operators is defined as where is a unitary TF shift operator, acting as The standard H-S inner product, , is

where and are the respective kernels of the H.-S. operators and . Throughout this article, we assume an infinite time domain and suppress replace with .

We now review the basic unitary TF representations of HS operators [6]. The generalized Weyl symbol is defined as

where . The Weyl correspondence is given by , and the Kohn-Nirenberg correspondence (time–varying transfer function) by [20]. (When we suppress the superscript, this means validity for any .) The TF shifting of operators corresponds to a shift of the symbol, which shows that whenever is a TF localization operator that selects signals centered in the origin of the TF plane then localizes signal components centered around . The generalized spreading function (GSF) of a linear operator [6] is

The GSF is the symplectic Fourier transform of the generalized Weyl symbol



When the Weyl symbol is smoothly varying in time and frequency, then the generalizing spreading function decays in and .

3 Time Varying Spectrum

For a nonstationary process, a time–varying spectrum may be defined as the expectation of (1)


A prominent example for is the Wigner–Ville spectrum [13]. Priestley’s evolutionary spectrum [14, 15] is a different, popular definition of a stochastic time–varying spectrum that cannot be brought into the form of (1).

We consider circular complex, zero–mean Gaussian processes with trace–class correlation kernel

The trace–class convention implies a HS inner product representation of , alternatively written as the trace of the product operator:


The expected ambiguity function is defined as the GSF of the correlation operator [8]


With the generalized Wigner–Ville spectrum, defined as

Eq. (2) carries over to a “nonstationary Wiener–Khintchine relation”:


These relation ships are summarized in Table 1.

Weyl symbol of correlation Generalized W-V spectrum
GSF of correlation Expected ambiguity function

Table 1

As an example, the real–valued generalized Wigner–Ville spectrum can be written as

where the –dependent prototype operator is given by:


Since both the Weyl symbol and the spreading function are unitary representations of HS operators we can rewrite the general time–varying spectrum,

Note furthermore that the GSF of the TF shifted prototype operator is just a modulated version of the GSF of the original version:

thus in particular .

4 Underspread Processes

The bias-variance analysis of Sec. 5 is valid for any circular Gaussian process with a trace class covariance. We now restict our consideration to the case where the process’ expected ambiguity function is zero outside a rectangle in the ambiguity plane. Our requirement that the expected ambiguity is double band-limited implies that the Weyl symbol is smooth in time and frequency.

We denote the maximum temporal correlation width and the maximum spectral correlation width ; i.e., we assume that the expected ambiguity function satisfies


where is the –valued indicator function of a centered rectangle with area . According to the recently introduced terminology we call a process with underspread and in the converse case overspread [7]. For asymptotics we assume that the underspread parameter is very small: . The underspread parameter, , corresponds to the expansion parameter , which is used in the analysis of evolutionary spectra [15].

As to the relevance and realizability of the underspread processes we note that practically important linear time–varying (LTV) systems, as e.g. the mobile radio channel or underwater acoustic channel [21]

, are characterized by an (at least in good approximation) restricted spreading function (this is the field where the underspread/overspread terminology was originally introduced). Now, we apply stationary white noise

with to an underspread LTV system characterized by

where covers a centered rectangle with halfwidths and . Then the output process is nonstationary with correlation

Applying the triangle inequality to the spreading function of the product operator [11] gives

where the denotes double convolution. The output process is thus underspread with and . Hence, we have shown that underspread processes are realizable and relevant.

In view of the “nonstationary Wiener–Khintchine relation” (4), the overspread/underspread classification may be interpreted as a smoothness condition for the time–varying spectrum of the process. Applying the sampling theorem on the symbol level leads to a discrete Weyl–Heisenberg expansion of the correlation operator [11]:

valid for a sampling grid with

and the prototype operator defined by


The critical spread corresponds to the Nyquist sampling density . Hence, considering bandlimited processes, for the rate of innovation in the process second order statistics is equal to the sampling rate of the realization [8]. However, a robust estimation procedure maps a time series with samples on a model with less than coefficients such that the critical spread is a treshold for robust estimation of the generalized Wigner–Ville spectrum. It is furthermore remarkable that the evolutionary spectrum of an underspread process is 2D bandlimited in exactly the same manner as the generalized Wigner–Ville spectrum [11].

It should be noted that one can view the stationarity assumption underlying any time–invariant spectrum estimation as a limit case of (1) since the expected ambiguity function of a wide–sense stationary process is characterized by ideal concentration on the –axis:

where is the autocorrelation function.

The Wiener–Khintchine relation requires strict band-limiting the ambiguity plane. The remainder of our analysis requires only a concentration in the ambiguity plane with characteristic spread, , but not complete band limitation.

5 Reproducing Kernel Hilbert Space

We now show that time-frequency distributions are a reproducing kernel Hilbert spaces (RKHS) [5] using the Wigner-Ville kernel. A RKHS is Hilbert space of complex valued functions, defined on a set , that has a reproducing kernel defined on with two properties: (i) for each t, the function K(s,t) lies in and (ii) for each and each one has the reproducing property:

In our case, the Hilbert space, , is the set of Weyl symbols of underspread operators satisfying a given spreading constraint (1). The reproducing kernel is given by the Weyl symbol of the prototype operator:

This is in fact a reproducing kernel as (i) for each remains underspread since

and (ii) one has the reproducing formula as follows:


WERNER: DO YOU MEAN 1 for the Wigner Ville kernel or for the kernel in2 or WHAT?

6 Qtfi Estimation

We now consider QTFI estimators of the time varying spectrum of the signal process when it is contaminated with noise. We are given a single noisy observation, of the signal process :

where is statistically independent, zero–mean, circular complex Gaussian white noise. To estimate we use a generally different QTFI representation of the observation:

We define the “bias operator” as The QTFI estimator is consistent with classical, “non–parametric” time–invariant spectrum estimation where the predominant class of estimators [22, 18] can be basically written as a frequency parametrized quadratic form:


7 Bias and Variance Analysis

With the statistical independence of signal and noise and using (2) we have the following expectation of the estimate:


such that the bias is given by

Using the Schwarz inequality for operator inner products and triangle inequality, we immediately get a tight bound for the maximum bias:


where the operator norm is the HS norm. We assume knowledge of the noise level such that we can trivially correct the TF–independent bias term:

where denotes the corrected estimate.

The variance,

is evaluated using of Isserlis’ fourth order moment formula (for the special case of circular complex variables),

one has:

The Schwarz inequality for the operator inner product leads to a bound on the maximum variance,


proportional to the HS norm of the prototype operator .

Global Mean Square Error. The bias and variance results are complicated TF–dependent expressions. Due to our restriction to QTFI estimators we need TF–invariant, thus global indicators for the estimator performance. After correcting for the TF independent bias term, we characterize the global square bias as follows:


Just as for the bias we give a global characterization of the variance. The TF independent term is given by:

We define a total variance as the integral over the TF dependent variance terms, one has:


Equations (4) and(5) are derived in the appendix.

Observe that any of the global variance constants; i.e., the maximum variance , the TF–independent variance term , and the total variance are proportional to the HS norm of the prototype operator:


8 Estimator Optimization

Classical spectrum estimation produces smooth spectra since — due to the absence of a model — smoothing is the actual tool for variance reduction. The proposed estimators usually are the result of mean–squared error considerations. In the present work, we deviate from this point of view in a pragmatic way: we restrict ourselves to underspread processes whose true spectra are itself smooth (in the sense of 2D bandlimitation) such that there exist a whole class of unbiased estimators. While such a modelling ingredient may be questionable for time–invariant spectrum analysis we feel that it is necessary for time–varying spectral estimation. The reason lies in the often overlooked point that frequency parametrization is matched to any stationary process (the Fourier transform diagonalizes the correlation operator) while TF parametrization is not matched to a general nonstationary process. From the point of view of operator diagonalization it is the class of underspread processes where TF–parametrization is appropriate [11].

Unbiased estimation without further assumption on the signal process requires a vanishing “bias operator”, i.e., . In the case of the generalized Wigner–Ville spectrum, the prototype operator (cf. (6)) is not HS since


so that one can exclude finite–variance unbiased estimation of the generalized Wigner–Ville spectrum without a priori knowledge on the process. This is well–known [13].

Based upon the known support of one has a large class of nontrivial unbiased estimators (with nonvanishing “bias operator”, )

We interpret minimum variance in the sense of the combined consideration of the global variance constants . Due to (6) one has to select the unbiased estimator with minimum HS norm prototype operator. Using (1) this turns out to be trivial: the minimum–variance unbiased (MVUB) QTFI estimator is obtained by setting the spreading function of the prototype operator zero wherever possible:

When is the smallest indicator function containing the support of , then the MVUB QTFI estimator can be written as:

In particular, for the –parametrized real–valued generalized Wigner–Ville spectrum one has:

This estimator is optimal among all QTFI estimators thus in the sense of global variance minimization. The estimate is locally stable since it minimizes a bound on the maximum variance () and it is unbiased for arbitrary time and frequency, but it deviates from the local TF–dependent MVUB estimate.

Mean–Squared Error. The theoretical MVUB estimator serves well as a starting point for obtaining practical estimators with good mean–squared error performance. The mean squared error is given by . For any process that satisfies the spreading constraint (1) one can formally redefine the estimation target via the prototype operator of any unbiased estimator:

so that one can obtain a useful bound on the integrated mean–squared error


with . This bound is based on (4), (5) and .

9 Matched Multi-Window Estimator

The eigenfunction decomposition of the prototype operator

shows that is a weighted sum of rank one projections. Equivalently, any QTFI representation can be written as a weighted sum of spectrograms with orthonormal windows [20]. For practicality, we require our estimator to be based on a finite–rank prototype operator with finite–length eigenfunctions. The MVUB estimator of Sec. 6 does not satisfy these requirements. Thus, we choose the finite–rank, time–limited estimator which minimizes the upper bound on the integrated mean–squared error as given by (2). When we impose the additional requirement that the prototype operator be projection type with normalized trace, has the representation:


where denotes the rank–one projection on the orthonormal window functions and is the rank. In this case, , and the optimization of (2) reduces to minimizing subject to orthonormality constraints on the . We define the matched multi-window estimator as the quadratic form based on a prototype operator the minimizes subject to (1). The optimization is performed in a two step procedure: we optimize the windows subject to a fixed rank and then we optimize the rank. For practicality, we impose that the have support on . To impose this time localization on the optimization of , we define as the projection onto the centered interval and require . Minimizing (2) yields the optimal windows equation:


The optimum window set is independent of . For the specific case where is an ideal bandpass (which may be considered as a theoretically optimal estimator for stationary processes) (2) yields the time–limited and optimally bandlimited prolate spheriodal wave functions consistent with [22, 18].

A more realistic and simpler family of tapers are the discrete sinusoidal tapers, , where , and is the number of points [17]. The resulting sinusoidal multi-taper spectral estimate is , where is the local Fourier transform centered at time with length . is the instantanteous spectral density, and is the number of tapers. The sinusoidal tapers are asymptotically optimal when the bias error is local.

10 Free Parameter Optimization

For a strongly underspread process , is approximately constant in the support of . Using the optimal window functions of (2), we approximate with such rank one projections. In this case, reduces to . Optimizing (2) with respect to for moderate noise level yields


11 Conclusions

We have studied time–varying spectral estimation via quadratic TF–invariant estimators. For circular complex Gaussian signal and noise processes we have presented explicit (local and global) bias and variance results. For the specific case of an underspread process the design of matched multi-window estimators has been based on approximating a theoretical MVUB estimator.

The theoretical MVUB estimator as derived in Section 6 is a specific case of the recently proposed optimum kernel design for Wigner–Ville spectrum estimation [19]. We emphasize that [19] requires a complete knowledge of a second order statistic what makes this approach purely theoretical while our proposed estimator uses a more realistic, incomplete a priori knowledge of the process statistics.

For Cohen’s class time varying spectra. Using the reproducing kernel Hilbert space formalism, we derive expressions for the leading order bias and variance. Underspread processes are band limited in the ambiguity plane and smooth in the time frequency domain. For underspread processes, we give unbiased minimum variance estimators.


We now derive (4) which equates the integral square bias with the inner product of the squared GSF of the “bias operator” and the process’ expected ambiguity function:

Derivation of (5):

together with

(which follows directly from the trace formula of the Weyl correspondence [6]); as well as



  • [1] L. Cohen, “Generalized phase–space distribution functions,” J. Math. Phys., Vol. 7, 1966, pp. 781-786.
  • [2] Cohen, L. (1989). “Time-frequency distributions - a review.” Proc. I.E.E.E. Vol. 77, 941-981.
  • [3] F. Hlawatsch and W. Kozek, “Time–frequency weighting and displacement effects in linear time–varying systems,” Proc. IEEE ISCAS–92, Mai 1992, pp. 1455–1458.
  • [4] Hlawatsch, F. and Boudreaux-Bartels G.F. (1992). “Linear and quadratic time-frequency representations.” I.E.E.E. Signal Processing Mag. 9, 21-67.
  • [5] T. Kailath, “RKHS Approach to Detection and Estimation Problems—Part V: Parameter Estimation,” IEEE Trans. Info. Theory, Sept. 1971, pp. 530–548.
  • [6] W. Kozek, “On the generalized Weyl correspondence and its application to time–frequency analysis of linear time–varying systems,” Proc. IEEE Int. Symp. on Time–Frequency and Time–Scale Analysis, Victoria, Canada, Oct. 1992, pp. 167–170.
  • [7] W. Kozek, “Optimally Karhunen–Loeve–like STFT expansion of nonstationary processes,” Proc. IEEE ICASSP–93, Minneapolis, MN, April 1993, Vol. 4, pp. 428–431.
  • [8] W. Kozek, F. Hlawatsch, H. Kirchauer and U. Trautwein, “Correlative time–frequency analysis and classification of nonstationary random processes,” Proc. IEEE Workshop on Time–Frequency/Time–Scale Analysis, Philadelphia, PA, (1994), 417–420.
  • [9] W. Kozek, H. G. Feichtinger and T. Strohmer, “Time–frequency synthesis of statistically matched Weyl–Heisenberg prototype signals,” this issue.
  • [10] W. Kozek, “Matched generalized Gabor expansion of nonstationary processes,” Proc. IEEE Int. Conf.  Signals, Systems, and Computers, Pacific Grove, CA, Nov. 1993, pp. 499–503.
  • [11] W. Kozek, “Matched Weyl–Heisenberg Expansions of Nonstationary Environments,” Ph.D. Thesis, University of Technology Vienna, under review.
  • [12] W. Kozek and K.S. Riedel, Quadratic time–varying spectral estimation for underspread processes; Proc. IEEE Workshop on Time–Frequency/Time–Scale Analysis, Philadelphia, PA, (1994), 460–463.
  • [13] W. Martin and P. Flandrin,“Wigner–Ville spectral analysis of nonstationary processes,” IEEE Trans. Acoust., Speech, Signal Processing, Vol. 33, No. 6, Dec. 1985, pp. 1461–1470.
  • [14] M. Priestley, “Evolutionary spectra and non–stationary processes,” J. Roy. Statist. Soc. Ser. B, Vol. 27, 1965, pp. 204–237.
  • [15] K. S. Riedel, “Optimal data based kernel estimation of evolutionary spectra,” IEEE Trans. Signal Processing, Vol. 41, No. 7, July 1993, pp. 2439–2447.
  • [16] K. S. Riedel, “Optimal kernel estimation of the instantaneous frequency.” I.E.E.E. Trans. Signal Processing, Vol. 42 p. 2644-2649, (Oct. 1994).
  • [17] K.S. Riedel, A. Sidorenko, “Minimum bias multiple taper spectral estimation.” I.E.E.E. Trans. Signal Processing Vol. 43, p. 188-195, (Jan. 1995).
  • [18] K.S. Riedel, A. Sidorenko, D.J. Thomson, “Spectral estimation of plasma fluctuations I: Comparison of methods.” Physics of Plasmas, Vol. 1, March 1994, pp. 485-500.
  • [19] A. M. Sayeed and D. L. Jones, “Optimal kernels for Wigner–Ville spectral estimation,” Proc. IEEE ICASSP–94, Sydney, Australia, April 1994.
  • [20] R. G. Shenoy and T.W. Parks, “The Weyl correspondence and time–frequency analysis,” IEEE Trans. Signal Processing, Vol. 42, No. 2, Feb. 1994, pp. 318–331.
  • [21] K. A. Sostrand, “Mathematics of the time–varying channel,” Proc. of NATO Adv. Study Inst. on Sign. Proc. Vol. 2, Enschede (The Netherlands), 1968.
  • [22] D. J. Thompson,“Spectrum estimation and harmonic analysis,” Proc. IEEE, Vol. 70, No. 9, Sept. 1982, pp.1055–1096.