In this paper, we consider the combined state and drift estimation problem for the stochastic heat equation
over the domain and for , with zero Dirichlet boundary conditions and zero initial condition for all , without loss of generality. We denote by a cylindrical Wiener process where characterises the strength of the model error. The unknown drift function is assumed to belong to a Sobolev space of appropriate regularity. We specify the properties of and more precisely in Section 2 below.
Linear stochastic partial differential equations (SPDEs) of the form (1) are well understood from analytical and numerical perspectives. See, for example, Da Prato & Zabczyk (1992); Hairer (2009); Lord et al. (2014). In this paper, we specifically focus on the estimation of the states , , as well as the unknown drift function from noisy state measurements given by
where for all , denotes the strength of the measurement noise, and is another cylindrical Wiener process independent of the model error .
Our focus is on asymptotic posterior contraction rates, which have been widely studied in the context of non-parametric stationary inverse problems. See, for example, Knapik et al. (2011); Giné & Nickl (2016); Ghosal & van der Vaart (2017). We note that pure parameter estimation problems for SPDEs given exact observations of the states are also well-studied. See, for example, Cialenco (2018) for a recent survey. A first step towards non-parametric time-dependent inverse problems has been taken in Yan (2019). However, there they are framed as inference problems over a fixed time interval resulting in smoothing problems, whereas filtering problems are the focus of this paper. Furthermore, state and drift estimation are treated separately. In addition, parameter estimation for a class of linear SPDEs from noiseless local state measurements has been considered in Altmeyer & Reiß (2019), whereas we assume noisy observations of the full states throughout this paper.
In this paper, the infinite-dimensional, time-dependent combined state and drift estimation problem is formulated and analysed in terms of the well-known Kalman-Bucy filter equations for the posterior mean and covariance operator in an augmented state space. See, for example, Jazwinski (1970); Simon (2006); Curtain (1975) for an introduction to the Kalman-Bucy filter in finite and infinite dimensions. More recently, the Kalman-Bucy filter has been reformulated as a set of mean-field equations termed the ensemble Kalman-Bucy filter. See, for example Bergemann & Reich (2012); Wiljes et al. (2018); Nüsken et al. (2019). These mean-field equations allow for a concise formulation of our time-dependent estimation problem. Furthermore, although not considered in this paper, the Kalman-Bucy mean-field equations can be generalised to non-Gaussian estimation problems (Yang et al., 2013; Taghvaei et al., 2017). A key observation arising from the analysis of the Kalman-Bucy mean-field equations is the time-scale separation in the dynamics of the state and parameter variables, allowing us to apply the concept of slow manifolds (Verhulst, 2007).
The remainder of this paper is structured as follows. A mathematical formulation of the combined state and drift estimation problem in terms of Fourier modes is provided in Section 2. Furthermore, the time-dependent estimation problem is formulated in terms of Kalman–Bucy mean-field equations. This formulation is applied to a stationary drift estimation problem in Section 3. Asymptotic posterior contraction rates are then derived and compared to existing results for a closely related non-parametric Bayesian inference problem. These results are extended to the combined state and drift estimation problem in Section 4. The essential analysis of the single-mode Kalman-Bucy filter equations is carried out in Section 4.1. A numerical exploration of the combined state and parameter estimation problem is carried out in Section 5. The paper concludes with Section 6.
2. Mathematical problem formulation
In this paper, we analyse the state and drift estimation problem using a spectral representation in terms of Fourier modes. We provide the essential background in this section. It is well known that solutions of (1) can be expanded in Fourier modes , that is,
Because of (1), the Fourier modes obey the stochastic differential equations (SDEs)
Here , , denote the Fourier coefficients of , and independent standard Brownian motions. The initial conditions are for all . Therefore (1) can be viewed as a stochastic evolution equation on the separable Hilbert space of square integrable functions of sufficient spatial regularity with zero boundary conditions.
The measurement model (2) can also be transformed into Fourier space, yielding
with representing independent standard Brownian motions, independent of the model errors for all . Throughout this paper, we will exclusively work with the formulation (4) and (5) of our state and drift estimation problem in Fourier space. We now proceed to formulate a continuous-time Bayesian inference framework for this problem, which is used to derive asymptotic posterior contraction rates in Section 4.
The posterior Bayesian random variables for the state and drift at time, conditioned on the observed data , , are denoted by and , respectively. Here represents all the data up to time , where
with the innovation given by
and Kalman gains
Denoting the distribution of the joint random variable at time by , we define
for any measurable function . Hence, for example, , and
denotes the variance of.
The prior assumptions are that almost surely, and that the , , are independent Gaussian random variables with mean zero and variance
for a suitable . Since (4) and (5) are linear in the unknowns, and remain Gaussian under the evolution equations (7), and we investigate their behaviour in terms of their mean and covariance for . We further assume that the true drift function has Sobolev regularity , that is
For example, (15) is satisfied if
for any and any sequence of coefficients with bounded as . These coefficients can, for example, be realisations of i.i.d. uniform random variables from the interval .
In addition to an analysis of the continuous-time Bayesian inference problem (7), we also study the (linear) dependence of the estimators (Bayesian posterior means)
These two quantities characterise the systematic bias in the estimators (17), while the implied variances
are a measure of the ‘frequentist’ uncertainty of the estimators (17).
According to Lemma 8.2 from Ghosal & van der Vaart (2017), the posterior contraction rate in the th Fourier mode of the drift estimator , that is,
is provided by
Equation (20) holds for any nonnegative function with . Here denotes the variance of .
The main contributions from our subsequent analysis consist of providing bounds for the three quantities , , and in terms of the statistical models (4) and (5), and additionally studying the asymptotic behaviour of the asymptotic contraction rate , defined by
in terms of the Sobolev regularity of the true drift function and the variance of the prior characterised by in (14). Here, the asymptotic contraction rate is to be understood in the sense that
for any nonnegative function with . In physical space,
denotes the inverse Fourier transform of.
We first discuss a stationary formulation of the drift estimation problem in Section 3, which is closely linked to available results for non-parametric Bayesian inverse problems (Knapik et al., 2011; Giné & Nickl, 2016; Ghosal & van der Vaart, 2017). The combined state and drift estimation problem is analysed in Section 4.
A more general class of time-scaled prior variances over the unknown set of static parameters has been considered in Knapik et al. (2011). This more general class of prior distributions could be incorporated into our analysis as well. However, we restrict the subsequent analysis to the simpler case (14).
3. Stationary drift estimation problem
Before addressing the full time-dependent problem, we first consider the stationary drift estimation problem in which and in (4), giving rise to the forward model
for . The forward model (26) leads to a mildly ill-posed inverse problem. We follow a Bayesian approach by placing a Gaussian mean-zero prior with variance given by (14) on the Fourier coefficients of the unknown drift function. Let us summarise our assumptions for ease of reference in the following:
Let us denote the Bayesian posterior estimate of the drift at time by , . We recall that is Gaussian for all provided is Gaussian. Additionally, recall that we denote the mean of by and the variance by . The Kalman–Bucy evolution equations (7) reduce to the following equations for the mean and the variance of the th Fourier mode:
where denotes the Kalman gain factor
The initial conditions are provided by Assumption 3.1.
Alternatively, the Kalman–Bucy equations (27) for the mean and the variance can be reformulated as mean-field equations in directly, that is,
These equations are a special case of (7), and provide the starting point for numerical implementation in the form of the ensemble Kalman–Bucy filter (Bergemann & Reich, 2012), as well as extensions to nonlinear and non-Gaussian Bayesian inference problems (Yang et al., 2013; Taghvaei et al., 2017).
The evolution equation (27b) for the posterior variance has the closed form solution
The trace of the covariance of the full joint posterior process thus satisfies the asymptotic estimate
Next, we carry out a ‘frequentist’ analysis of the mean in terms of its bias with respect to the true , and its variance with respect to the measurement noise . We rewrite (27a) as
Denoting the expectation value of with respect to the measurement noise by , we obtain
with initial condition . Equation (33) has the closed-form solution
Hence, based on (16), the -norm of the frequentist bias satisfies the asymptotic estimate
This result follows again from asymptotic bounds for infinite sums.
We finally investigate the time evolution of the frequentist variance . Starting from the stochastic differential equation
using Itô’s lemma yields
The initial variance is for all . In order to analyse the solution behaviour of (37), we introduce and find the associated evolution equation
from which we conclude that , and
In fact, the explicit solution of (37) is
The ‘frequentist’ uncertainty is characterised by
so the probability of the event
has probability tending to zero for any nonnegative function with . Here
denotes the Gaussian distribution with meanand variance .
where are i.i.d. Gaussian random variables with mean zero and variance one. The associated inference problem corresponds to a sequence of measurements with measurement error decreasing as . In our continuous-time problem, we obtain the same asymptotic rates as in the case considered above with under the formal equivalence .
4. Time-dependent state and drift estimation
We now return to the full dynamic model (4) subject to observations (5). We primarily wish to estimate the drift function . However, because of the stochastic model errors in (4), we also need to estimate the states . We start with a careful analysis of the single mode system for both small- and large- Fourier modes.
4.1. Analysis of the single-mode filtering problem
In this section, we conduct a careful analysis of the single-mode filtering and parameter estimation problem. We suppress the dependence on the mode number , and introduce the parameter . The signal process is therefore given by
with given initial condition almost surely. Observations of the process are given by
with . The complete observation path up to time is denoted by .
Recall that the mean-field Kalman–Bucy equations are given by (7). We again drop the dependence on the Fourier mode number in the subsequent analysis. The mean-field equations (7) give rise to the following evolution equations in the conditional means and :
The deviations and thus satisfy
and introducing the shorthand
, one is left with the ordinary differential equations
for the mean values , as well as
for the deviations . The expectation value satisfies the evolution equation
while the deviation satisfies
We have thus decomposed the mean-field Kalman–Bucy equations into the analysis of the three subsystems (49), (51) and (55). Here, (51) describes the systematic bias in the Bayesian mean estimator, while (55) and (49) characterise the ‘frequentist’ and ‘Bayesian’ uncertainties, respectively.
and the covariance
which are given by
The time-dependent linear stochastic differential equations (55) can also be analysed in terms of the variances
and the covariance
These quantities satisfy the linear time-dependent ordinary differential equations
in the variables
The initial conditions are , and .
It holds that
as , that is, is a stable equilibrium point of (62).
The lemma follows from the decay property of
and the fact that for sufficiently large with an appropriate constant . In particular,
and for the gradient :
Therefore, the total time derivative satisfies
Furthermore, for large enough ,
and becomes small relative to (67) since . Hence, for again sufficiently large and with an appropriate constant ,
and the desired convergence result follows. ∎
Lemma 4.1 implies that, as for the pure drift estimation problem, the Bayesian (filtering) variance asymptotically covers the ‘frequentist’ (data) variance.
4.1.1. The large- Fourier mode case
We now investigate the case in more detail. The equations (58) possess a slow manifold for sufficiently small (Verhulst, 2007; Shchepakina et al., 2014). To leading order, solutions on the slow manifold satisfy
In other words, the long-time variance of is governed by , and of by
up to higher-order terms in . Here we have introduced the new Kalman gain factor
We also notice that the Kalman gain factor satisfies . See the Appendix for more details on the derivation of the reduced system (72).