EM algorithm for stochastic hybrid systems

1 Introduction

A stochastic hybrid system (SHS, hereafter), also known as a switching diffusion [3], is a continuous-time Markov process $Z$ with state space $S = {e_{1}, \dots, e_{k}} \times R^{d}$ consisting of both discrete and continuous parts, namely, ${e_{1}, \dots, e_{k}}$ and $R^{d}$ respectively. The elements ${e_{i}}$ are, without loss of generality, specified as the standard basis of $R^{k}$ in this article. Denoting by $⟨ \cdot, \cdot ⟩$ the inner product of $R^{k}$ or $R^{d}$ , $⟨ e_{i}, e_{j} ⟩ = δ_{i j}$ , where $δ_{i j}$ is Kronecker’s delta. The discrete part of $Z$ , denoted by $X$

, can be seen as a continuous-time semi-Markov chain with state space

{e_{1}, \dots, e_{k}}

and “

Q

matrix” of the form

Q (Y_{t}) = [q_{i j} (Y_{t})]

, where

Y

is the continuous part of

Z

. In other words,

P (X_{t + h} = e_{j} | X_{t} = e_{i}, Y_{t} = y) = (δ_{j i} + q_{j i} (y)) h + o (h)

(1)

as $h \to 0$ . Here, $Q (y) = [q_{i j} (y)]$ is a $Q$ matrix for each $y$ , that is, $q_{j i} (y) \geq 0$ for $j \neq i$ and $\sum_{j \neq i} q_{j i} (y) = - q_{i i} (y)$ for all $y \in R^{d}$ . The continuous part $Y$ is a semi-Markov process on $R^{d}$ and defined as the solution of a stochastic differential equation

˙ Y = μ (X, Y) + ϵ ˙ W

for some $R^{d}$ -valued function $μ$ , where $ϵ > 0$ and $˙ W$ is a $d$

dimensional white noise. The generator

L

of this Markov process is given by

L f (e_{i}, y) = ⟨ μ, \nabla_{y} f ⟩ (e_{i}, y) + \frac{1}{2} ϵ^{2} Δ_{y} f (e_{i}, y) + k \sum j = 1 (f (e_{j}, y) - f (e_{i}, y)) ⟨ e_{j}, Q (y) e_{i} ⟩ .

(2)

There is a huge amount of literature on the analysis and applications of SHS. See e.g., [3, 4, 7, 5] and the references therein. The author’s motivation to study SHS is its potential application to the analysis of single-molecule dynamics which has several unobservable switching states [2]. In this article, we consider parametric estimations of the Q matrix $Q (y) = Q^{θ} (y)$ and of the drift coefficient $μ (z) = μ^{θ} (z)$ based on a partial observation where $Y$ is monitored continuously in time, while $X$ is unobserved. When both $Q$ and $μ$ do not depend on $y$ , the system is a hidden Markov model studied in Elliot et al. [1]. Extending results developed in [1], we derive a finite-dimensional filter and the EM algorithm for the SHS. In Section 2, we describe the basic properties of SHS as the solution of a martingale problem. In Section 3, we derive the likelihood function under complete observations of both $X$ and $Y$ on a time interval $[0, T]$ . In Section 4, we consider the case where the discrete part $X$ is unobservable, and construct a finite dimensional filter extending Elliot et al. [1]. In Section 5, again by extending Elliot et al. [1], we construct the EM algorithm for parametric estimations under the partial observation.

2 A construction as a weak solution

Here we construct a SHS as a weak solution, that is, we construct a distribution on the path space $D ([0, T]; S)$ which is a solution of the martingale problem with the generator (2).

A direct application of Theorem (5.2) of Stroock [6] provides the following.

Theorem 2.1

Let $μ$ be a bounded Borel function and $q_{i j}$ , $1 \leq i, j \leq k$ be bounded continuous functions. Then, for any $z \in S$

, there exists a unique probability measure

P_{z}

D ([0, T]; S)

such that

Z_{0} = z

and

f (Z_{t}) - \int_{0}^{t} L f (Z_{s}) d s

is a martingale under $P_{z}$ for any $f \in C_{0}^{0, \infty} ({e_{1}, \dots, e_{k}} \times R^{d})$ , where $Z : t \mapsto Z_{t}$ is the canonical map on $D ([0, T]; S)$ . Moreover, $Z$ is a strong Markov process with ${P_{z}}_{z \in S}$ .

The uniqueness part of Theorem 2.1 is important in this article. For the existence, we give below an explicit construction, which plays a key role to solve a filtering problem later.

First, we construct a SHS with $μ = 0$ in a pathwise manner. Without loss of generality, assume $ϵ = 1$ . Note that $Y$ is then a $d$ dimensional Brownian motion. Let $(Ω, F, P^{0})$ be a probability space on which a $d$ dimensional Brownian motion $Y$

and an i.i.d. sequence of exponential random variables

{E_{n}}

that is independent of

Y

are defined. Conditionally on

Y

, a time-inhomogeneous continuous-time Markov chain

X

with (1) is defined using the exponential variables. More specifically, given

X_{0} = e_{i}

, let

τ_{1} = min 1 \leq j \leq k τ_{1}^{j}, τ_{1}^{j} = inf {t > 0; \int_{0}^{t} q_{j i} (Y_{s}) d s > E_{j}}

and $X_{t} = X_{0}$ for $0 \leq t < τ_{1}$ , $X_{τ_{1}} = e_{J}$ with $J = a r g m i n τ_{1}^{j}$ . The construction goes in a recursive manner; given $X_{τ_{n}} = e_{i}$ , let

τ_{n + 1} = min 1 \leq j \leq k τ_{n + 1}^{j}, τ_{n + 1}^{j} = inf {t > τ_{n}; \int_{τ_{n}}^{t} q_{j i} (Y_{s}) d s > E_{n k + j}}

and $X_{t} = X_{τ_{n}}$ for $τ_{n} \leq t < τ_{n + 1}$ , $X_{τ_{n + 1}} = e_{J}$ with $J = a r g m i n τ_{n + 1}^{j}$

. Properties of the exponential distribution verifies the following lemma.

Lemma 2.1

Assume $q_{i j} (y)$ is bounded and continuous in $y \in R^{d}$ for each $(i, j)$ . Then,

P^{0} (X_{t + h} = e_{j} | X_{t} = e_{i}, Y) = (δ_{j i} + q_{j i} (Y_{t})) h + o (h)

(3)

and (1) with $P = P^{0}$ .

By Itô’s formula, for any $f \in C_{b}^{0, 2} ({e_{1}, \dots, e_{k}} \times R^{d})$ , we have

\begin{matrix} f (X_{t + h}, Y_{t + h}) = f (X_{t}, Y_{t}) + & \int_{t}^{t + h} ⟨ \nabla_{y} f (X_{s}, Y_{s}), d Y_{s} ⟩ + \frac{1}{2} ϵ^{2} \int_{t}^{t + h} Δ_{y} f (X_{s}, Y_{s}) d s + \sum t < s \leq t + h (f (X_{s}, Y_{s}) - f (X_{s -}, Y_{s})), \end{matrix}

from which together with (1) it follows

lim h \to 0 \frac{E^{0} [f (X_{t + h}, Y_{t + h}) | X_{t} = e_{i}, Y_{t} = y] - f (e_{i}, y)}{h} = L^{0} f (e_{i}, y),

where $E^{0}$ is the expectation under $P^{0}$ and $L^{0} f = L f$ with $μ = 0$ in (2). Note only this, we have also that

U_{t}^{0, f} := f (X_{t}, Y_{t}) - \int_{0}^{t} L^{0} f (X_{s}, Y_{s}) d s

is a martingale with respect to the filtration ${F_{t}}$ generated by $Z = (X, Y)$ . Even more importantly, Lemma 2.1 implies the following.

Lemma 2.2

Under the same conditions of Lemma 2.1, for any $g \in C^{0} ({e_{1}, \dots, e_{k}})$ ,

V_{t}^{0, g} := g (X_{t}) - \int_{0}^{t} L^{0} g (X_{s}) d s

is a martingale with respect to the natural filtration of $X$ under the conditional probability measure given $Y$ , where $L^{0} g = L^{0} f$ with $f (x, y) = g (x)$ . In particular,

X_{t} - \int_{0}^{t} Q (Y_{s}) X_{s} d s

is a martingale under the conditional probability measure $P^{0} (\cdot | Y)$ .

Now we construct a SHS for a general bounded Borel function $μ$ . Let

Λ_{T} = exp {\frac{1}{ϵ^{2}} \int_{0}^{T} ⟨ μ (X_{t}, Y_{t}), d Y_{t} ⟩ - \frac{1}{2 ϵ^{2}} \int_{0}^{T} | μ (X_{t}, Y_{t}) |^{2} d t} .

(4)

By the boundedness of $μ$ , Novikov’s conditions is satisfied and so, $Λ$ is an ${F_{t}}$ -martingale under $P^{0}$ . Therefore,

\frac{d P}{d P^{0}} = Λ_{T}

defines a probability space $(Ω, F_{T}, P)$ .

Theorem 2.2

Let $Q = [q_{i j}]$ be a Q matrix-valued bounded continuous function and $μ$ be an $R^{d}$ -valued bounded Borel function. Under $P$ , $Z = (X, Y)$ is a Markov process with generator (2). Further for any $f \in C_{b}^{0, 2} ({e_{1}, \dots, e_{k}} \times R^{d})$ ,

U_{t}^{f} := f (Z_{t}) - \int_{0}^{t} L f (Z_{s}) d s

is an ${F_{t}}$ martingale.

Proof: By the Bayes formula,

E [f (Z_{t + h}) | F_{t}] = \frac{E^{0} [Λ_{T} f (Z_{t + h}) | F_{t}]}{E^{0} [Λ_{T} | F_{t}]} = E^{0} [\frac{Λ_{t + h}}{Λ_{t}} f (Z_{t + h}) | F_{t}] .

Since

\frac{Λ_{t + h}}{Λ_{t}} = exp {\frac{1}{ϵ^{2}} \int_{t}^{t + h} ⟨ μ (Z_{s}), d Y_{s} ⟩ - \frac{1}{2 ϵ^{2}} \int_{t}^{t + h} | μ (Z_{s}) |^{2} d s}

and $Z$ is Markov under $P^{0}$ , $E [f (Z_{t + h}) | F_{t}] = E [f (Z_{t + h}) | Z_{t}]$ , meaning that it is Markov under $P$ as well. By Itô’s formula,

d Λ_{t} = \frac{1}{ϵ^{2}} Λ_{t} μ (Z_{t}) d Y_{t}

and

\begin{matrix} Λ_{t + h} f (Z_{t + h}) = & Λ_{t} f (Z_{t}) + \int_{t}^{t + h} f (Z_{s}) d Λ_{s} + \int_{t}^{t + h} Λ_{s -} d U_{s}^{0, f} + \int_{t}^{t + h} Λ_{s} L^{0} f (Z_{s}) d s + \int_{t}^{t + h} Λ_{s} ⟨ μ, \nabla_{y} f ⟩ (Z_{s}) d s . \end{matrix}

Therefore,

Λ_{t + h} U_{t + h}^{f} = Λ_{t} U_{t}^{f} + \int_{t}^{t + h} U_{s}^{f} d Λ_{s} + \int_{t}^{t + h} Λ_{s -} d U_{s}^{0, f},

meaning that $Λ U^{f}$ is a martingale under $P^{0}$ . The Bayes formula then implies that $U^{f}$ is a martingale under $P$ . In particular, the generator is given by $L$ . ////

Corollary 2.1

Under the same condition of Theorem 2.2,

V_{t} := X_{t} - \int_{0}^{t} Q (Y_{s}) X_{s} d s

is a martingale.

By the uniqueness result of Theorem 2.1, the law of $Z$ under $P$ coincides with $P_{z}$ with $z = Z_{0}$ .

3 The likelihood under complete observations

Here we consider a statistical model ${P^{θ}}_{θ \in Θ}$ and derive the likelihood under complete observation of a sample path $Z = (X, Y)$ on a time interval $[0, T]$ . For each $θ \in Θ$ , $P^{θ}$ denotes the distribution on $D ([0, T]; S)$ induced by a Markov process $Z$ with generator

L^{θ} f (e_{i}, y) = ⟨ μ^{θ}, \nabla_{y} f ⟩ (e_{i}, y) + \frac{1}{2} ϵ^{2} Δ_{y} f (e_{i}, y) + k \sum j = 1 (f (e_{j}, y) - f (e_{i}, y)) ⟨ e_{j}, Q^{θ} (y) e_{i} ⟩,

where $μ^{θ}$ is a family of $R^{d}$ -valued bounded Borel functions and $Q^{θ} = [q_{i j}^{θ}]$ is a family of $Q$ matrix-valued bounded continuous functions. Note that $ϵ > 0$ is almost surely identified from a path of $Y$ by computing its quadratic variation. It is therefore assumed to be known hereafter. The initial distribution $P^{θ} \circ Z_{0}^{- 1}$ is also assumed to be known and not to depend on $θ$ .

Theorem 3.1

Let $θ, θ_{0} \in Θ$ and assume that

y \mapsto \frac{q_{i j}^{θ} (y)}{q_{i j}^{θ_{0}} (y)}, y \mapsto \frac{q_{i j}^{θ_{0}} (y)}{q_{i j}^{θ} (y)}

are bounded for each $(i, j)$ , where $0 / 0 = 1$ . Then, $P^{θ}$ is equivalent to $P^{θ_{0}}$ , and the log likelihood

L_{T} (θ, θ_{0}) := log \frac{d P^{θ}}{d P^{θ_{0}}} ({Z_{t}}_{t \in [0, T]})

is given by

\begin{matrix} L_{T} (θ, θ_{0}) = & k \sum i, j = 1 ⎧ ⎨ ⎩ \int_{0}^{T} log \frac{q_{j i}^{θ} (Y_{t})}{q_{j i}^{θ_{0}} (Y_{t})} d N_{t}^{j i} - \int_{0}^{T} (q_{j i}^{θ} (Y_{t}) - q_{j i}^{θ_{0}} (Y_{t})) ⟨ X_{t}, e_{i} ⟩ d t ⎫ ⎬ ⎭ + \frac{1}{ϵ^{2}} \int_{0}^{T} ⟨ μ^{θ} (Z_{t}) - μ^{θ_{0}} (Z_{t}), d Y_{t} ⟩ - \frac{1}{2 ϵ^{2}} \int_{0}^{T} (| μ^{θ} (Z_{t}) |^{2} - | μ^{θ_{0}} (Z_{t}) |^{2}) d t, \end{matrix}

where $N^{j i}$ is the counting process of the transition from $e_{i}$ to $e_{j}$ :

N_{t}^{j i} = \int_{0}^{t} ⟨ X_{s -}, e_{i} ⟩ ⟨ e_{j}, d X_{s} ⟩ .

(5)

Proof: The proof is standard but given for the readers’ convenience. Let

L_{τ}^{j i} = \int_{0}^{τ} log \frac{q_{j i}^{θ} (Y_{t})}{q_{j i}^{θ_{0}} (Y_{t})} d N_{t}^{j i} - \int_{0}^{τ} (q_{j i}^{θ} (Y_{t}) - q_{j i}^{θ_{0}} (Y_{t})) ⟨ X_{t}, e_{i} ⟩ d t

and

\begin{matrix} L_{τ}^{0} & = \frac{1}{ϵ^{2}} \int_{0}^{τ} ⟨ μ^{θ} (Z_{t}) - μ^{θ_{0}} (Z_{t}), d Y_{t} ⟩ - \frac{1}{2 ϵ^{2}} \int_{0}^{τ} (| μ^{θ} (Z_{t}) |^{2} - | μ^{θ_{0}} (Z_{t}) |^{2}) d t = \frac{1}{ϵ^{2}} \int_{0}^{τ} ⟨ μ^{θ} (Z_{t}) - μ^{θ_{0}} (Z_{t}), d Y_{t} - μ^{θ_{0}} (Z_{t}) d t ⟩ - \frac{1}{2 ϵ^{2}} \int_{0}^{τ} | μ^{θ} (Z_{t}) - μ^{θ_{0}} (Z_{t}) |^{2} d t . \end{matrix}

By Itô’s formula,

\begin{matrix} exp {L_{τ}^{j i}} & = 1 - \int_{0}^{τ} exp {L_{t}^{j i}} (q_{j i}^{θ} (Y_{t}) - q_{j i}^{θ_{0}} (Y_{t})) ⟨ X_{t}, e_{i} ⟩ d t + \sum 0 < t \leq τ (exp {L_{t}^{j i}} - exp {L_{t -}^{j i}}) = 1 + \int_{0}^{τ} exp {L_{t -}^{j i}} ⎛ ⎜ ⎝ \frac{q_{j i}^{θ} (Y_{t})}{q_{j i}^{θ_{0}} (Y_{t})} - 1 ⎞ ⎟ ⎠ [d N_{t}^{j i} - q_{j i}^{θ_{0}} (Y_{t}) ⟨ X_{t}, e_{i} ⟩ d t] \end{matrix}

and by (5),

d N_{t}^{j i} - q_{j i}^{θ_{0}} (Y_{t}) ⟨ X_{t}, e_{i} ⟩ d t = ⟨ X_{t -}, e_{i} ⟩ ⟨ e_{j}, d X_{t} - Q^{θ_{0}} (Y_{t}) X_{t} d t ⟩ .

(6)

Therefore, by Corollary 2.1, $exp {L^{j i}}$ and $exp {L^{0}}$ are orthogonal local martingales under $P^{θ_{0}}$ . The assumed boundedness further implies that they are martingales. This implies that $E_{t} := exp {L_{t} (θ, θ_{0})}$ is a martingale under $P^{θ_{0}}$ .

It only remains to show that $E U^{θ, f}$ is a martingale under $P^{θ_{0}}$ for any $f \in C_{b}^{0, 2}$ , where

U_{t}^{θ, f} = f (Z_{t}) - f (Z_{0}) - \int_{0}^{t} L^{θ} f (Z_{s}) d s .

By Itô’s formula,

E_{τ} U_{τ}^{θ, f} = \int_{0}^{τ} E_{t} d U_{t}^{θ, f} + \int_{0}^{τ} U_{t}^{θ, f} d E_{t} + \int_{0}^{τ} ⟨ μ^{θ} - μ^{θ_{0}}, \nabla_{y} f ⟩ (Z_{t}) d t + \sum 0 < t \leq τ Δ E_{t} Δ U_{t}^{θ, f}

and

Δ E_{t} = E_{t -} k \sum i, j = 1 ⎛ ⎜ ⎝ \frac{q_{j i}^{θ} (Y_{t})}{q_{j i}^{θ_{0}} (Y_{t})} - 1 ⎞ ⎟ ⎠ (N_{t}^{j i} - N_{t -}^{j i}), Δ U_{t}^{θ, f} = f (X_{t}, Y_{t}) - f (X_{t -}, Y_{t}) .

Since

Δ E_{t} Δ U_{t}^{θ, f} = E_{t -} k \sum i, j = 1 ⎛ ⎜ ⎝ \frac{q_{j i}^{θ} (Y_{t})}{q_{j i}^{θ_{0}} (Y_{t})} - 1 ⎞ ⎟ ⎠ (N_{t}^{j i} - N_{t -}^{j i}) (f (e_{j}, Y_{t}) - f (e_{i}, Y_{t})),

we have

\begin{matrix} \sum 0 < t \leq τ Δ E_{t} Δ U_{t}^{θ, f} = \int_{0}^{τ} E_{t -} k \sum i, j = 1 ⎛ ⎜ ⎝ \frac{q_{j i}^{θ} (Y_{t})}{q_{j i}^{θ_{0}} (Y_{t})} - 1 ⎞ ⎟ ⎠ (f (e_{j}, Y_{t}) - f (e_{i}, Y_{t})) d N_{t}^{j i} = \int_{0}^{τ} E_{t -} k \sum i, j = 1 ⎛ ⎜ ⎝ \frac{q_{j i}^{θ} (Y_{t})}{q_{j i}^{θ_{0}} (Y_{t})} - 1 ⎞ ⎟ ⎠ (f (e_{j}, Y_{t}) - f (e_{i}, Y_{t})) [d N_{t}^{j i} - q_{j i}^{θ_{0}} (Y_{t}) ⟨ X_{t}, e_{i} ⟩ d t] + \int_{0}^{τ} E_{t} k \sum i, j = 1 (q_{j i}^{θ} (Y_{t}) - q_{j i}^{θ_{0}} (Y_{t})) (f (e_{j}, Y_{t}) - f (e_{i}, Y_{t})) ⟨ X_{t}, e_{i} ⟩ d t . \end{matrix}

Consequently, we have

\begin{matrix} E_{τ} U_{τ}^{θ, f} = & \int_{0}^{τ} E_{t} d U_{t}^{θ_{0}, f} + \int_{0}^{τ} U_{t}^{θ, f} d E_{t} \int_{0}^{τ} E_{t -} k \sum i, j = 1 ⎛ ⎜ ⎝ \frac{q_{j i}^{θ} (Y_{t})}{q_{j i}^{θ_{0}} (Y_{t})} - 1 ⎞ ⎟ ⎠ (f (e_{j}, Y_{t}) - f (e_{i}, Y_{t})) [d N_{t}^{j i} - q_{j i}^{θ_{0}} (Y_{t}) ⟨ X_{t}, e_{i} ⟩ d t], \end{matrix}

which is a martingale under $P^{θ_{0}}$ by (6). ////

4 A finite-dimensional filter

Here we extend the filtering theory of hidden Markov models developed by Elliot et al. [1] to the SHS

\begin{matrix} d X_{t} = Q (Y_{t}) X_{t} d t + d V_{t}, d Y_{t} = μ (X_{t}, Y_{t}) d t + ϵ d W_{t}, \end{matrix}

where $V$ is a martingale (recall Corollary 2.1). In this section we assume we observe only a continuous sample path $Y$ on a time interval $[0, T]$ while $X$ is hidden. The system is a hidden Markov model in [1] when both $Q$ and $μ$ do not depend on $Y$ . By this dependence, $V$ is not independent of $W$ and so, the argument in [1] cannot apply here any more. We however show in this and the next sections that the results in [1] remain valid. Namely, a finite-dimensional filter and the EM algorithm can be constructed for the SHS. A key for this is Lemma 2.2.

Denote by $F^{Y}$ the natural filtration of $Y$ . The filtering problem is to infer $X$ from the observation of $Y$ , that is, to compute $E [X_{t} | F_{t}^{Y}]$ . The smoothing problem is to compute $E [X_{t} | F_{T}^{Y}]$ for $t \leq T$ . Denote $E_{t}^{0} [H] = E^{0} [H | F_{t}^{Y}]$ for a given integrable random variable $H$ , where $E^{0}$ is the expectation under $P^{0}$ in Section 2. For a given process $H$ , the Bayes formula gives

E [H_{t} | F_{t}^{Y}] = \frac{E_{t}^{0} [Λ_{t} H_{t}]}{E_{t}^{0} [Λ_{t}]},

(7)

where $Λ$ is defined by (4). Denoting $⟨ e_{i}, μ (e_{j}, y) ⟩ = c_{i j} (y)$ , $C (y) = [c_{i j} (y)]$ , we can write $μ (Z_{s}) = C (Y_{s}) X_{s}$ .

Theorem 4.1

Under the same conditions of Theorem 2.2, if $H$ is of the form

d H_{t} = α_{t} d t + ⟨ β_{t}, d X_{t} ⟩ + ⟨ δ_{t}, d Y_{t} ⟩,

(8)

where $α, β, δ$ are bounded predictable processes, then

\begin{matrix} E_{t}^{0} [Λ_{t} H_{t}] = & H_{0} + \frac{1}{ϵ^{2}} \int_{0}^{t} ⟨ C (Y_{s}) E_{s}^{0} [Λ_{s} H_{s} X_{s}] + ϵ^{2} E_{s}^{0} [Λ_{s} δ_{s}], d Y_{s} ⟩ + \int_{0}^{t} E_{s}^{0} [Λ_{s} α_{s}] + E_{s}^{0} [Λ_{s} ⟨ β_{s}, Q (Y_{s}) X_{s -} ⟩] + E_{s}^{0} [Λ_{s} ⟨ δ_{s}, C (Y_{s}) X_{s} ⟩] d s . \end{matrix}

(9)

Proof: Itô’s formula gives

Λ_{t} H_{t} = H_{0} + \int_{0}^{t} H_{s -} d Λ_{s} + \int_{0}^{t} Λ_{s} d H_{s} + \int_{0}^{t} Λ_{s} ⟨ δ_{s}, μ (Z_{s}) ⟩ d s .

Take the conditional expectation under $P^{0}$ given $F_{t}^{Y}$ to get (9). Here, we have used the fact that $Y / ϵ$ is a $d$ dimensional Brownian motion under $P^{0}$ as well as Lemma 2.2. ////

Theorem 4.2

Under the same conditions of Theorem 2.2, for each $i = 1, \dots, k$ ,

\begin{matrix} ⟨ e_{i}, E_{t}^{0} [Λ_{t} X_{t}] ⟩ = ⟨ e_{i}, X_{0} ⟩ + \frac{1}{ϵ^{2}} \int_{0}^{t} ⟨ e_{i}, E_{s}^{0} [Λ_{s} X_{s}] ⟩ ⟨ C (Y_{s}) e_{i}, d Y_{s} ⟩ + \int_{0}^{t} ⟨ e_{i}, Q (Y_{s}) E_{s}^{0} [Λ_{s} X_{s}] ⟩ d s, \end{matrix}

(10)

and

E_{t}^{0} [Λ_{t}] = 1 + \int_{0}^{t} ⟨ C (Y_{s}) E_{s}^{0} [Λ_{s} X_{s}], d Y_{s} ⟩ .

(11)

Proof: Let $H_{t} = ⟨ e_{i}, X_{t} ⟩$ and $H_{t} = 1$ in (9) to get (10) and (11) respectively. Here we have used that $⟨ e_{i}, X_{s} ⟩ C (Y_{s}) X_{s} = ⟨ e_{i}, X_{s} ⟩ C (Y_{s}) e_{i}$ . ////

Note that

X_{s} = k \sum i = 1 ⟨ e_{i}, X_{s} ⟩ e_{i}

and so, (10

) is a linear equation on the vector valued process

E_{t}^{0} [Λ_{t} X_{t}]

that is easy to solve. Then (11) is also solved, and

E [X_{t} | F_{t}^{Y}]

is obtained from (7).

Theorem 4.3

Under the same conditions of Theorem 2.2, for each $i = 1, \dots, k$ , for any $τ \leq t$ ,

⟨ e_{i}, E_{t}^{0} [Λ_{t} X_{τ}] ⟩ = ⟨ e_{i}, E_{τ}^{0} [Λ_{τ} X_{τ}] ⟩ + \frac{1}{ϵ^{2}} \int_{τ}^{t} ⟨ e_{i}, E_{s}^{0} [Λ_{s} X_{τ}] ⟩ ⟨ C (Y_{s}) e_{i}, d Y_{s} ⟩ .

Proof: Let $H_{t} = ⟨ e_{i}, X_{t \land τ} ⟩$ in (9). ////

This is also a linear equation and so, the smoothing problem $E [X_{τ} | F_{T}^{Y}]$ is easily solved via (7).

5 The EM algorithm

Here we consider again the parametric family ${P^{θ}}$ introduced in Section 3. We assume that a continuous sample path $Y$ is observed on a time interval $[0, T]$ while $X$ is hidden. We construct the EM algorithm to estimate $θ$ . Under the same assumptions as in Theorem 3.1, the law of $Y$ under $P^{θ}$ is equivalent to that under $P^{θ_{0}}$ and the log likelihood function is given by

L^{Y} (θ, θ_{0}) = log E^{θ_{0}} [\frac{d P^{θ}}{d P^{θ_{0}}} ∣ ∣ F_{T}^{Y}] .

The maximum likelihood estimator is therefore given by

^θ = {a r g m a x}_{θ \in Θ} L^{Y} (θ, θ_{0}) .

Note that $^θ$ does not depend on the choice of $θ_{0}$ because by the Bayes formula,

\begin{matrix} L^{Y} (θ, θ_{0}) & = log E^{θ_{1}} [\frac{d P^{θ_{0}}}{d P^{θ_{1}}} \frac{d P^{θ}}{d P^{θ_{0}}} ∣ ∣ F_{T}^{Y}] - log E^{θ_{1}} [\frac{d P^{θ_{0}}}{d P^{θ_{1}}} ∣ ∣ F_{T}^{Y}] = L^{Y} (θ, θ_{1}) - L^{Y} (θ_{0}, θ_{1}) \end{matrix}

(12)

for any $θ_{1} \in Θ$ . Now, we recall the idea of the EM algorithm. Let

Q (θ^{*}, θ) = E^{θ} [log \frac{d P^{θ^{*}}}{} d P^{θ} ∣ ∣ F_{T}^{Y}] .

By Jensen’s inequality and (12),

Q (θ^{*}, θ) \leq log E^{θ} [\frac{d P^{θ^{*}}}{d P^{θ}} ∣ ∣ F_{T}^{Y}] = L^{Y} (θ^{*}, θ) = L^{Y} (θ^{*}, θ_{0}) - L^{Y} (θ, θ_{0}),

which means that the sequence defined by

θ_{n + 1} = {a r g m a x}_{θ \in Θ} Q (θ, θ_{n})

makes $L^{Y} (θ_{n}, θ_{0})$ increasing. Under an appropriate condition the sequence ${θ_{n}}$ converges to the maximum likelihood estimator $^θ$ , for which we refer to Wu [8].

The computation of $Q (θ, θ_{0})$ is a filtering problem for which can apply the results in Section 3. Now we state the main result of this article.

Theorem 5.1

Let $Λ$ be defined by (4) with $μ = μ^{θ_{0}}$ . Under the condition of Theorem 3.1, we have

Q (θ, θ_{0}) = \frac{E_{T}^{0} [Λ_{T} L_{T} (θ, θ_{0})]}{E_{T}^{0} [Λ_{T}]},

\begin{matrix} E_{t}^{0} [Λ_{t} L_{t} (θ, θ_{0})] = & \frac{1}{ϵ^{2}} \int_{0}^{t} ⟨ C^{θ_{0}} (Y_{s}) E_{s}^{0} [Λ_{s} L_{s} (θ, θ_{0}) X_{s}] + C_{s} (θ, θ_{0}) E_{s}^{0} [Λ_{s} X_{s}], d Y_{s} ⟩ + \int_{0}^{t} (A \end{matrix}

EM algorithm for stochastic hybrid systems

The Expectation-Maximization Algorithm for Continuous-time Hidden Markov Models

A Continuous Time Framework for Discrete Denoising Models

Geometric fluid approximation for general continuous-time Markov chains

Decisiveness of Stochastic Systems and its Application to Hybrid Models

Decisiveness of Stochastic Systems and its Application to Hybrid Models (Full Version)

Stochastic Resonance for a Model with Two Pathways

Explicit approximations for nonlinear switching diffusion systems in finite and infinite horizons

1 Introduction

2 A construction as a weak solution

Theorem 2.1

Lemma 2.1

Lemma 2.2

Theorem 2.2

Corollary 2.1

3 The likelihood under complete observations

Theorem 3.1

4 A finite-dimensional filter

Theorem 4.1

Theorem 4.2

Theorem 4.3

5 The EM algorithm

Theorem 5.1

EM algorithm for stochastic hybrid systems

Related Research

The Expectation-Maximization Algorithm for Continuous-time Hidden Markov Models

A Continuous Time Framework for Discrete Denoising Models

Geometric fluid approximation for general continuous-time Markov chains

Decisiveness of Stochastic Systems and its Application to Hybrid Models

Decisiveness of Stochastic Systems and its Application to Hybrid Models (Full Version)

Stochastic Resonance for a Model with Two Pathways

Explicit approximations for nonlinear switching diffusion systems in finite and infinite horizons

1 Introduction

2 A construction as a weak solution

Theorem 2.1

Lemma 2.1

Lemma 2.2

Theorem 2.2

Corollary 2.1

3 The likelihood under complete observations

Theorem 3.1

4 A finite-dimensional filter

Theorem 4.1

Theorem 4.2

Theorem 4.3

5 The EM algorithm

Theorem 5.1