A semi-Lagrangian scheme for Hamilton-Jacobi-Bellman equations with oblique boundary conditions

1. Introduction

In this work we deal with the numerical approximation of the following parabolic Hamilton-Jacobi-Bellman (HJB) equation

(1.1)

In the system above, $T > 0$ , $O \subset R^{N}$ is a nonempty smooth bounded open set and $H$ and $L$ are nonlinear functions having the form

(1.2)		$H (t, x, p, M)$	$=$	$supa∈A{−12{\rm Tr}(σ(t,x,a)σ(t,x,a)⊤M)−⟨μ(t,x,a),p⟩−f(t,x,a)},$
(1.3)		$L (t, x, p)$	$=$	$sup b \in B {⟨ γ (x, b), p ⟩ - g (t, x, b)},$

where $A \subset R^{N_{A}}$ and $B \subset R^{N_{B}}$ are nonempty compact sets, $σ : [0, T] \times ¯ ¯¯ ¯ O \times A \to R^{N \times N_{σ}}$ , with $1 \leq N_{σ} \leq N$ , $μ : [0, T] \times ¯ ¯¯ ¯ O \times A \to R^{N}$ , $f : [0, T] \times ¯ ¯¯ ¯ O \times A \to R$ , $γ : \partial O \times V \to R^{N}$ , with $V \subseteq R^{N_{B}}$ being an open set containing $B$ , $g : [0, T] \times \partial O \times B \to R$ , and $Ψ : ¯ ¯¯ ¯ O \to R$ .

If $A = {a}$ and $B = {b}$ , for some $a \in R^{N_{A}}$ and $b \in R^{N_{B}}$ , and $γ (x, b) = n (x)$ , with $n (x)$

being the unit outward normal vector to

¯ ¯¯ ¯ O

x \in \partial O

, then (1.1) reduces to a standard linear parabolic equation with Neumann boundary conditions. In the general case, and after a simple change of the time variable in order to write (1.1) in backward form, the HJB equation (1.1) appears in the study of optimal control of diffusion processes with controlled reflection on the boundary

\partial O

(see e.g. [27] for the first order case, i.e.

σ \equiv 0

, and [26, 11] for the general case). Since the HJB equation (1.1) is possibly degenerate parabolic, one cannot expect the existence of classical solutions and we have to rely on the notion of viscosity solution (see e.g. [16]). Moreover, as it has been noticed in [25, 27], in general the boundary condition in (1.1) does not hold in the pointwise sense and we have to consider a suitable weak formulation of it. We refer the reader to [27, 4] and [16, 6, 7, 24, 12], respectively, for well-posedness results for HJB equations with oblique boundary condition in the first and second order cases.

The study of the numerical approximation of solutions to HJB and, more generally, fully nonlinear second order Partial Differential Equations (PDEs), has made important progress over the last few decades. Most of the related literature consider the case where

O = R^{N}

, or where a Dirichlet boundary condition is imposed on the boundary

\partial O

. We refer the reader to [19, 20, 30] and the references therein for the state of the art on this topic. By contrast, the numerical approximation of solutions to (1.1) has been much less explored. Indeed, to the best of our knowledge only the methods in [31, 1] can be applied to approximate (1.1) in the particular first order case (

σ \equiv 0

). Moreover, in [31], where a finite difference scheme is proposed, the function defining the boundary condition has the particular form

L (t, x, p, b) = ⟨ n (x), p ⟩

. On the other hand, both references consider Hamiltonians which are not necessarily convex with respect to

p

. Let us also mention the reference [2], where, in the context of mean curvature motion with nonlinear Neumann boundary conditions, the authors propose a discretization that combines a Semi-Lagrangian (SL) scheme in the main part of the domain with a finite difference scheme near the boundary.

The main purpose of this article is to provide a consistent, stable, monotone and convergent SL scheme to approximate the unique viscosity solution to (1.1). By the results in [6], the latter is well-posed in $C ([0, T] \times ¯ ¯¯ ¯ O)$ under the assumptions in Sect. 2 below. Semi-Lagrangian schemes to approximate the solution to (1.1) when $O = R^{N}$ (see e.g. [13, 17]) can be derived from the optimal control interpretation of (1.1) and a suitable discretization of the underlying controlled trajectories. These schemes enjoy the feature that they are explicit and stable under an inverse Courant-Friedrichs-Lewy (CFL) condition and, consequentely, they allow large time steps. A second important feature is that they permit a simple treatement of the possibly degenerate second order term in $H$ . The scheme that we propose for $O \neq R^{N}$ preserves these two properties and seems to be the first convergent scheme to approximate (1.1) with the rather general asumptions in Sect. 2. In particular, our results cover the stochastic and degenerate case. Consequently, from the stochastic control point of view, our scheme allows to approximate the so-called value function of the optimal control of a controlled diffusion process with possibly oblique reflection on the boundary $\partial O$ (see [11]). The main difficulty in devising such a scheme is to be able to obtain a consistency type property at points in the space grid which are near the boundary $\partial O$ while maintaining the stability. This is achieved by considering a discretization of the underlying controlled diffusion which suitably emulates its reflection at the boundary in the continuous case. We refer the reader to [28] for a related construction of a semi-discrete in time approximation of a second order non-degenerate linear parabolic equation.

The remainder of this paper is structured as follows. In Sect. 2 we state our assumptions, we recall the notion of viscosity solution to (1.1) and the well-posedness result. In Sect. 3 we provide the SL scheme as well as its probabilistic interpretation (in the spirit of [28]). The latter will play an important role in Sect. 4, which is devoted to show a consistency type property and the stability of the SL scheme. By using the half-relaxed limits technique introduced in [5], we show in Sect. 5 our main result, which is the convergence of solutions to the SL scheme towards the unique viscosity solution to (1.1). The convergence is uniform in $[0, T] \times ¯ ¯¯ ¯ O$ and holds under the same asymptotic condition between the space and time steps than in the case $O = R^{N}$ . Next, in Sect. 6 we first illustrate the numerical convergence of the SL scheme in the case of a one-dimensional linear equation with homogeneous Neumann boundary conditions. In this case the numerical results confirm that the boundary condition in (1.1) is not satisfied at every $x \in \partial O$ , but it is satisfied in the viscosity sense recalled in Sect. 2 below. In a second example, we consider a two dimensional degenerate second order nonlinear equation on a circular domain with non-homogeneous Neumann and oblique boundary conditions. In the last example, we consider a two-dimensional non-degenerate nonlinear equation on a non-smooth domain. Due to the lack of regularity of $\partial O$ , our convergence result does not apply. However, the SL scheme can be successfully applied, which suggests that our theoretical findings could hold for more general domains. This extension as well as the corresponding study in the stationary framework remain as interesting subjects of future research. Finally, we provide in the Appendix of this work some theoretical results concerning oblique projections and the regularity of the distance to $\partial O$ , which play a key role in the definition of the scheme and in the proof of its main properties.

Acknowledgements

The first two authors would like to thank the Italian Ministry of Instruction, University and Research (MIUR) for supporting this research with funds coming from the PRIN Project $2017$ ( $2017$ KKJP $4$ X entitled “Innovative numerical methods for evolutionary partial differential equations and applications”). Xavier Dupuis thanks the support by the EIPHI Graduate School (contract ANR-17-EURE-0002). Elisa Calzola, Elisabetta Carlini and Francisco J. Silva were partially supported by KAUST through the subaward agreement OSR- $2017$ -CRG $6$ - $3452$ . $04$ .

2. Preliminaries

As mentioned in the introduction, it will be simpler to describe our approximation scheme when (1.1) is written in backward form. This can be done by a simple change of the time variable and a possible modification of the time dependency of $H$ . Let us set $O_{T} := [0, T) \times O$ and ${¯ ¯¯ ¯ O}_{T} = [0, T] \times ¯ ¯¯ ¯ O$ . We consider the HJB equation

(HJB)

where $H$ and $L$ are respectively given by (1.2) and (1.3).

For notational convenience, throughout this article, we will write $γ_{b} (x) = γ (x, b)$ for all $x \in \partial O$ and $b \in B$ . Our standing assumptions for the data in (HJB) are the following.

$O \subseteq R^{N}$ ( $1 \leq N \leq 3$ ) is a nonemtpy, bounded domain with boundary $\partial O$ of class $C^{3}$ .
The functions $σ$ , $μ$ , $f$ , $g$ and $Ψ$ are continuous. Moreover, for every $a \in A$ , the functions $σ (\cdot, \cdot, a)$ and $μ (\cdot, \cdot, a)$ are Lipschitz continuous, with Lipschitz constants independent of $a \in A$ .
The function $γ$ is of class $C^{1}$ . We also assume that

$(\forall (x, b) \in \partial O \times B) | γ_{b} (x) | = 1 % a n d ⟨ n (x), γ_{b} (x) ⟩ > 0,$

where, for every $x \in \partial O$ , we recall that $n (x)$ denotes the unit outward normal vector to $¯ ¯¯ ¯ O$ at $x$ .

We now recall the notion of viscosity solution to $(H J B)$ (see [6]). We need first to introduce some notation. Given a bounded function $z : {¯ ¯¯ ¯ O}_{T} \to R$ , its upper semicontinuous (resp. lower semicontinuous) envelope is defined by

(2.1)

(\forall (t, x) \in {¯ ¯¯ ¯ O}_{T}) z^{*} (t, x) := limsup \begin{matrix} (s, y) \in {¯ ¯¯ ¯ O}_{T}, (s, y) \to (t, x) \end{matrix} z (s, y) ⎛ ⎜ ⎜ ⎝ resp. % z_{*} (t, x) := liminf \begin{matrix} (s, y) \in {¯ ¯¯ ¯ O}_{T}, (s, y) \to (t, x) \end{matrix} z (s, y) ⎞ ⎟ ⎟ ⎠ .

Definition 2.1.

[Viscosity solution]

(i) An upper semicontinuous function $u_{1} : {¯ ¯¯ ¯ O}_{T} \to R$ is a viscosity subsolution to (HJB) if for any $(t, x) \in {¯ ¯¯ ¯ O}_{T}$ and $ϕ \in C^{2} ({¯ ¯¯ ¯ O}_{T})$ such that $u_{1} - ϕ$ has a local maximum at $(t, x)$ , we have

(2.2)

- \partial_{t} ϕ (t, x) + H (t, x, D ϕ (t, x), D^{2} ϕ (t, x)) \leq 0,

if $(t, x) \in O_{T}$ ,

(2.3)

min {- \partial_{t} ϕ (t, x) + H (t, x, D ϕ (t, x), D^{2} ϕ (t, x)), L (t, x, D ϕ (t, x))} \leq 0,

if $(t, x) \in [0, T) \times \partial O$ and,

(2.4)

u_{1} (t, x) \leq Ψ (x),

if $(t, x) \in {T} \times ¯ ¯¯ ¯ O$ .

(ii) A lower semicontinuous function $u_{2} : {¯ ¯¯ ¯ O}_{T} \to R$ is a viscosity supersolution to (HJB) if for any $(t, x) \in {¯ ¯¯ ¯ O}_{T}$ and $ϕ \in C^{2} ({¯ ¯¯ ¯ O}_{T})$ such that $u_{2} - ϕ$ has a local minimum at $(t, x)$ , we have

(2.5)

- \partial_{t} ϕ (t, x) + H (t, x, D ϕ (t, x), D^{2} ϕ (t, x)) \geq 0,

if $(t, x) \in O_{T}$ ,

(2.6)

max {- \partial_{t} ϕ (t, x) + H (t, x, D ϕ (t, x), D^{2} ϕ (t, x)), L (t, x, D ϕ (t, x))} \geq 0,

if $(t, x) \in [0, T) \times \partial O$ and,

(2.7)

u_{2} (t, x) \geq Ψ (x),

if $(t, x) \in {T} \times ¯ ¯¯ ¯ O$ .

(iii) A bounded function $u : {¯ ¯¯ ¯ O}_{T} \to R$ is a viscosity solution to (HJB) if $u^{*}$ and $u_{*}$ , defined in (2.1), are, respectively, sub- and supersolutions to (HJB).

Remark 2.1.

As shown in [12, Proposition 6], relation (2.4) can be replaced by

(2.8)

min {- \partial_{t} ϕ (t, x) + H (t, x, D ϕ (t, x), D^{2} ϕ (t, x)), u_{1} (t, x) - Ψ (x)} \leq 0,

if $(t, x) \in {T} \times O$ , and

(2.9)

min {- \partial_{t} ϕ (t, x) + H (t, x, D ϕ (t, x), D^{2} ϕ (t, x)), L (t, x, D ϕ (t, x)), u_{1} (t, x) - Ψ (x)} \leq 0,

if $(t, x) \in {T} \times \partial O$ . Similarly, condition (2.7) can be replaced by

(2.10)

max {- \partial_{t} ϕ (t, x) + H (t, x, D ϕ (t, x), D^{2} ϕ (t, x)), u_{2} (t, x) - Ψ (x)} \geq 0,

if $(t, x) \in {T} \times O$ , and

(2.11)

max {- \partial_{t} ϕ (t, x) + H (t, x, D ϕ (t, x), D^{2} ϕ (t, x)), L (t, x, D ϕ (t, x)), u_{2} (t, x) - Ψ (x)} \geq 0,

if $(t, x) \in {T} \times \partial O$ .

The following well-posedness result for $(H J B)$ has been shown in [6, Theorem II.1] (see also [11]).

Theorem 2.1.

Assume (H1)-(H3). Then there exists a unique viscosity solution $u \in C (¯ ¯¯ ¯ O)$ to (HJB).

Remark 2.2.

(i) [Comparison principle and uniqueness] The existence of at most one solution to (HJB) follows from the following comparison principle (see [6, Theorem II.1] and also [11, Proposition 3.4]). If $u_{1} : {¯ ¯¯ ¯ O}_{T} \to R$ is a bounded viscosity subsolution to (HJB) and $u_{2} : {¯ ¯¯ ¯ O}_{T} \to R$ is a bounded viscosity supersolution to (HJB), then

u_{1} \leq u_{2} in {¯ ¯¯ ¯ O}_{T} .

(ii) [Existence] Once a comparison principle has been shown, the existence of a solution to (HJB) follows usually from the existence of sub- and supersolutions to (HJB) and Perron’s method. In Sect. 5, we construct sub- and supersolutions to (HJB) as suitable limits of solutions to the approximation scheme that we present in the next section. Together with the comparison principle, this yields an alternative existence proof of solutions to (HJB).

An different and interesting technique to show the existence of a solution to (HJB) is to consider a suitable stochastic optimal control problem, with controlled reflection of the state trajectory at the boundary $\partial O$ , and to show that the associated value function is a viscosity solution to (HJB). This strategy has been followed in [11].
(iii) [Continuity] The continuity of the unique viscosity solution to (HJB) follows directly from the comparison principle and the continuity properties required in the definition of sub- and supersolutions to (HJB). Notice that, as usual for parabolic problems with Neumann type boundary conditions, we do not require any compatibility condition between $Ψ$ and the operator $L$ at the boundary $\partial O$ .

3. The fully discrete scheme

We introduce in this section a fully discrete SL scheme that approximates the unique viscosity solution to $(H J B)$ . Throughout this section, we assume that (H1)-(H3) are fulfilled.

3.1. Discretization of the space domain $O$

Let us fix $Δ x > 0$ and consider a polyhedral domain $O_{Δ x} \subseteq R^{N}$ such that

(3.1)

d (O, O_{Δ x}) = inf {| x - y | | x \in O, y \in O_{Δ x}} \leq C {(Δ x)}^{2},

for some $C > 0$ . A construction of such a domain $O_{Δ x}$ can be found in [8, Section 3] for $N = 2$ or $N = 3$ , which explain the dimension constraint in (H1). However, the results in the remainder of this article can be extended to $N > 3$ , provided that a numerical domain $O_{Δ x}$ satisfying (3.1) exists. Let $T_{Δ x}$ be a triangulation of $O_{Δ x}$ consisting of simplicial finite elements $T$ with vertices in $G_{Δ} = {x_{i} | i = 1, \dots, N_{Δ x}}$ (for some $N_{Δ x} \in N$ ). We assume that $Δ x$ is the mesh size, i.e. the maximum of the diameters of $T \in T_{Δ x}$ , all the vertices on $\partial O_{Δ x}$ belong to $\partial O$ , at most one face of each element $T \in T_{Δ x}$ , with vertices on $\partial O_{Δ x}$ , intersects $\partial O_{Δ x}$ , and $T_{Δ x}$ satisfies the following regularity condition: there exists $δ \in (0, 1)$ , independent of $Δ x$ , such that each $T \in T_{Δ x}$ is contained in a ball of radius $Δ x / δ$ and contains a ball of radius $δ Δ x$ . As in [18], we introduce an auxiliary exact triangulation ${ˆ T}_{Δ x}$ of $¯ ¯¯ ¯ O$ with vertices in $G_{Δ x}$ . The boundary elements of ${ˆ T}_{Δ x}$ are allowed to be curved and we have

¯ ¯¯ ¯ O = ⋃ ˆ T \in T_{Δ x} ˆ T .

Denoting by $p_{T}$ the projection on $T \in T_{Δ x}$ , the projection $p_{Δ x} : ¯ ¯¯ ¯ O \to {¯ ¯¯ ¯ O}_{Δ x} \cap ¯ ¯¯ ¯ O$ is defined by

	$p_{Δ x} (x) = p_{T} (x),$	if $x \in ˆ T \in {ˆ T}_{Δ x}$
		and the element $T \in T_{Δ x}$ has the same vertices than $ˆ T$ .

Set $I_{Δ x} = {1, \dots, N_{Δ x}}$ and denote by ${ψ_{i} | i \in I_{Δ x}}$ the linear finite element $P_{1}$ basis function on $T_{Δ x}$ . More precisely, for each $i \in I_{Δ x}$ , $ψ_{i} : O_{Δ x} \to R$ is a continuous function, affine on each $T \in T_{Δ x}$ , $0 \leq ψ_{i} \leq 1$ , $ψ_{i} (x_{i}) = 1$ , $ψ_{i} (x_{j}) = 0$ for all $i$ , $j \in I_{Δ x}$ with $i \neq j$ , and $\sum_{i = 1}^{N_{Δ x}} ψ_{i} (x) = 1$ for all $x \in O_{Δ x}$ . For any $ϕ : G_{Δ x} \to R$

its linear interpolation

I [ϕ]

on the mesh

{ˆ T}_{Δ x}

is defined by

(3.2)

I [ϕ] (x) := N_{Δ x} \sum i = 1 ψ_{i} (p_{Δ x} (x)) ϕ (x_{i}), for all x \in ¯ ¯¯ ¯ O .

Lemma 3.1.

Let $ϕ \in C^{2} (¯ ¯¯ ¯ O)$ and denote by $ϕ |_{G_{Δ x}}$ its restriction to $G_{Δ x}$ . Then there exists a constant $C_{ϕ} > 0$ , independent of $Δ x$ , such that

(3.3)

sup x \in ¯ ¯¯ ¯ O ∣ ∣ ϕ (x) - I [ϕ |_{G_{Δ x}}] (x) ∣ ∣ \leq C_{ϕ} (Δ x)^{2} .

Proof.

Let $x \in ¯ ¯¯ ¯ O$ and let $T \in T_{Δ x}$ and $ˆ T \in {ˆ T}_{Δ x}$ be two elements having the same vertices and such that $x \in ˆ T$ . By the triangular inequality

| ϕ (x) - I [ϕ |_{G_{Δ x}}] (x) | \leq | ϕ (x) - ϕ (p_{T} (x)) | + | ϕ (p_{T} (x)) - I [ϕ |_{G_{Δ x}}] (x) | .

Using that $ϕ$ is Lipschitz, we deduce from (3.1) the existence of $C_{1} > 0$ , independent of $Δ x$ and $x \in ¯ ¯¯ ¯ O$ , such that $| ϕ (x) - ϕ (p_{T} (x)) | \leq C_{1} (Δ x)^{2}$

. In addition, by standard error estimates for

P_{1}

interpolation (see for instance [15]) and (3.2), there exists

C_{2} > 0

, independent of

Δ x

and

x \in ¯ ¯¯ ¯ O

, such that

| ϕ (p_{T} (x)) - I [ϕ |_{G_{Δ x}}] (x) | \leq C_{2} (Δ x)^{2}

. Relation (3.3) follows from these two estimates. ∎

Figure 1. Reflection: reflected characteristic ${~ y}_{k, i}^{s} (a)$ (red square) starting from $x_{i}$ (black circle), which exits from $O$ and arrives in $y_{k, i}^{s} (a)$ (black square). The red segment represents the oblique direction $γ_{b}$ and the black circle the projected point $p^{γ_{b}} (y_{k, i}^{s} (a)$ ).

3.2. A semi-Lagrangian scheme

Let $Δ t > 0$ , set $N_{Δ t} := ⌊ T / Δ t ⌋$ , $I_{Δ t} := {0, \dots, N_{Δ t}}$ and $I_{Δ t}^{*} := I_{Δ t} ∖ {N_{T}}$ . We define the time grid $G_{Δ t} := {t_{k} | t_{k} = k Δ t, k \in I_{Δ t}}$ . Given $(k, i) \in I_{Δ t}^{*} \times I_{Δ x}$ , $a \in A$ , and $ℓ = 1, \dots, N_{σ}$ , we define the discrete characteristics

(3.4)

y_{k, i}^{\pm, ℓ} (a) = x_{i} + Δ t μ (t_{k}, x_{i}, a) \pm \sqrt{N_{σ} Δ t} σ^{ℓ} (t_{k}, x_{i}, a) .

Let $I = {+, -} \times {1, \dots, N_{σ}}$ and let $¯ c > 0$ be a fixed constant. For any $δ > 0$ we set

(\partial O)_{δ} := {x \in R^{N} | d (x, \partial O) < δ} .

By Proposition 7.1 in the Appendix, there exist $R > 0$ and two $C^{1}$ functions $(\partial O)_{R} \times B ∋ (x, b) \mapsto p^{γ_{b}} (x) \in \partial O$ and $(\partial O)_{R} \times B ∋ (x, b) \mapsto d^{γ_{b}} (x) \in R$ , uniquely determined, such that

(3.5)

x = p^{γ_{b}} (x) + d^{γ_{b}} (x) γ_{b} (p^{γ_{b}} (x)), % for all (x, b) \in (\partial O)_{R} \times B .

Therefore, there exists $¯ ¯¯¯¯¯ ¯ Δ t > 0$ such that for all $Δ t \in [0, ¯ ¯¯¯¯¯ ¯ Δ t]$ , $(k, i) \in I_{Δ t}^{*} \times I_{Δ x}$ , $a \in A$ , $b \in B$ , and $s \in I$ , the reflected characteristic

(3.6)

{~ y}_{k, i}^{s} (a, b) := {\begin{matrix} y_{k, i}^{s} (a) & if y_{k, i}^{s} (a) \in ¯ ¯¯ ¯ O, p^{γ_{b}} (y_{k, i}^{s} (a)) - ¯ c \sqrt{Δ t} γ_{b} (p^{γ_{b}} (y_{k, i}^{s} (a))) & otherwise \end{matrix}

is well-defined. In Figure 1 we illustrate how the reflected characteristic is computed from the projection $p^{γ_{b}} (y_{k, i}^{s} (a))$ of $y_{k, i}^{s} (a)$ onto $\partial O$ parallel to $γ_{b}$ . Let us also set

(3.7)		${~ d}_{k, i}^{s} (a, b)$	$:= {\begin{matrix} 0 & if y_{i, k}^{s} (a) \in ¯ ¯¯ ¯ O, d^{γ_{b}} (y_{k, i}^{s} (a)) + ¯ c \sqrt{Δ t} & otherwise, \end{matrix}$
(3.8)		${~ g}_{k, i}^{s} (a, b)$	$:= ⎧ ⎨ ⎩ \begin{matrix} 0 & if y_{k, i}^{s} (a) \in ¯ ¯¯ ¯ O, g (t_{k}, p^{γ_{b}} (y_{k, i}^{s} (a)), b) & % otherwise . \end{matrix}$

Notice that if $y_{k, i}^{s} (a) \notin ¯ ¯¯ ¯ O$ , then (3.5), (3.6), and (3.7) imply that

(3.9)

{~ y}_{k, i}^{s} (a, b) = y_{k, i}^{s} (a) - {~ d}_{k, i}^{s} (a, b) γ_{b} (p^{γ_{b}} (y_{k, i}^{s} (a))) .

For $(k, i) \in I_{Δ t}^{*} \times I_{Δ x}$ and $Φ : G_{Δ x} \to R$ , let us define $S_{k, i} [Φ] : A \times B \to R$ by

(3.10)

S_{k, i} [Φ] (a, b) := \frac{1}{2 N_{σ}} \sum s \in I [I [Φ] ({~ y}_{k, i}^{s} (a, b)) + {~ d}_{k, i}^{s} (a, b) {~ g}_{k, i}^{s} (a, b)] + Δ t f (t_{k}, x_{i}, a)

and set

(3.11)

S_{k, i} [Φ] := inf \begin{matrix} a \in A, b \in B \end{matrix} S_{k, i} [Φ] (a, b) .

In the remainder of this work, we will consider the following fully discrete SL scheme to approximate the solution to $(H J B)$ .

(HJB

{\rm\tiny disc}

)

\begin{matrix} U_{k, i} & = & S_{k, i} [U_{k + 1, (\cdot)}], % for (k, i) \in I_{Δ t}^{*} \times I_{Δ x}, U_{N_{Δ t}, i} & = & Ψ (x_{i}), for i \in I_{Δ x} . \end{matrix}

3.3. Probabilistic interpretation of the scheme

The fully-discrete SL to approximate the solution to (HJB) in the unbounded case, i.e. $O = R^{d}$ , has a natural interpretation in terms of a discrete time, finite state, Markov control process (see e.g. [13, Section 3]). We show below that a similar interpretation holds for (HJB ${\rm\tiny disc}$ ). The latter will play an important role in the stability analysis of (HJB ${\rm\tiny disc}$ ) presented in the next section. Given $k \in I_{Δ t}^{*}$ and $a \in A$ , $b \in B$ , let us define the controlled transition law

(3.12)

p_{k, i, j} (a, b) := \frac{1}{2 N_{σ}} \sum s \in I β_{j} ({~ y}_{k, i}^{s} (a, b)), for all i, j \in I_{Δ x} .

We say that $(π_{k})_{k \in I_{Δ t}^{*}}$ is a $N_{Δ t}$ -policy if for all $k \in I_{Δ t}^{*}$ we have $π_{k} : G_{Δ x} \to A \times B$ . The set of $N_{Δ t}$ -policies is denoted by $Π_{N_{Δ t}}$ . Let us fix $k \in I_{Δ t}^{*}$ and, for notational convenience, set $X_{k} = G_{Δ x}^{N_{Δ t} - k + 1}$ . Associated to $x_{i} \in G_{Δ x}$ and $π \in Π_{N_{Δ t}}$

, there exists a probability measure

P^{k, x_{i}, π}

2^{X_{k}}

(the powerset of

X_{k}

) and a Markov chain

{X_{m} | m = k, \dots, N_{Δ t}}

, with state space

G_{Δ x}

, such that

(3.13)

P^{k, x_{i}, π} (X_{k} = x_{i}) = 1 and P^{k, x_{i}, π} (X_{m + 1} = x_{j} | X_{m} = x_{i}) = p_{m, i, j} (π_{m} (x_{i})),

for $m = k, \dots, N_{Δ t} - 1$ . Now, consider a family ${ξ_{k + 1}, \dots, ξ_{N_{Δ t}}}$ of $R^{N_{σ}}$

-valued independent random variables, which are also independent of

{X_{m} | m = k, \dots, N_{Δ t}}

, and with common distribution given by

P (ξ_{m} = \pm e_{ℓ}) = \frac{1}{2 N_{σ}}, for m = k + 1, \dots, N_{Δ t} and ℓ = 1, \dots, N_{σ},

where $e_{ℓ}$ denotes the $ℓ$ -th canonical vector of $R^{N_{σ}}$ . By a slight abuse of notation (see (3.4)), for $m = k, \dots, N_{Δ t} - 1$ , $x_{i} \in G_{Δ x}$ , and $a \in A$ , let us set

(3.14)

y_{m} (x_{i}, a) = x_{i} + Δ t μ (t_{m}, x_{i}, a) + \sqrt{N_{σ} Δ t} σ (t_{m}, x_{i}, a) ξ_{m + 1} .

For $m = k, \dots, N_{Δ t} - 1$ , $x_{i} \in G_{Δ x}$ , $a \in A$ , and $b \in B$ , define the random variable

(3.15)

h (t_{m}, x_{i}, a, b) = ⎧ ⎨ ⎩ \begin{matrix} 0 & if y_{m} (x_{i}, a) \in ¯ ¯¯ ¯ O, (d^{γ_{b}} (y_{m} (x_{i}, a)) + ¯ c \sqrt{Δ t}) g (t_{m}, p^{γ_{b}} (y_{m} (x_{i}, a)), b) & otherwise. \end{matrix}

For all $i \in I_{N_{Δ x}}$ and $π \in Π_{N_{Δ t}}$ , let us define

\begin{matrix} J_{k, i} (π) & = & E_{P^{k, x_{i}, π}} (\sum_{m = k}^{N_{Δ t} - 1} [Δ t f (t_{m}, X_{m}, α_{m}) + h (t_{m}, X_{m}, α_{m}, β_{m})] + Ψ (X_{N_{Δ t}})), J_{N_{Δ t}, i} (π) & = & Ψ (x_{i}), \end{matrix}

where, for notational convenience, we have denoted, respectively, by $α_{m}$ and $β_{m}$ the first $N_{A}$ and the last $N_{B}$ coordinates of $π_{m} (X_{m})$ . Notice that, by construction and (3.10), we have that

Jk,i(π)=\SSk,i[Jk+1,(⋅)(π)](αk,βk).

Moreover, setting

\begin{matrix} {^U}_{k, i} & = & {inf}_{π \in Π_{N_{Δ t}}} J_{k, i} (π), {^U}_{N_{Δ t}, i} & = & Ψ (x_{i}), \end{matrix}

for all $i \in G_{Δ x}$ , the dynamic programming principle (see e.g. [23, Theorem 12.1.5]) implies that ${^Uk,i|k∈IΔt,i∈IΔx}$ satisfies (HJB ${\rm\tiny disc}$ ). Since the latter has a unique solution, we deduce that $U_{k, i} = {^U}_{k, i}$ for all $k \in I_{Δ t}$ and $i \in I_{Δ x}$ .

Remark 3.1.

Scheme (HJB ${\rm\tiny disc}$ ) can thus be interpreted as a Markov chain discretization of an stochastic control problem with oblique reflection in the boundary (see e.g. [11]).

4. Properties of the fully discrete scheme

In this section, we establish some basic properties of (HJB ${\rm\tiny disc}$ ).

Proposition 4.1.

The following hold:
(i) (Monotonicity) For all $U, V : G_{Δ x} \to R$ with $U \leq V$ , we have

S_{k, i} [U] \leq S_{k, i} [V], for k \in I_{Δ t}^{*} and i \in I_{Δ x} .

(ii) (Commutation by constant) For any $c \in R$ and $U : G_{Δ x} \to R$ ,

S_{k, i} [U + c] = S_{k, i} [U] + c, for k \in I_{Δ t}^{*} and i \in I_{Δ x} .

Proof.

Both assertions follow directly from (3.10) and (HJB ${\rm\tiny disc}$ ). ∎

We show in Proposition 4.2 below a consistency result for (HJB ${\rm\tiny disc}$ ). For this purpose, let us set

$H (t, x, p, M, a)$	$=$	$−12{\rm Tr}(σ(t,x,a)σ(t,x,a)⊤M)−⟨μ(t,x,a),p⟩−f(t,x,a),$
		$for (t, x, p, M, a) \in {¯ ¯¯ ¯ O}_{T} \times R^{N} \times R^{N \times N_{σ}} \times A,$
$L (t, x, p, b)$	$=$	$⟨ γ (x, b), p ⟩ - g (t, x, b),$
		$for (t, x, p, b) \in [0, T] \times \partial O \times R^{N} \times B,$

and for all $k \in I_{Δ t}^{*}$ , $i \in I_{Δ x}$ , $s \in I$ , $q \in R^{N}$ , $a \in A$ , and $b \in B$ , define

(4.3)

{~ L}_{k, i}^{s} (q, a, b) := ⎧ ⎪ ⎨ ⎪ ⎩ \begin{matrix} 0 & if y_{k, i}^{s} (a) \in ¯ ¯¯ ¯ O, L (t_{k}, p^{γ_{b}} (y_{k, i}^{s} (a)), q, b) & % otherwise. \end{matrix}

Proposition 4.2 (Consistency).

Let $ϕ \in C^{3} (¯ ¯¯ ¯ O)$ and denote by $ϕ |_{G_{Δ x}}$ its restriction to $G_{Δ x}$ . Then the following hold:

For all $k \in I_{Δ t}^{*}$ , $i \in I_{Δ x}$ , $a \in A$ , and $b \in B$ , we have

$\begin{matrix} S_{k, i} [ϕ |_{G_{Δ x}}] (a, b) - ϕ (x_{i}) = & - Δ t H (t_{k}, x_{i}, D ϕ (x_{i}), D^{2} ϕ (x_{i}), a) - \frac{1}{2 N_{σ}} \sum s \in I {~ d}_{k, i}^{s} (a, b) ({~ L}_{k, i}^{s} (D ϕ (x_{i}), a, b) - \sqrt{Δ t} K_{k, i}^{s} (a, b)) + O (Δ t \sqrt{Δ t} + (Δ x)^{2}), \end{matrix}$

where the set of constants ${K_{k, i}^{s} (a, b) | k \in I_{Δ t}^{*}, i \in I_{Δ x}, s \in I, a \in A, b \in B}$ is bounded, independently of $(Δ t, Δ x)$ .
For all $k \in I_{Δ t}^{*}$ and $i \in I_{Δ x}$ , we have

$\begin{matrix} S_{k, i} [ϕ |_{G_{Δ x}}] - ϕ (x_{i}) = & - sup \begin{matrix} a \in A, b \in B \end{matrix} {Δ t H (t_{k}, x_{i}, D ϕ (x_{i}), D^{2} ϕ (x_{i}), a) + \frac{1}{2 N_{σ}} \sum s \in I {~ d}_{k, i}^{s} (a, b) ({~ L}_{k, i}^{s} (D ϕ (x_{i}), a, b) - \sqrt{Δ t} K_{k, i}^{s} (a, b))} + O (Δ t \sqrt{Δ t} + (Δ x)^{2}) . \end{matrix}$

Proof.

In what follows, we denote by $C > 0$ a generic constant, which is independent of $k$ , $i$ , $s$ $a$ , $b$ , $Δ t$ and $Δ x$ . Since assertion (ii) follows directly from $(i)$ , we only show the latter.

For every $s \in I$ , (3.4) and (3.7) imply that $0 \leq {~ d}_{k, i}^{s} (a, b) \leq C \sqrt{Δ t}$ . Thus, by (3.4), (3.9), and a second order Taylor expansion of $ϕ$ around $x_{i}$ , for every $ℓ = 1, \dots, N_{σ}$ , we have

\begin{matrix} ϕ ({~ y}_{k, i}^{\pm, ℓ} (a, b)) = ϕ (x_{i}) + Δ t ⟨ D ϕ (x_{i}), μ (t_{k}, x_{i}, a) ⟩ + \frac{N_{σ} Δ t}{2} ⟨ D^{2} \end{matrix}

A semi-Lagrangian scheme for Hamilton-Jacobi-Bellman equations with oblique boundary conditions

Authors

A fully semi-Lagrangian discretization for the 2D Navier--Stokes equations in the vorticity--streamfunction formulation

A Hybrid Semi-Lagrangian Cut Cell Method for Advection-Diffusion Problems with Robin Boundary Conditions in Moving Domains

Convergence of a Lagrangian-Eulerian scheme via the weak asymptotic method

The waiting time phenomenon in spatially discretized porous medium and thin film equations

A convergent SAV scheme for Cahn–Hilliard equations with dynamic boundary conditions

Stable Boundary Conditions and Discretization for PN Equations

How to train your solver: A method of manufactured solutions for weakly-compressible SPH

1. Introduction

Acknowledgements

2. Preliminaries

Definition 2.1.

Remark 2.1.

Theorem 2.1.

Remark 2.2.

3. The fully discrete scheme

3.1. Discretization of the space domain $O$

Lemma 3.1.

Proof.

3.2. A semi-Lagrangian scheme

3.3. Probabilistic interpretation of the scheme

Remark 3.1.

4. Properties of the fully discrete scheme

Proposition 4.1.

Proof.

Proposition 4.2 (Consistency).

Proof.

A semi-Lagrangian scheme for Hamilton-Jacobi-Bellman equations with oblique boundary conditions

Authors

Related Research

A fully semi-Lagrangian discretization for the 2D Navier--Stokes equations in the vorticity--streamfunction formulation

A Hybrid Semi-Lagrangian Cut Cell Method for Advection-Diffusion Problems with Robin Boundary Conditions in Moving Domains

Convergence of a Lagrangian-Eulerian scheme via the weak asymptotic method

The waiting time phenomenon in spatially discretized porous medium and thin film equations

A convergent SAV scheme for Cahn–Hilliard equations with dynamic boundary conditions

Stable Boundary Conditions and Discretization for PN Equations

How to train your solver: A method of manufactured solutions for weakly-compressible SPH

This week in AI

1. Introduction

Acknowledgements

2. Preliminaries

Definition 2.1.

Remark 2.1.

Theorem 2.1.

Remark 2.2.

3. The fully discrete scheme

3.1. Discretization of the space domain O

Lemma 3.1.

Proof.

3.2. A semi-Lagrangian scheme

3.3. Probabilistic interpretation of the scheme

Remark 3.1.

4. Properties of the fully discrete scheme

Proposition 4.1.

Proof.

Proposition 4.2 (Consistency).

Proof.

3.1. Discretization of the space domain $O$