Convergence results and low order rates for nonlinear Tikhonov regularization with oversmoothing penalty term

1 Introduction

The subject of this paper are nonlinear operator equations of the form

F u = f^{†},

(1.1)

where $F : X \supset D (F) \to Y$ is a nonlinear operator between infinite-dimensional Hilbert spaces $X$ and $Y$ with norms $∥ \cdot ∥$ . We suppose that the right-hand side $f^{†} \in Y$ is approximately given as $f^{δ} \in Y$ satisfying the deterministic noise model

∥ f^{δ} - f^{†} ∥ \leq δ,

(1.2)

with the noise level $δ \geq 0$ . Throughout the paper, it is assumed that the considered equation (1.1) has a solution $u^{†} \in D (F)$ and is (at least locally at $u^{†}$ ) ill-posed (cf. [12]).

For finding stable approximations to the solution $u^{†} \in D (F)$ of equation (1.1), we consider the Tikhonov regularization, where the regularized solutions are minimizers of the extremal problem

T_{α}^{δ} (u) := ∥ F u - f^{δ} ∥^{2} + α ∥ u - ¯ ¯ ¯ u ∥_{1}^{2} \to min subject to u \in D (F),

(1.3)

with a regularization parameter $α > 0$ . In this context, $∥ \cdot ∥_{1}$ is assumed to be a norm of a densely defined subspace $X_{1}$ of $X$ , which is stronger than the original norm $∥ \cdot ∥$ in $X$ . Throughout this paper we suppose that the initial guess $¯ ¯ ¯ u$ occurring in the penalty term of $T_{α}^{δ} (u)$ satisfies the condition

¯ ¯ ¯ u \in D := D (F) \cap X_{1} .

(1.4)

Precisely, we define the stronger norm $∥ \cdot ∥_{1}$ by a generator $B : X \supset D (B) \to X$ , which is a self-adjoint and positive definite unbounded linear operator with dense domain $D (B)$ , i.e. we have for some constant $m > 0$

∥ B u ∥ \geq m ∥ u ∥ for u \in D (B) .

(1.5)

This allows us to introduce norms

∥ u ∥_{τ} := ∥ B^{τ} u ∥, u \in X_{τ} (τ \in R),

(1.6)

where $X_{τ} := D (B^{τ})$ for $τ > 0$ , and $X_{τ} := X$ for $τ \leq 0$ . The fractional powers are defined by means of the resolution of the identity generated by the inverse operator $B^{- 1}$ , see, e.g., [6, Section 2.3]. Note that the system of spaces $(X_{τ})_{τ \in R}$ , equipped with the respective norms, is strongly related to the Hilbert scale generated by the operator $B$ . However, for $τ < 0$ , topological completion of the spaces $X_{τ} = X$ with respect to the norm $∥ \cdot ∥_{τ}$ is not needed in our setting and thus is omitted.

In the present work, we discuss the nonlinear Tikhonov regularization (1.3) in particular with an oversmoothing penalty term, where we have $u^{†} \notin X_{1} = D (B)$ , or in other words $∥ u^{†} ∥_{1} = + \infty .$ This continues studies started in papers [9, 10] and [7], where convergence rates and numerical case studies are provided for a priori and a posteriori parameter choices, respectively, under certain smoothness assumptions on $u^{†}$ and structural conditions on $F$ . Under the same structural conditions, which are also similar to those in the corresponding seminal paper for linear operator equations by Natterer [18], we present as the novelty of this paper convergence results based on the Banach–Steinhaus theorem without needing any smoothness assumptions. The error estimates derived in the context of convergence assertions moreover allow us to prove low order convergence rates under associated (for example logarithmic) source conditions.

The outline of the remainder is as follows: in Section 2, we introduce Hilbert scales and formulate the basic assumptions, and in addition we establish well-posedness of Tikhonov regularization used in our setting. Then in Section 3, we introduce auxiliary elements needed for the proof of the convergence results, and in addition we provide first error estimates for Tikhonov regularization which are based on those auxiliary elements and which are needed for the subsequent convergence proofs. The regularizing properties of an a priori parameter choice as well as a discrepancy principle are considered in Section 4. The suggested discrepancy principle is considered in a form that is suitable for misfit functionals which may depend discontinuously on the regularization parameter $α > 0$ . Finally, as a byproduct of derived error estimates, we can prove low order convergence rates in Section 5.

2 Prerequisites and assumptions

2.1 Main assumptions

In the following assumption we briefly summarize the structural properties of the operator $F$ , of its domain $D (F)$ , in particular with respect to the the solution $u^{†}$ of equation (1.3). For examples of nonlinear inverse problems, which satisfy these assumptions (or at least substantial parts of it), we refer to [5, 7] and to the appendices of the papers [10, 25].

Assumption 2.1.

The operator $F : X \supset D (F) \to Y$ is sequentially continuous on $D (F)$ with respect to the weak topologies of the Hilbert spaces $X$ and $Y$ .
The domain of definition $D (F) \subset X$ is a closed and convex subset of $X$ .
Let $D := D (F) \cap D (B) = D (F) \cap X_{1}$ be a non-empty set.
Let the solution $u^{†} \in D (F)$ to equation (1.1) with right-hand side $f^{†}$ be an interior point of the domain $D (F)$ .
Let the data $f^{δ} \in Y$ satisfy the noise model (1.2) and let the initial guess $¯ ¯ ¯ u$ satisfy (1.4).
Let $a > 0$ and let there exist finite constants $0 < c_{a} \leq C_{a}$ such that the inequality chain

(2.1)

holds true for all $u \in D$ .

Remark 2.2.

From item (f) (left-hand inequality) of Assumption 2.1 we have for $u^{†} \in X_{1}$ that $u^{†}$ is the uniquely determined solution to equation (1.1) in the set $D$ . For $u^{†} \notin X_{1}$ there is no solution at all to (1.1) in $D$ . But in both cases alternative solutions $u^{*} \notin X_{1}$ with $u^{*} \in D (F)$ and $F u^{*} = f^{†}$ cannot be excluded.

2.2 Properties of regularized solutions of the Tikhonov regularization

Let for $α > 0$ minimizers of the Tikhonov functional $T_{α}^{δ}$ denote by $u_{α}^{δ}$ , i.e. we have $T_{α}^{δ} (u_{α}^{δ}) = {min}_{u \in D (F)} T_{α}^{δ} (u)$ and evidently by definition of the penalty term $u_{α}^{δ} \in D$ .

Example 2.3.

Let in this example $F = A : X \to Y$ with $D (F) = X$ be a bounded linear operator with non-closed range $R (A)$ , and for simplicity let $¯ ¯ ¯ u = 0$ . In this setting, Tikhonov regularized solutions $u_{α}^{δ}$ solve the linear operator equation

(A^{*} A + α B^{- 2}) u_{α}^{δ} = A^{*} f^{δ} .

(2.2)

In the special situation of an injective operator $A$ and of a scale generator $B = (A^{*} A)^{- q / 2}$ with $q > 0$ , this gives

(A^{*} A + α (A^{*} A)^{q}) u_{α}^{δ} = A^{*} f^{δ},

and Assumption 2.1 is satisfied with $a = 1 / q$ then. The oversmoothing case $u^{†} \notin X_{1}$ here means that $u^{†} \notin R ((A^{*} A)^{q / 2})$ . This situation is discussed in the analysis of fractional Tikhonov regularization, and we refer for example to [3, 8, 15]. In Natterer’s paper [84] the analog

to the inequality chain (2.1) is the basis for error estimates and convergence rates results for linear operator equations. The constant $a > 0$ characterizes here the degree of ill-posedness of the problem.

The extremal problem (1.3) for finding regularized solutions is well-posed with respect to existence of minimizers and their stability in a sense specified in the following proposition. This follows by standard results from regularization theory (cf., e.g., [22, Chapter 2.6], [23, 24] and [21, Section 4.1.1]). So we give a sketch of proof only.

Proposition 2.4.

Let Assumption 2.1 be satisfied.

There exists for all $α > 0$ a minimizer $u_{α}^{δ}$ of the Tikhonov functional $T_{α}^{δ}$ in the set $D$ .
Each minimizing sequence of $T_{α}^{δ}$ over $D$ has a subsequence that converges strongly in $X_{1}$ to a minimizer $u_{α}^{δ} \in D$ of the Tikhonov functional.
For every $α > 0$ the regularized solutions $u_{α}^{δ}$ are stable with respect to small perturbations in the data $f^{δ}$ .

Proof.

The basic ingredients needed for the proof are as follows:

The operator $F$ , when considered as $F : X_{1} \supset D \to Y$ , is sequentially continuous with respect to the weak topologies on $X_{1}$ and $Y$ . This implies that the misfit functional $u \in D \mapsto ∥ F u - f^{δ} ∥ \in R$ is sequentially continuous with respect to the weak topology on $X_{1}$ .
The set $D$ is weakly closed in $X_{1}$ .
The stabilizing functional $∥ \cdot - ¯ ¯ ¯ u ∥_{1}^{2}$ is sequentially weakly lower continuous on $X_{1}$ .

The statement in the second item follows from the two facts that (i) a linear self-adjoint operator $B : X \supset D (B) \to X$ is weakly closed, and that (ii) each closed convex subset of a Hilbert space is weakly closed.

From these ingredients, it follows that each minimizing sequence $(u_{n}) \subset D$ of the Tikhonov functional has a subsequence which converges weakly in $X_{1}$ to a minimizer $u_{α}^{δ}$ , and the corresponding subsequence of $(∥ u_{n} - ¯ ¯ ¯ u ∥_{1})$ converges to $∥ u_{α}^{δ} - ¯ ¯ ¯ u ∥_{1}$ .

Remark 2.5.

We note that the minimizer of the Tikhonov functional may be non-unique. For an example, see [22, Example 1 in Chapter 2.6]. On the other hand, it should be mentioned that the properties of Tikhonov regularization in Hilbert spaces are well investigated when the penalty functional in the Tikhonov functional is replaced by $u \mapsto ∥ u - ¯ ¯ ¯ u ∥^{2}$ , cf., e.g., [6, Chapter 10] or [20, Section 3.1] and the references therein, respectively.

One of the two main goals of this study is to discuss convergence results for the Tikhonov regularization with oversmoothing penalty, i.e. $u^{†} \notin X_{1}$ (note, however, that this is not explicitly required anywhere), and the regularization error $u_{α}^{δ} - u^{†}$ is still measured in the norm of $X$ . This continues former studies like [7] under the assumption $u^{†} \in X_{p}$ for some $0 < p < 1$ . In contrast to those papers, the focus of the present work is, although also not explicitly required anywhere, on the case $u^{†} \notin X_{p}$ for each $0 < p < 1$ , consequently on the situation characterized by $p = 0$ . On the other hand, we also mention convergence assertions for $u^{†} \in X_{p}$ with $p \geq 1$ under the inequality chain (2.1).

3 Auxiliary elements and preparatory results

3.1 Auxiliary elements

In this section we consider auxiliary elements which are needed to verify our convergence results. As a preparation, we introduce the bounded, injective, selfadjoint, positive semidefinite linear operator

G := B^{- (2 a + 2)} : X \to X,

(3.1)

where the operator $B$ obeying the condition (1.5) is defined in Section 1, and $a > 0$ is introduced by item (f) of Assumption 2.1. Note that the range $R (G)$ of $G$ is not closed and hence zero is an accumulation point of the spectrum $σ (G) \subset [0, ∥ G ∥]$ of $G$ . In this context, we also mention that $u \in X_{p} (p > 0)$ is equivalent to $u \in R (G^{\frac{p}{2 a + 2}})$ , which means that $u$ obeys a power-type source condition $u = G^{\frac{p}{2 a + 2}} w$ with some source element $w \in X$ . In the case $p = 0$ , i.e. if $u \in X$ , but $u \notin X_{p}$ for all $p > 0$ , then it was shown in [16] and [11] that there exist an index function³³3According to [17] we call a function $φ : (0, \infty) \to (0, \infty)$ index function if it is continuous, non-decreasing and satisfies the limit condition ${lim}_{t \to 0} φ (t) = 0$ . $φ$ (for example of logarithmic type, cf. [13]) and a source element $w \in X$ such that a (low order) source condition $u = φ (G) w$ is satisfied.

The auxiliary elements based on the operator $G$ from (3.1) are defined as follows:

{ˆ u}_{α} := ¯ ¯ ¯ u + G (G + α I)^{- 1} (u^{†} - ¯ ¯ ¯ u) = u^{†} - α (G + α I)^{- 1} (u^{†} - ¯ ¯ ¯ u) for α > 0,

(3.2)

where the solution $u^{†}$ of the operator equation (1.1) and the corresponding initial guess $¯ ¯ ¯ u$ are as introduced above. The basic properties of the auxiliary elements are summarized in Lemma 3.1.

In order to specify the limit behaviour of different positive functions occurring in error estimates, we use in the sequel a collection of non-negative functions named $f_{i} (α) (i = 1, 2, \dots)$ and defined for $α > 0$ with the property

lim α \to 0 f_{i} (α) = 0,

(3.3)

to be supposed for all indices $i$ . Consequently, we have for all $i$ that $f_{i} (α) = o (1)$ as $α \to 0$ . Note that pairwise products $f_{i} (α) f_{j} (α)$ and linear combinations $K_{i} f_{i} (α) + K_{j} f_{j} (α)$ with non-negative constants $K_{i}, K_{j}$ can again be written as such a function $f_{k} (α) = o (1)$ as $α \to 0.$

Lemma 3.1.

There are functions $f_{i} (α) (i = 1, 2, 3)$ for $α > 0$ satisfying (3.3) such that the auxiliary elements from (3.2) have the following properties:

$∥ {ˆ u}_{α} - u^{†} ∥ = f_{1} (α) = o (1)$ as $α \to 0$ ,
$∥ {ˆ u}_{α} - u^{†} ∥_{- a} = f_{2} (α) α^{a / (2 a + 2)} = o (α^{a / (2 a + 2)})$ as $α \to 0$ ,
$∥ {ˆ u}_{α} - ¯ ¯ ¯ u ∥_{1} = f_{3} (α) α^{- 1 / (2 a + 2)} = o (α^{- 1 / (2 a + 2)})$ as $α \to 0$ .

Proof.

We show first that

∥ G^{θ} (G + α I)^{- 1} u ∥ = o (α^{θ - 1}) as α \to 0

(3.4)

holds for all $0 \leq θ < 1$ and $u \in X$ . It is well known that

∥ G (G + α I)^{- 1} ∥ \leq 1, ∥ (G + α I)^{- 1} ∥ \leq α^{- 1} for α > 0.

Then the interpolation inequality implies the estimate

∥ G^{θ} (G + α I)^{- 1} ∥ \leq α^{θ - 1} for α > 0 and 0 \leq θ \leq 1.

(3.5)

Note that the operator $G$ is selfadjoint and positive semidefinite, and thus the fractional powers $G^{θ}$ are well-defined.

In addition, for fixed $0 \leq θ < 1$ and any $u \in R (G^{q})$ with $q > 0$ chosen so small such that $θ + q \leq 1$ , we have, from (3.5) with $θ$ replaced by $θ + q$ ,

α^{1 - θ} ∥ G^{θ} (G + α I)^{- 1} u ∥ = α^{1 - θ} ∥ G^{θ} (G + α I)^{- 1} G^{q} v ∥ \leq α^{q} ∥ v ∥ \to 0 as α \to 0,

(3.6)

where $u = G^{q} v$ . Since for arbitrary $q > 0$ , the range of the operator $G^{q}$ is dense in $X$ , i.e., $¯ ¯¯¯¯¯¯¯¯¯¯¯¯¯ ¯ R (G^{q}) = X$ , due to (3.5) and (3.6) the Banach–Steinhaus theorem (cf., e.g., [14, Problem 10.1] or [19, Theorem 1.1.4]) may be applied to the operators $α^{1 - θ} G^{θ} (G + α I)^{- 1}$ for $α \to 0$ , with $0 \leq θ < 1$ being fixed. This finally gives the asymptotics (3.4) as $α \to 0$ .

For the functions

$f_{1} (α)$	$= ∥ α (G + α I)^{- 1} (u^{†} - ¯ ¯ ¯ u) ∥,$	(3.7)
$f_{2} (α)$	$= α^{- a / (2 a + 2)} ∥ G^{a / (2 a + 2)} [α (G + α I)^{- 1} (u^{†} - ¯ ¯ ¯ u) ∥,$	(3.8)
$f_{3} (α)$	$= α^{- (2 a + 1) / (2 a + 2)} ∥ G^{(2 a + 1) / (2 a + 2)} [α (G + α I)^{- 1} (u^{†} - ¯ ¯ ¯ u)] ∥,$	(3.9)

the statements of the lemma are now easily obtained from (3.4) and the following three representations,

	${ˆ u}_{α} - u^{†}$	$= - α (G + α I)^{- 1} (u^{†} - ¯ ¯ ¯ u),$
	$B^{- a} ({ˆ u}_{α} - u^{†})$	$= - G^{a / (2 a + 2)} [α (G + α I)^{- 1} (u^{†} - ¯ ¯ ¯ u)],$
	$B ({ˆ u}_{α} - ¯ ¯ ¯ u)$	$=G(2a+1)/(2a+2)[(G+αI)−1(u†−¯¯¯u)].\vbox\hruleheight0.4pt\vrule height 3.225pt width 0.5pt \vrule w% idth 0.5pt\hruleheight0.4pt$

3.2 Some estimates for oversmoothing Tikhonov regularization

Lemma 3.2.

Let Assumption 2.1 be satisfied. Then there is a function $f_{4} (α)$ for $α > 0$ satisfying (3.3) such that we have for all $δ > 0$

Proof.

For $α > 0$ small enough we have ${ˆ u}_{α} \in D$ , because item (a) of Lemma 3.1 holds and $u^{†}$ is an interior point of $D (F)$ . Thus

The first term on the right-hand side of the latter estimate can be written as

This is a consequence of item (b) of Lemma 3.1. The second term on the right-hand side of the latter estimate attains the form

\sqrt{α} ∥ {ˆ u}_{α} - ¯ ¯ ¯ u ∥_{1} \leq f_{3} (α) α^{a / (2 a + 2)},

based on item (c) of Lemma 3.1. This yields the function

f_{4} (α) := C_{a} f_{2} (α) + f_{3} (α) = o (1) as α \to 0,

which completes the proof of the lemma.

Corollary 3.3.

Let Assumption 2.1 be satisfied. Then there is a function $f_{5} (α)$ for $α > 0$ satisfying (3.3) and a constant $K_{1} > 0$ such that we have for all $δ > 0$

∥ u_{α}^{δ} - u^{†} ∥_{- a} \leq f_{5} (α) α^{a / (2 a + 2)} + K_{1} δ .

Proof.

It follows from the left-hand estimate in (2.1) and Lemma 3.2 that

and hence the required assertion of the corollary with $f_{5} (α) := f_{4} (α) / c_{a}$ and $K_{1} := 2 / c_{a} .$

The error $∥ u_{α}^{δ} - u^{†} ∥$ is now estimated by the following series of error estimates. Using the triangle inequality and Lemma 3.1, we obtain

(3.10)

and below we consider the term $∥ u_{α}^{δ} - {ˆ u}_{α} ∥$ in more detail. From the interpolation inequality for bounded linear, self-adjoint and positive semidefinite operators on Hilbert spaces, cf. [6, (2.49)], it follows

∥ u_{α}^{δ} - {ˆ u}_{α} ∥

\leq ∥ u_{α}^{δ} - {ˆ u}_{α} ∥_{- a}^{1 / (a + 1)} ∥ u_{α}^{δ} - {ˆ u}_{α} ∥_{1}^{a / (a + 1)} .

(3.11)

Both terms on the right-hand side of the estimate (3.11) can be estimated by using Corollary 3.3 and Lemma 3.1 in the following manner: Precisely, we find with $f_{6} (α) := f_{2} (α) + f_{5} (α)$ and $f_{7} (α) := f_{3} (α) + f_{4} (α)$ the estimates

	$∥ u_{α}^{δ} - {ˆ u}_{α} ∥_{- a}$
	$∥ u_{α}^{δ} - {ˆ u}_{α} ∥_{1}$

Thus we can continue estimating (3.11). Introducing $f_{8} (α) := max {f_{6} (α), f_{7} (α)}$ and $K_{2} = max {K_{1}, 1}$ , we obtain

	$∥ u_{α}^{δ} - {ˆ u}_{α} ∥$
		$\leq {(f_{8} (α) α^{a / (2 a + 2)} + K_{2} δ)}_{8}^{1 / (a + 1)} {(α^{- 1 / 2} (f_{8} (α) α^{a / (2 a + 2)} + K_{2} δ))}^{a / (a + 1)}$
		$= α^{- a / (2 a + 2)} (f_{8} (α) α^{a / (2 a + 2)} + K_{2} δ) = f_{8} (α) + K_{2} \frac{δ}{α^{a / (2 a + 2)}} .$

From the latter estimate and (3.10), the following proposition now immediately follows by considering $f_{9} (α) := f_{1} (α) + f_{8} (α)$ there.

Proposition 3.4.

Let Assumption 2.1 be satisfied. Then there is a function $f_{9} (α)$ for $α > 0$ satisfying (3.3) and a constant $K_{2} > 0$ such that we have for all $δ > 0$

(3.12)

The inequality (3.12), which is valid for arbitrary noise levels $δ > 0$ and regularization parameters $α > 0$ , allows us to formulate in the subsequent section sufficient conditions for the convergence of the error norm $∥ u_{α}^{δ} - u^{†} ∥$ of the regularized solutions.

4 Convergence results

4.1 Main theorem

The following main theorem is an immediate consequence of the error estimates outlined in the preceding section. The formulated convergence result follows immediately from the inequality (3.12).

Theorem 4.1.

Let Assumption 2.1 be satisfied. Then for any a priori parameter choice $α_{*} = α (δ)$ and any a posteriori parameter choice $α_{*} = α (δ, y^{δ})$ , the regularized solutions $u_{α_{*}}^{δ}$ converge in the norm of the Hilbert space $X$ to the solution $u^{†}$ of the operator equation (1.1) for $δ \to 0$ , i.e. ${lim}_{δ \to 0} ∥ u_{α_{*}}^{δ} - u^{†} ∥ = 0$ , whenever

α∗→0and\footnotesize\raisebox{-1.42% 2638pt}{δ}\footnotesize\raisebox{2.27622pt}{αa/(2a+2)∗}→0asδ→0.

(4.1)

Remark 4.2.

α_{*}^{a / (2 a + 2)} \sim δ as δ \to 0,

(4.2)

then convergence cannot be derived in this way, because in that borderline case the second term on the right-hand side of inequality (3.12) does not tend to zero.

Remark 4.3.

The convergence result of Theorem 4.1 applies to both situations: (a) the classical case $u^{†} \in D (B)$ and moreover the case (b) of oversmoothing penalties $u^{†} \notin D (B)$ . Since this theorem is immediately based on formula (3.12), an inspection of the proof of this formula from Lemma 3.2 to Proposition 3.4 shows that both inequalities in (2.1) (left-hand inequality and right-hand inequality) as structural conditions on the nonlinearity of $F$ in a neighbourhood of $u^{†}$ are needed for the convergence result of Theorem 4.1. Nevertheless, for the oversmoothing case (b), this is real progress, since the convergence result of Theorem 4.1 does not use any form of additional smoothness on $u^{†}$ , and such convergence assertions without smoothness assumptions like $u^{†} \in X_{p}$ for some $p \in (0, 1)$ (cf. [10]) are missing by now for the oversmoothing case in the literature.

As is well known, in case (a) with $u^{†} \in D (B)$ the parameter choice condition

α∗→0and\footnotesize\raisebox{-1.42% 2638pt}{δ2}\footnotesize\raisebox{2.27622pt}{α∗}→0asδ→0,

which is stronger than (4.1), is always sufficient for convergence of regularized solutions, and inequalities occurring in (2.1) represent only tools for obtaining convergence rates. On the other hand, in the limit situation $α_{*} \sim δ^{2}$ of choosing the regularization parameter, one needs the left-hand inequality in (2.1) for obtaining convergence rates, and this inequality occurs here as a conditional stability estimate (cf. [7, Prop. 3], [5, Theorem 1.1] and references therein). Convergence is then a consequence of derived convergence rates.

4.2 A priori parameter choice of monomial type

In this subsection we consider in light of Theorem 4.1 the a priori parameter choice

α_{*} = α (δ) \sim δ^{κ}

(4.3)

for exponents $κ > 0$ . Then condition (4.1) is satisfied if and only if $0 < κ < 2 + \frac{2}{a}$ , and the borderline condition (4.2) holds if and only if $κ = 2 + \frac{2}{a}$ . This gives the following proposition.

Proposition 4.4.

For the a priori choice (4.3) of the regularization parameter $α > 0$ , condition (4.1) in Theorem 4.1 holds if and only if $0 < κ < 2 + \frac{2}{a}$ . For all $a > 0$ , the choice $α_{*} \sim δ^{2}$ yields convergence.

We can distinguish the $κ$ -intervals (A): $0 < κ < 2$ , (B): $κ = 2$ , and (C): $2 < κ < 2 + \frac{2}{a}$ for (4.3). Then we have $\frac{δ^{2}}{α_{*}} \to 0$ as $δ \to 0$ in situation (A) and $\frac{δ^{2}}{α_{*}} \sim 1$ in situation (B). Note that both situation also occur and yield convergent regularized solutions in the oversmoothing case $u^{†} \notin D (B)$ . This is a bit surprising, because the behaviour

\frac{δ^{2}}{α_{*}} \to \infty as δ \to 0

(4.4)

occurring in situation (C) was supposed in the literature to be typical for the case of oversmoothing penalties. Namely as is seen in [9], convergence rate results of the form

∥ u_{α_{*}}^{δ} - u^{†} ∥ = O (δ^{\frac{p}{a + p}}) as δ \to 0

are obtained under the both-sided structural condition (2.1) and in particular under the smoothness assumption $u^{†} \in X_{p}$ for $0 < p < 1$ whenever the a priori parameter choice of type (4.3) with prescribed exponent $κ = \frac{2 (a + 1)}{a + p} = 2 + \frac{2 (1 - p)}{a + p}$ applies. Evidently, this prescribed $κ$ satisfies the conditions (4.4) and $2 < κ < 2 + \frac{2}{a}$ for all $0 < p < 1$ . It is important to note that $p = 0$ coincides with the borderline case $κ = 2 + \frac{2}{a}$ which, however, is not sufficient for convergent regularized solutions.

4.3 A discrepancy principle

For the specification of an appropriate discrepancy principle, the behaviour of the misfit functional needs to be described, for $δ > 0$ fixed. The basic properties are summarized in the following proposition.

Proposition 4.5.

Let Assumption 2.1 be satisfied. Then for $δ > 0$ fixed, the function is non-decreasing, with

(4.5)

We have ${lim}_{α \to \infty} ∥ u_{α}^{δ} - ¯ ¯ ¯ u ∥ = 0$ .

Proof.

We start with the verification of the first statement of the proposition. As a preparation, we show that the function is non-increasing. Indeed, for $0 < α \leq β$ fixed we have

	$T_{β}^{δ} (u_{β}^{δ})$	$\leq T_{β}^{δ} (u_{α}^{δ}) = T_{α}^{δ} (u_{α}^{δ}) + (β - α) ∥ u_{α}^{δ} - ¯ ¯ ¯ u ∥_{1}^{2}$
		$\leq T_{α}^{δ} (u_{β}^{δ}) + (β - α) ∥ u_{α}^{δ} - ¯ ¯ ¯ u ∥_{1}^{2}$

and thus $∥ u_{β}^{δ} - ¯ ¯ ¯ u ∥_{1} \leq ∥ u_{α}^{δ} - ¯ ¯ ¯ u ∥_{1}$ . The first statement of the proposition is now easily obtained: for $0 < α \leq β$ we have

		$= T_{α}^{δ} (u_{α}^{δ}) \leq T_{α}^{δ} (u_{β}^{δ}) = ∥ F u_{β}^{δ} - f^{δ} ∥^{2} + α ∥ u_{β}^{δ} - ¯ ¯ ¯ u ∥_{1}^{2}$

and thus $∥ F u_{α}^{δ} - f^{δ} ∥ \leq ∥ F u_{β}^{δ} - f^{δ} ∥$ .

Next we consider the latter statement of the proposition. There holds

(4.6)

and thus in particular $∥ u_{α}^{δ} - ¯ ¯ ¯ u ∥_{1} = O (α^{- 1 / 2})$ as $α \to \infty$ . The estimate (1.5) implies $∥ u_{α}^{δ} - ¯ ¯ ¯ u ∥ = O (α^{- 1 / 2})$ as $α \to \infty$ , which implies the latter statement of the proposition.

The first statement in (4.5) follows directly from Lemma 3.2, and we finally consider the second statement in (4.5). From (4.6) we already know that ${lim}_{α \to \infty} ∥ F u_{α}^{δ} - f^{δ} ∥$ $\leq ∥ F ¯ ¯ ¯ u - f^{δ} ∥$ . Conversely, sequential weak continuity of the operator $F$ implies weak convergence $F u_{α}^{δ} ⇀ F ¯ ¯ ¯ u$ as $α \to \infty$ , and thus . This completes the proof of the proposition.

Remark 4.6.

Notice that in the proof of Proposition 4.5, no use of the fundamental estimates (2.1) on the smoothing property of $F$ is made in fact. Notice also that the statement of Proposition 4.5 is quite similar to related results for Tikhonov regularization with non-oversmoothing penalty, cf. [1], [22, Section 2.6] and [24, Section 6.7],

It follows from Proposition 4.5 that the following version of the discrepancy principle (cf. [23, 24]) is implementable. It determines, for each noise level $δ > 0$ , an approximation $u^{δ} \in D$ . Possibly discontinuities of the misfit functional are taken into account.

Algorithm 4.7 (Discrepancy principle).

If $∥ F ¯ ¯ ¯ u - f^{δ} ∥ \leq b δ$ holds, then choose $α_{*} = \infty$ , i.e., $u^{δ} := u_{\infty}^{δ} := ¯ ¯ ¯ u \in D$ .
Otherwise choose a finite parameter $α =: α_{*} > 0$ such that

(4.7)

where $b > 1$ and $c > 1$ are finite constants, and let $u^{δ} := u_{α_{*}}^{δ}$ .

Algorithm 4.7 can be realized by the following strategy.

Remark 4.8 (Sequential discrepancy principle).

Practically, a parameter $α_{*}$ satisfying condition (4.7) can be determined, e.g., by choosing a constant $θ > 1$ and an initial guess $α^{(0)} > 0$ and proceeding then as follows:

If $∥ F u_{α^{(0)}}^{δ} - f^{δ} ∥ \geq b δ$ holds then, with the notation $α^{(k)} = θ^{- k} α^{(0)}$ , proceed for $k = 1, 2, \dots$ until $∥ F u_{α^{(k)}}^{δ} - f^{δ} ∥ \leq b δ \leq ∥ F u_{α^{(k - 1)}}^{δ} - f^{δ} ∥$ is satisfied for the first time; define $α_{*} = α^{(k)}$ then.
If $∥ F u_{α^{(0)}}^{δ} - f^{δ} ∥ \leq b δ$ holds then, with the notation $α^{(k)} = θ^{- k} α^{(0)}$ , proceed for $k = 1, 2, \dots$ until $∥ F u_{α^{(k - 1)}}^{δ} - f^{δ} ∥ \leq b δ \leq ∥ F u_{α^{(k)}}^{δ} - f^{δ} ∥$ is satisfied for the first time; define $α_{*} = α^{(k - 1)}$ then.

The regularizing properties of Algorithm 4.7 are stated in the following theorem.

Theorem 4.9.

Let Assumption 2.1 be satisfied. For the a posteriori parameter choice introduced in Algorithm 4.7, we have

∥ u^{δ} - u^{†} ∥ \to 0, \frac{δ}{α_{*}^{a / (2 a + 2)}} \to 0 as δ \to 0.

(4.8)

Proof.

For an arbitrary countable noise level set $Δ \subset R_{+}$ having the origin as only accumulation point, we consider the following three cases: (a) $α_{*} = \infty$ for each $δ \in Δ$ , (b) $α_{*} \to 0$ as $Δ ∋ δ \to 0$ , and (c) $α_{*} < \infty$ for each $δ \in Δ$ , ${liminf}_{Δ ∋ δ \to 0} α_{*} > 0$ . Below we show that in each of those three cases, (4.8) holds, if the noise level in addition satisfies $δ \in Δ$ , respectively. The main statement of the theorem then follows by arguing for subsequences. Note that in cases (a) and (c), the second statement in (4.8) trivially holds.

The case $α_{*} = \infty$ for $δ \in Δ$ means $∥ F ¯ ¯ ¯ u - f^{δ} ∥ \leq b δ$ for $δ \in Δ$ , and thus $F ¯ ¯ ¯ u = f^{†}$ , and then $u^{δ} = ¯ ¯ ¯ u = u^{†}$ for $δ \in Δ$ .
Suppose that $α_{*} \to 0$ as $Δ ∋ δ \to 0$ . From Lemma 3.2, we obtain

$b δ \leq ∥ F u_{β_{*}}^{δ} - f^{δ} ∥ \leq o (β_{*}^{a / (2 a + 2)}) + δ = o (α_{*}^{a / (2 a + 2)}) + δ,$

and thus $δ / α_{*}^{a / (2 a + 2)} \to 0$ as $Δ ∋ δ \to 0$ . Proposition 3.4 then yields $∥ u_{α_{*}}^{δ} - u^{†} ∥ \to 0$ as $Δ ∋ δ \to 0$ .
Next suppose that both $α_{*} < \infty$ for $δ \in Δ$ , and

$liminf Δ ∋ δ \to 0 α_{*} > 0.$ (4.9)
- We first observe that
  
  $c_{a} ∥ u_{α_{*}}^{δ} - u^{†} ∥_{- a} \leq ∥ F u_{α_{*}}^{δ} - f^{†} ∥ \leq ∥ F u_{α_{*}}^{δ} - f^{δ} ∥ + δ \leq (b + 1) δ,$
  
  so $∥ u_{α_{*}}^{δ} - u^{†} ∥_{- a} = O (δ)$ as $Δ ∋ δ \to 0$ . Note that the asymptotics (4.9) is not needed for this result.
- There holds
  
  $∥ B u_{α_{*}}^{δ} ∥ = O (1) as Δ ∋ δ \to 0.$ (4.10)
  
  This easily follows from (4.6) and (4.9), in combination with the estimate .
- We next show
  
  $u^{†} \in D (B) .$ (4.11)
  
  Estimates (4.10) and (1.5) imply $∥ u_{α_{*}}^{δ} ∥ = O (1)$ as $Δ ∋ δ \to 0$ , and thus we obtain weak convergence
  
  $u_{α_{*}}^{δ} ⇀ v, B u_{α_{*}}^{δ} ⇀ w as Δ ∋ δ \to 0$
  
  for some elements $v, w \in X$ . Those two weak convergence statements initially hold for subsequences only, but then without loss of generality we may assume that it holds for the whole considered system.
  
  Since the operator $B$ is weakly closed, we have
  
  $v \in D (B) = X_{1} .$
  
  The operator $F$ is sequentially weakly continuous, thus
  
  $F u_{α_{*}}^{δ} ⇀ F v as Δ ∋ δ \to 0.$
  
  Algorithm 4.7 implies $∥ F u_{α_{*}}^{δ} - F u^{†} ∥ \to 0$ as $δ \to 0$ , so $F v = F u^{†}$ , and the lower bound in (2.1) finally gives $v = u^{†}$ , which means (4.11).
  
  The interpolation inequality then gives
  
  This completes the proof of the theorem.

Note that in the oversmoothing situation $u^{†} \notin X_{1}$ , case (c) in the proof of Theorem 4.9 does not emerge, cf. (4.11). This fact is, in a similar setting, already observed in [10, Lemma 1].

Remark 4.10.

Notice that the situation (b) in the proof of Theorem 4.9 is the regular case in applications. The case (c) is an exceptional case which, in the non-oversmoothing case, can be excluded, if the exact penalization veto is satisfied. This veto had been introduced in the paper [1]; see also [2].

5 Low order convergence rates

Our convergence assertion established in the main theorem formulated in Subsection 4 is due to the error estimate (3.12) derived in Section 3. The presented sufficient conditions for convergence are based on the Banach–Steinhaus theorem and do not need any form of solution smoothness. In other words, the case $p = 0$ is included, where $u^{†}$ does not satisfy a power-type source condition. However, as already mentioned above, there exists at least a source condition of lower order for solution element $u^{†} \in X$ . Precisely, there is always an index function $φ$ and a source element $w \in X$ such that

u^{†} - ¯ ¯ ¯ u = φ (G) w .

(5.1)

Based on formula (3.12) and taking into account the representations (3.7), (3.8) and (3.9), we can derive for such source condition low order convergence rates in the case of oversmoothing penalties as a byproduct of the studies presented in Section 3. We will outline this in the following. First we obtain from [4, Prop. 3.3] the following lemma, where we refer to [17] for the concept of qualification of a regularization method.

Lemma 5.1.

An index function $φ$ is a qualification of the classical Tikhonov regularization related to the operator $G$ , which means that there are positive constants $C$ and $¯ ¯¯ ¯ α$ such that

sup 0 < λ \leq ∥ G ∥ \frac{α φ (λ)}{λ + α} \leq C φ (α) (0 < α \leq ¯ ¯¯ ¯ α),

whenever for some $0 < ¯ t \leq ∥ G ∥$ the quotient function $φ (t) / t$ is non-increasing for $0 < t \leq ¯ t$ .

Corollary 5.2.

If, for the index function $φ$ and all exponents $η > 0$ , the quotient functions $t^{η} / φ (t)$ are strictly increasing for sufficiently small $t > 0$ , then the index function $t^{θ} φ (t)$ is for all $0 \leq θ < 1$ a qualification of the classical Tikhonov regularization related to the operator $G$ .

Proof.

We have that for all $0 \leq θ < 1$ the quotient function $\frac{t^{θ} φ (t)}{t} = \frac{φ (t)}{t^{1 - θ}}$ with $1 - θ > 0$ is non-increasing for sufficiently small $t > 0$ . Consequently there are according to Lemma 5.1 positive constants $C$ and $¯ ¯¯ ¯ α$ depending on $θ$ such that

sup0<λ≤∥G∥αλθφ(λ)λ+α≤Cαθφ(α)(0<α≤¯¯¯¯α).\vbox\hruleheight0.4ptwidth100ht3.225ptwidth0.5pt\vrulewidth0.5pt\hruleheight0.4ptwidth100

(5.2)

Theorem 5.3.

Let Assumption 2.1 and the source condition (5.1) be satisfied, where it is supposed that for all $η > 0$ the index function $φ$ has an strictly increasing quotient function $t^{η} / φ (t)$ for sufficiently small $t > 0$ . Then we have, for some positive constant $K_{0}$ and $K_{2}$ from (3.12) and for all $δ > 0$ and sufficiently small $α > 0$ , the error estimate

(5.3)

Proof.

The functions $f_{1}, f_{2}$ and $f_{3}$ from Lemma 3.1 satisfy

f_{1} (α) = O (φ (α)), f_{2} (α) = O (φ (α)), f_{3} (α) = O (φ (α)) as α \to 0.

These properties are immediate consequences from (5.2) taking into account the three representations (3.7), (3.8) and (3.9). Since the function $f_{9} (α)$ in the error estimate (3.12) can be estimated from above by a linear combination of the functions $f_{1}, f_{2}$ and $f_{3}$ , there is a positive constant $K_{0}$ such that $f_{9} (α) \leq K_{0} φ (α)$ holds for sufficiently small $α > 0$ .

This provides us directly with the following low order convergence rate result.

Corollary 5.4.

Set under the assumptions of Proposition 5.3 $ψ (α) := φ (α) α^{\frac{a}{2 a + 2}}$ and $α_{*} = α (δ) := ψ^{- 1} (δ)$ . Then we have

∥ u_{α_{*}}^{δ} - u^{†} ∥ = O (φ (ψ^{- 1} (δ)) a s δ \to 0.

Example 5.5.

In this example, we consider source conditions (5.1) of logarithmic type with the function

φ (t) = φ_{log}^{κ} (t) := (- log (t))^{- κ} (κ > 0),

which is strictly concave for sufficiently small $t > 0$ and can be extended to $(0, \infty)$ as an index function. Is is evident for all $η, κ > 0$ that the quotient function $t^{η} / φ_{log}^{κ} (t)$ is strictly increasing for sufficiently small $t > 0$ and Corollary 5.2 applies. This yields the error estimate (5.3) written as

∥ u_{α}^{δ} - u^{†} ∥ \leq K_{0} {(- log (α))}^{- κ} + K_{2} \frac{δ}{α^{a / (2 a + 2)}} .

For the a priori choice $α_{*} =$

Convergence results and low order rates for nonlinear Tikhonov regularization with oversmoothing penalty term

Authors

Related Research

Tikhonov functionals with a tolerance measure introduced in the regularization

On Tikhonov functionals penalized by Bregman distances

Source Conditions for non-quadratic Tikhonov Regularisation

A new interpretation of (Tikhonov) regularization

Convergence Rates for Oversmoothing Banach Space Regularization

Nonlinear Tikhonov regularization in Hilbert scales with oversmoothing penalty: inspecting balancing principles

Maximal Spaces for Approximation Rates in ℓ^1-regularization

This week in AI

1 Introduction

2 Prerequisites and assumptions

2.1 Main assumptions

Assumption 2.1.

Remark 2.2.

2.2 Properties of regularized solutions of the Tikhonov regularization

Example 2.3.

Proposition 2.4.

Proof.

Remark 2.5.

3 Auxiliary elements and preparatory results

3.1 Auxiliary elements

Lemma 3.1.

Proof.

3.2 Some estimates for oversmoothing Tikhonov regularization

Lemma 3.2.

Proof.

Corollary 3.3.

Proof.

Proposition 3.4.

4 Convergence results

4.1 Main theorem

Theorem 4.1.

Remark 4.2.

Remark 4.3.

4.2 A priori parameter choice of monomial type

Proposition 4.4.

4.3 A discrepancy principle

Proposition 4.5.

Proof.

Remark 4.6.

Algorithm 4.7 (Discrepancy principle).

Remark 4.8 (Sequential discrepancy principle).

Theorem 4.9.

Proof.

Remark 4.10.

5 Low order convergence rates

Lemma 5.1.

Corollary 5.2.

Proof.

Theorem 5.3.

Proof.

Corollary 5.4.

Example 5.5.