## 1. Introduction

A function in the tensor product of two Hilbert spaces and may be identified with a compact operator . This identification is possible when the norm on is not weaker than the injective norm, i.e., in a certain sense the norm on is compatible with the norms on and . In such a case we can decompose as

(1.1) |

for a non-negative non-increasing sequence and
orthonormal
systems
and
. This is the well known
*singular value decomposition (SVD)* and it provides low-rank
approximations via

if, e.g., is the canonical norm on a Hilbert tensor space.

If is the Sobolev space of once weakly differentiable functions, the above assumption is not satisfied and there is no SVD for a function in general. The focus of this work is to explore variants of the SVD in different ambient spaces in the -norm. In part I, we showed that low-rank approximations in the Tucker format in exist. More precisely,

###### Theorem 1.1.

is weakly closed and therefore proximinal in .

We also showed under which conditions

i.e., belongs to the tensor product of its minimal subspaces. Finally, we analyzed the -error of the -SVD for a general order .

In this part, we consider the intersection space structure of

We analyze the -error of the - and -SVDs. We also consider an isometric embedding of into a space which allows the direct application of the SVD.

The paper is organized as follows. In Section 2, we consider the - and -SVDs and generalizations to higher dimensions. In Section 3, we consider the SVD in higher-dimensional spaces of mixed smoothness, exponential sum approximations and an isometric embedding of that allows a direct application of the SVD. We conclude in Section 4 with some simple numerical experiments with different types of low-rank approximations. Section 5 summarizes the results of part I and part II.

## 2. SVD in and

Before we proceed with analyzing SVDs in and , we consider the corresponding singular values and compare them to -singular values.

### 2.1. and Singular Values

We consider a function as an element of the intersection space We first consider as a Hilbert Schmidt operator defined by

The difference to simply viewing as an integral kernel arises when we consider the adjoint

The corresponding left and right singular functions and are respectively given by

and

(2.1) |

with the corresponding singular values sorted in decreasing order. Note that, unlike in the previous subsection, in general . To guarantee this we would have to require . This means that the sum does not make sense in in general, only in .

Similarly, we can interpret as a Hilbert Schmidt operator

defined by

with an adjoint given by The corresponding singular functions and satisfy

and

(2.2) |

where are the corresponding singular values, sorted in decreasing order. We make the following immediate observation.

###### Proposition 2.1.

Let and let , and be the SVD of interpreted as an element of , and respectively. Then, we have for all

###### Proof.

The first statement is given by

and

Analogously for the second statement. ∎

Note that the upper bounds in Proposition 2.1 do not necessarily hold component-wise, i.e., the inequalities

do not hold in general. This is due to the fact that when estimating the injective norm

the functions are not orthonormal in and the sequence is not necessarily decreasing.

Naturally, we can ask whether we can derive a bound of the sort

for some sequence . Though we do not believe this is possible without further assumptions, we can nonetheless improve the bounds. This indicates that indeed the quantities and are closely related. This will later be confirmed by numerical observations.

###### Theorem 2.2.

Let and assume the -SVD converges in . Then, we have

###### Proof.

We consider the -SVD
.
is identified with an operator
.
For any ,

converges in and for any ,
convergences in .
Thus,

On the other hand, utilizing the -SVD of , we have

and thus

Substituting , we obtain
since are -orthonormal.
Finally, taking the -norm of both sides and since
are -orthonormal, we obtain

The statement for follows analogously by
identifying with an operator from to
. This completes the
proof.
∎

The factors in the bounds in Theorem 2.2 reflect how , normalized in , scales w.r.t. , normalized in . For instance, if behaves like Fourier or wavelet basis, then . In this case, the right hand side in Theorem 2.2 evaluates to This leads precisely to the upper bound of Proposition 2.1. Analogous conclusions hold when considering , and .

Extending the results of this subsection to using HOSVD singular values and, e.g., the Tucker format is straightforward. Since we can consider matricizations w.r.t. to each separately, the analysis effectively reduces to the case . Difficulties arise only when considering simultaneous projections in different components of the tensor product space. There we have to assume the rescaled singular values converge, as was done in part I of this work.

### 2.2. and projections

Given the singular functions

and
associated with and
SVDs of respectively, we consider the finite
dimensional subspaces

(2.3) | ||||

and the corresponding -orthogonal projections

(2.4) |

The tensor product is well defined on , and on this space it holds

(2.5) |

However, the interpretation is problematic when considering on the closure of . Take, e.g., the projection . This is an orthogonal projection on and we have But in general unless . Thus, the subsequent application does not necessarily make sense and is not continuous.

Notice the difference with the projections and for -SVD from part I (for and ). First, we had since both the left and right projections already give the best rank approximation in . Second, we required only -orthogonality, thus preserving -regularity in the image. Thus, made sense on , although the sequence of projections does not necessarily converge in . To that end, we had to additionally assume in part I the convergence of the rank- approximations , or convergence of the rescaled -singular values.

In the present case, although we obtain optimality in the stronger -norm, we lose convergence or possibly even boundedness in the -norm. Thus, we can ask ourselves if is bounded from to , i.e., if ?

Specifically, what are the minimal assumptions - if any - that we require in order to achieve this? The next example shows that indeed even for simple projections this property is not guaranteed.

###### Example 2.3.

Let and consider the space . We know . Consider defined by Clearly, is a linear functional. Moreover, since any such is absolutely continuous, is bounded in the -norm. Thus, . By the Riesz representation theorem, there exists a unique , such that , for all .

Define the one dimensional subspace . The corresponding -orthogonal projection is given by Consider the sequence

. Clearly, for any , , and , for all . Thus, can not be continuous in .

A closer look at the preceding example shows that such a function differentiated twice yields the delta distribution. Therefore, it can not be in . On the other hand, if the function has -regularity, as the next statement shows, we can indeed obtain boundedness in .

###### Lemma 2.4.

Let . In addition, assume the second unidirectional derivatives of exist in the distributional sense and are bounded, i.e., Finally, assume satisfies either zero Dirichlet or zero Neumann boundary conditions. Then, the projections defined in (2.4) can be bounded as

###### Proof.

One can easily verify that and are twice weakly differentiable for any . For any , we can write The coefficients can be written as

where the boundary term vanishes due to the boundary conditions. Thus, we get

Analogously for . This completes the proof. ∎

Note that in principle the assumption on the boundary conditions can be replaced or avoided, as long as we can estimate the appearing boundary term. The assumption can be avoided entirely by using an estimate for the -norm via the Gagliardo-Nirenberg inequality, although this would yield a crude estimate and dimension dependent regularity requirements.

Under the conditions of Lemma 2.4, we can assert that is indeed continuous. Since is a uniform crossnorm

and similarly for . Thus, By density, we can uniquely extend onto and the identity (2.5) holds.

One might argue that requiring and to be continuous in is unnecessary, since we only need that the mappings and are continuous. The following example shows that indeed need not be continuous even on elementary tensor products, if is not continuous.

###### Example 2.5.

To summarize our findings, let us define the finite dimensional subspaces Under the assumptions of Lemma 2.4, and . This can also be observed by, e.g., considering (2.1) and integrating the second term by parts. We can estimate the error as follows.

###### Theorem 2.6.

Let the assumptions of Lemma 2.4 be satisfied. Moreover, define the constants

Then, the projection error is bounded as

###### Proof.

For the lower bound observe first that

Since is the optimal rank approximation in the -norm, we can further estimate

and similarly for . This gives the lower bound.

For the upper bound, since is orthogonal in the -norm, we get

Comments

There are no comments yet.