Exponential tractability of linear weighted tensor product problems in the worst-case setting for arbitrary linear functionals

We study the approximation of compact linear operators defined over certain weighted tensor product Hilbert spaces. The information complexity is defined as the minimal number of arbitrary linear functionals which is needed to obtain an ε-approximation for the d-variate problem. It is fully determined in terms of the weights and univariate singular values. Exponential tractability means that the information complexity is bounded by a certain function which depends polynomially on d and logarithmically on ε^-1. The corresponding un-weighted problem was studied recently by Hickernell, Kritzer and Woźniakowski with many negative results for exponential tractability. The product weights studied in the present paper change the situation. Depending on the form of polynomial dependence on d and logarithmic dependence on ε^-1, we study exponential strong polynomial, exponential polynomial, exponential quasi-polynomial, and exponential (s,t)-weak tractability with max(s,t)>1. For all these notions of exponential tractability, we establish necessary and sufficient conditions on weights and univariate singular values for which it is indeed possible to achieve the corresponding notion of exponential tractability. The case of exponential (s,t)-weak tractability with max(s,t)<1 is left for future study.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

02/02/2021

Tractability of approximation in the weighted Korobov space in the worst-case setting – a complete picture

In this paper, we study tractability of L_2-approximation of one-periodi...
10/05/2021

Segre-Driven Radicality Testing

We present a probabilistic algorithm to test if a homogeneous polynomial...
06/29/2011

The Rate of Convergence of AdaBoost

The AdaBoost algorithm was designed to combine many "weak" hypotheses th...
05/09/2022

Exponential tractability of L_2-approximation with function values

We study the complexity of high-dimensional approximation in the L_2-nor...
01/11/2021

On the power of standard information for tractability for L_2-approximation in the randomized setting

We study approximation of multivariate functions from a separable Hilber...
01/12/2021

On the power of standard information for tractability for L_2-approximation in the average case setting

We study multivariate approximation in the average case setting with the...
10/07/2020

A note on Korobov lattice rules for integration of analytic functions

We study numerical integration for a weighted Korobov space of analytic ...

1 Introduction

This paper deals with tractability of linear multivariate problems, which has been studied in a large number of papers, and is at the core of the research field of Information-Based Complexity (IBC). For introductions to IBC, we refer to the books [29] and [30]. For a recent and comprehensive overview of results on tractability, we refer the interested reader to the trilogy [17, 18, 19].

In the present paper we study the information complexity of a compact linear operator from a separable Hilbert space into another Hilbert space . The information complexity is defined as the minimal number of linear functionals needed by an algorithm which approximates to within an error threshold of . As shown in [5], without loss of generality we may consider only continuous linear functionals from the class .

We use different notions of tractability to describe how the information complexity of a given problem depends on and as tends to infinity in an arbitrary way. A problem is called intractable if its information complexity depends exponentially on or . If a problem is tractable, we describe sub-exponential dependence on some powers of and  by using the classification into various notions of tractability, which can be summarized by the algebraic (abbreviated ALG) and exponential (abbreviated EXP) cases. For the algebraic case, we need to verify that is bounded by certain functions of and which are, in particular, not exponential in some powers of and . For the exponential case, we replace by , and consider the same notions of tractability as for the algebraic case.

Most papers on tractability have dealt with the notions of algebraic tractability, which can be said to be the “standard” case of tractability. For an overview of results we again refer to [17, 18, 19] and the references therein, as well as [20] and [32].

However, there is also a recent stream of work on exponential tractability, and this is what we are going to study here. For results on exponential tractability for general linear problems (without necessarily assuming tensor product structure), we refer the reader to the recent paper [12], for exponential tractability for linear problems on un-weighted tensor product spaces, we refer the reader to [4, 21, 22]. Further results on exponential tractability can, e.g., be found in the papers [1, 2, 6, 7, 8, 10, 11, 13, 14, 15, 16, 23, 27, 28, 31, 33].

For un-weighted tensor product problems, such as studied in [4], we have many negative results for exponential tractability. Our point of departure is to verify if these negative results can be changed if we switch to weighted tensor products with product weights. Indeed, this is the case. We now illustrate a sample of our results leaving the general results to Section 3.

For weighted tensor product problems, the information complexity depends on two non-increasing sequences, and . Here, the ’s are the squares of the ordered singular values of the univariate operator, and the ’s are product weights which moderate the importance of the univariate problem in the definition of the tensor product for the multivariate case. Without loss of generality we may assume that and, to omit the trivial case, that . To make the presentation of our results easier we also assume that all weights are positive. The un-weighted case is obtained if we take for all .

The concept of exponential strong polynomial tractability (EXP-SPT) is defined when there are two non-negative numbers and such that the information complexity is bounded by

Similarly, the concept of exponential polynomial tractability (EXP-PT) is defined when there are three non-negative numbers , and such that the information complexity is bounded by

Note that especially EXP-SPT is a quite demanding property since the upper bound on the information complexity must be independent of and at most polynomially dependent on . For EXP-PT we allow that the information complexity may depend polynomially on and . One might suspect that EXP-SPT and also EXP-PT hold only for extremely small ’s and ’s. As we shall see, this is indeed the case.

We prove that

  • EXP-SPT and EXP-PT are equivalent,

  • EXP-SPT holds if and only if

    where

    Furthermore, the exponent of EXP-SPT, defined as the infimum of those

    for which the estimate on the information complexity for EXP-SPT holds, is equal to

    .

As we now see, the relaxation from EXP-SPT to EXP-PT is not essential. For the un-weighted case, , both EXP-SPT and EXP-PT do not hold.

Let us check for which ’s and ’s we have EXP-SPT (and EXP-PT) for the weighted case. Since goes to infinity as tends to zero, we see that must go to infinity slower than . It is easy to check that for for some (maybe very large) , we get . Consider thus , this time with (maybe very small) . Again, it is easy to check that now goes to zero. Obviously we cannot yet claim EXP-SPT since it also depends on ’s. By the same token we conclude that must go to infinity slower than . So is not enough. For positive , consider then

We now have

Then , so that EXP-SPT now holds with the zero exponent. Further examples of different notions of exponential tractability will be presented in Section 3.


Of course, we may say that EXP-SPT (or EXP-PT) is a too strong notion of exponential tractability. We now present a result for a much weaker notion, namely for exponential weak tractability (EXP-WT) which holds if the logarithm of the information complexity divided by goes to zero if goes to infinity. We prove that EXP-WT holds if and only if

The conditions on the ’s and the ’s are now much more lenient but still do not hold for the un-weighted case. For the weighted case, we obtain EXP-WT if for we have

and goes to zero arbitrarily slowly.

We also consider other notions of exponential tractability such as EXP-QPT, exponential quasi-polynomial tractability, and EXP--WT, exponential -weak tractability for . The corresponding necessary and sufficient conditions on these notions of exponential tractability are presented in Theorem 1. The case of exponential -weak tractability with as well as EXP-UWT, exponential uniform weak tractability, are left for future research.


We end the introduction by presenting a couple of other open problems.

  • In this paper, we assume the class of all continuous linear functionals as information evaluations. It is of a practical interest to consider the class of only function values. In this case we assume that is a reproducing kernel Hilbert space so that function values are continuous linear functionals. The open problem is to find necessary and sufficient conditions for various notions of exponential tractability for the class , and to compare them to necessary and sufficient conditions for the class . We believe that the recent paper [9] may be very helpful for the solution of this problem.

  • As we already mentioned, we consider in this paper only product weights. It would be of interest to study more general weights and to see how the conditions for product weights can be changed.

We summarize the contents of the rest of this paper. In Section 2 we define the problem we study here. In Section 3 we present the results, and in Section 4 the proofs.

2 Problem Setting

We outline the formal setting considered in this paper. Let be a separable infinite-dimensional Hilbert space with inner product denoted by . Let be an arbitrary Hilbert space, and let be a compact linear operator. We stress that compactness of is a necessary condition to get a finite information complexity and any type of algebraic or exponential tractability. Then

is also a compact and self-adjoint non-negative operator. Let denote its eigenpair,

where the ’s are orthonormal and the ’s non-increasing. Without loss of generality we assume that and, to omit the trivial problem, that . Due to compactness of we have .

Let . Define

to be the -fold tensor product of . The inner product is denoted by . Similarly, define

and the -fold tensor product operator

Obviously, is compact. Then

is also a compact and self-adjoint non-negative operator. Let . The eigenpairs of are with

Hence, we have at least

positive eigenvalues of

. The square roots of the are the singular values of .

In the following we write . Subsets of will be denoted by . From [17, Sec. 5.3.1], elements can be decomposed as a sum of mutually orthogonal elements , , each of which belongs to , in the form

Furthermore, for the inner product is

and

(1)

Further information on this orthogonal decomposition can be found in [17, Sec. 5.3.1].

Eq. (1) shows that the contribution of each is the same, which suggests that any group of ’s is equally important in their contribution to the norm of . However, in this paper we are interested in the weighted setting which is motivated by the assumption that some groups of ’s are more important than others, or that an element does not depend on some groups of variables at all. Such a behavior can be modeled with the help of so-called weights. Here we restrict ourselves to product weights as in the first paper on weighted spaces [26].

Let be a sequence of non-increasing positive reals, which are called product weights. The case when some product weights ’s are zero is considered in Remark 4. For simplicity we assume that . For put

where the empty product is considered to be one, i.e., .

Now we define the weighted Hilbert space as a separable Hilbert space that is algebraically the same as the space but whose inner product for is given by

(2)

For product weights, the space can also be described as a tensor product space, namely,

In particular, if for all , then we have .

We study the sequence of operators given by

and consider the problem of approximating in the norm of for elements from the unit ball of . We shall show in the next section how the weights affect the singular values of .

The elements are approximated by algorithms which use at most information evaluations from the class which consists of all continuous linear functionals defined on . The general form of an algorithm is

where and is any mapping. The choice of can be adaptive, i.e., it may depend on the previously computed information .

The error is studied in the worst-case setting and is defined as

Denote by the initial error, i.e.,

which is just the operator norm of .

We are interested in the minimal number of information evaluations from the class in order to reduce the initial error by a factor of . To this end let

be the minimal error, where the infimum is extended over all algorithms which use at most information evaluations from the class . Then we study the information complexity for the normalized error criterion, which is defined by

It is known that for the class the information complexity is fully characterized in terms of the singular values of , or equivalently, in terms of the eigenvalues of .

We are interested in the behavior of when goes to infinity in an arbitrary way. This is the subject of tractability, see [17, 18, 19]

. The notions of tractability classify the order of growth of the information complexity. Standard tractability is studied in the

algebraic setting (ALG), as it is called nowadays. In this case, one describes the dependence of on the dimension and the error threshold, i.e., with respect to the pair . Recently also the exponential setting (EXP) gained much attention, and this setting is the central topic of the present paper. In the exponential setting, we study how behaves with respect to the pair . We are ready to define various notions of EXP tractabilities.

The problem is said to be:

  • Exponentially strongly polynomially tractable (EXP-SPT) if there are such that

    (3)

    The infimum over all exponents such that (3) holds for some is called the exponent of EXP-SPT and is denoted by .

  • Exponentially polynomially tractable (EXP-PT) if there are such that

  • Exponentially quasi-polynomially tractable (EXP-QPT) if there are such that

    (4)

    The infimum over all exponents such that (4) holds for some is called the exponent of EXP-QPT and is denoted by .

  • Exponentially -weakly tractable (EXP--WT) for positive and if

    If , we speak of exponential weak tractability (EXP-WT).

  • Exponentially uniformly weakly tractable (EXP-UWT) if EXP--WT holds for all positive and .

To shorten the notation, we often say that the problem is EXP-SPT, EXP-PT, etc., by saying that EXP-SPT, EXP-PT, etc., holds. As already mentioned, we do not consider EXP--WT with and exponential uniform tractability in this paper.

Remark 1.

In some papers, for example in [22], the notion of EXP--WT is called -weak tractability, where corresponds to and corresponds to in our notation.

The notions of quasi-polynomial, -weak and uniform weak tractabilities in the algebraic case were for the first time defined correspondingly in [3, 24, 25]. Here we adopt these concepts for exponential tractability by replacing by .


The main result of this work, a characterization of weighted linear tensor product problems with respect to exponential tractability, will be stated in the next section as Theorem 1. The proofs will be presented in Section 4.

3 The results

To begin with we introduce another, for our purpose more convenient, representation of the information complexity. It is known from [29], see also [17], how the information complexity depends on the singular values of , which are the same as the square-roots of the eigenvalues of the compact self-adjoint and positive definite linear operator .

Let and be as in the previous section. For define

From [17, Section 5.3]) we know that the eigenvalues of are

Clearly, is maximized for and then it is equal to . Hence, and the initial error is also one. This means that the problem is well normalized for all and all product weights .

The information complexity is now

(5)

Define

Then

and hence

(6)

Clearly,

Hence, for decreasing and increasing , the information complexity is non-increasing.

In the sequel we will work with the representation of the information complexity in (6). We show how weighted tensor product problems can be classified with respect to different notions of EXP tractability by means of the eigenvalues of the operator and of the weights . We remind the reader of what we assume about the ’s and ’s. We have

(7)

Note that for the problem becomes trivial since for all and . On the other hand, if then the problem is not well normalized. In this case, the initial error is and for the normalized error criterion we may work with instead of . By assuming that we simplify the notation. Note also that iff (as well as ) is compact. This assumption implies that the information complexity is finite for all and all .

For and , define

(8)

Then is well defined and always finite. Since , we have . Furthermore, goes to infinity if and only if all ’s are positive.

We also assume that the weights satisfy

(9)

The ordering of the ’s tells us that the successive subproblems are less and less important. The assumption that the weights are at most one is made for simplicity to guarantee that . The case of more general ’s and ’s is considered for algebraic tractability in [17, Section 5.3].

For and , define

Then is well defined. We put for , and note that for . Both  and are non-decreasing, and .

Now we are able to state our main result. To shorten the notation we write ”iff” instead of “if and only if”.

Theorem 1.

We have

  1. EXP-SPT holds iff

    If this holds then the exponent of EXP-SPT is .

  2. EXP-SPT and EXP-PT are equivalent.

  3. EXP-QPT holds iff

    If this holds then the exponent of EXP-QPT is .

  4. Let .

    EXP-WT holds iff

  5. Let and .

    EXP--WT holds iff

  6. Let and .

    EXP--WT holds iff

  7. Let , and .

    EXP--WT holds iff

  8. Let , and .

    EXP--WT holds iff

  9. Let and .

    EXP--WT holds iff

  10. Let and . EXP--WT holds for arbitrary ’s iff

  11. Let and .

    EXP--WT holds iff for arbitrary integers with and it is true that

    (10)

Before we present the proof of Theorem 1 we illustrate some of the results and discuss their meaning.

Example 1.

Let and for positive and . Then we have

and hence

  • and imply EXP-SPT with ;

  • and imply EXP-QPT with , but EXP-SPT does not hold.

Note that Items 4.-11. of Theorem 1 give a full characterization of EXP--WT for all . The following remarks are in order.

Remark 2.

The condition

is satisfied if and only if is of the form

where satisfies . So, for example, we have EXP-WT if and , and for .

Remark 3.

Consider , and described in Items 7. and 8. of Theorem 1:

For , we have a single largest eigenvalue and EXP--WT holds for arbitrary ’s as long as . In particular, this holds for the un-weighted case, for all .

For , we have a multiple largest eigenvalue and EXP--WT holds under the same conditions on the ’s but now we need to assume that not all ’s are one. In particular, this holds for

Remark 4.

Consider and described in Items 6. and 9. of Theorem 1. Then EXP--WT holds for arbitrary ’s, i.e., even for the un-weighted case , and for satisfying the same condition as before. This case was proved in [4].

Remark 5.

We briefly note what happens if some weights in (9) are zero, say for some . Obviously, monotonicity of the ’s implies that for all . Then for all containing one or more indices at least equal to . For such , we must assume in (2) that and adopt the convention that . In this case, is algebraically a proper subset of .

Assume first that . Then the only non-zero eigenvalues are . This means that the problem is trivial since for all and .

Assume then that . It is easy to check that we now have

Hence, and for . The factors and cannot change the fact that or are finite, and then EXP-SPT, EXP-PT and EXP-QPT are equivalent.

4 The proofs

We first show how the information complexity can be bounded in terms of and .

Lemma 1.

If then for we have

and for

Proof.

We use (6). Consider the eigenvalue .

  • If (in particular ) for some , then we have

    so that .

  • If and , then

    and again .

Hence, only

for with may belong to , and therefore

Furthermore for , we have

as claimed.

In order to show the remaining inequality we consider the eigenvalues

for .

For these eigenvalues we have

This implies that we have at least eigenvalues no less than . Hence

This completes the proof.


The next technical lemma will help to state the conditions for various notions of exponential tractability in a concise form.

Lemma 2.

Let be a non-increasing sequence of positive reals. Then we have

(11)

if and only if

(12)
Proof.

Assume that (11) holds. We have

and hence

Taking the logarithm we obtain

and therefore

Now (12) follows by letting .

If (12) holds then for every there exists a number such that

This implies