Behavior of the entropy numbers of classes of multivariate functions with mixed smoothness is studied here. This problem has a long history and some fundamental problems in the area are still open. The main goal of this paper is to develop a new method of proving the upper bounds for the entropy numbers. This method is based on recent developments of nonlinear approximation, in particular, on greedy approximation. This method consists of the following two steps strategy. At the first step we obtain bounds of the best -term approximations with respect to a dictionary. At the second step we use general inequalities relating the entropy numbers to the best -term approximations. For the lower bounds we use the volume estimates method, which is a well known powerful method for proving the lower bounds for the entropy numbers. It was used in a number of previous papers. Taking into account the fact that there are fundamental open problems in the area, we give a detailed discussion of known results and of open problems. We also provide some comments on the techniques, which were used to obtain known results. Then we formulate our new results and compare them to the known results.
Let be a Banach space and let denote the unit ball of with the center at . Denote by a ball with center and radius : . For a compact set and a positive number we define the covering number as follows
It is convenient to consider along with the entropy the entropy numbers :
Let be the univariate Bernoulli kernels
For and we define
where means convolution. In the univariate case we use the notation .
It is well known that in the univariate case
holds for all and . We note that condition is a necessary and sufficient condition for compact embedding of into . Thus (1.1) provides a complete description of the rate of in the univariate case. We point out that (1.1) shows that the rate of decay of depends only on and does not depend on and . In this sense the strongest upper bound (for ) is and the strongest lower bound is .
There are different generalizations of classes to the case of multivariate functions. In this section we only discuss known results for classes of functions with bounded mixed derivative. For further discussions see , Chapter 3 and .
For and one has
For and one has
It is known in approximation theory (see ) that investigation of asymptotic characteristics of classes in becomes more difficult when or takes value or than when . It turns out to be the case for too. It was discovered that in some of these extreme cases ( or equals or ) relation (1.2) holds and in other cases it does not hold. We describe the picture in detail. It was proved in  that (1.2) holds for , , . It was also proved that (1.2) holds for , (see  for and  for ). Summarizing, we state that (1.2) holds for and , for all (with appropriate restrictions on ). This easily implies that (1.2) also holds for , . For all other pairs , namely, for , and , the rate of is not known in the case . It is an outstanding open problem.
In the case this problem is essentially solved. We now cite the corresponding results. The first result on the right order of in the case was obtained by Kuelbs and Li  for , . It was proved in  that
holds for , . We note that the upper bound in (1.3) was proved under condition and the lower bound in (1.3) was proved under condition . Belinskii  proved the upper bound in (1.3) for under condition . Relation (1.3) for under assumption was proved in .
The case , was settled by Kashin and Temlyakov . The authors proved that
holds for , and
Let us make an observation on the base of the above discussion. In the univariate case the entropy numbers have the same order of decay with respect to for all pairs , . In the case we have three different orders of decay of which depend on the pair . For instance, in the case it is , in the case , , it is and in the case , it is .
We discussed above results on the right order of decay of the entropy numbers. Clearly, each order relation is a combination of the upper bound and the matching lower bound . We now briefly discuss methods that were used for proving upper and lower bounds. The upper bounds in Theorem 1.1 were proved by the standard method of reduction by discretization to estimates of the entropy numbers of finite-dimensional sets. Here results of ,  or  are applied. It is clear from the above discussion that it was sufficient to prove the lower bound in (1.2) in the case . The proof of this lower bound (see Theorem 1.2) is more difficult and is based on nontrivial estimates of the volumes of the sets of Fourier coefficients of bounded trigonometric polynomials. Theorem 2.4 (see below) plays a key role in this method.
That proof is based on Theorem 2.2 (see below).
Kuelbs and Li  discovered the fact that there is a tight relationship between small ball problem and the behavior of the entropy . Based on results obtained by Livshits and Tsirelson , by Bass , and by Talagrand  for the small ball problem, they proved
Proof of the most difficult part of (1.7) – the lower bound – is based on a special inequality, known now as the Small Ball Inequality, for the Haar polynomials proved by Talagrand  (see  for a simple proof).
We discussed above known results on the rate of decay of . In the case the picture is almost complete. In the case the situation is fundamentally different. The problem of the right order of decay of is still open for , and , . In particular, it is open in the case , , that is related to the small ball problem. We discuss in more detail the case , . We pointed out above that in the case the proof of lower bounds (the most difficult part) was based on the Small Ball Inequalities for the Haar system for and for the trigonometric system for all . The existing conjecture is that
for large enough . The upper bound in (1.8) follows from (1.6). It is known that the corresponding lower bound in (1.8) would follow from the -dimensional version of the Small Ball Inequality for the trigonometric system.
The main goal of this paper is to develop new techniques for proving upper bounds for the entropy numbers. We consider here slightly more general classes than classes . Let
be a vector with nonnegative integer coordinates () and
where denotes the integer part of a number . Define for
Consider the class (see )
It is well known that the class is embedded in the class for . Classes provide control of smoothness at two scales: controls the power type smoothness and controls the logarithmic scale smoothness. Similar classes with the power and logarithmic scales of smoothness are studied in the recent book of Triebel . Here is one more class, which is equivalent to in the case (see ). Consider a class , which consists of functions with a representation (see Subsection 2.2 below for the definition of )
In the case classes are wider than .
The main results of the paper are the following theorems in the case for the extreme values of and . First, we formulate two theorems for the case .
Let and . Then for
Let and . Then
Second, we formulate three theorems for the case .
We have for all
We have for , ,
We have for all , ,
Let us make some comments on Theorem 1.3. As we already mentioned above classes are close to classes but they are different. We show that they are different even in the sense of asymptotic behavior of their entropy numbers. We point out that the right order of is not known for . We confine ourselves to the case . It is proved in  that for
Theorem 1.3 gives for
Relation (1.16) is for the case . The general case of is also known in this case (see (1.2) and its discussion above and also see Section 3.6 of  for the corresponding results and historical comments). Relations (1.15) and (1.16) show that in the sense of entropy numbers the class behaves as a limiting case of classes when .
The proof of upper bounds in Theorems 1.3 and 1.4 is based on greedy approximation technique. It is a new and powerful technique. In particular, Theorem 1.4 gives the same upper bound as in (1.6) for the class , which is wider than any of the classes , , from (1.6). In Section 7 we develop mentioned above new technique, which is based on nonlinear -term approximations, to prove the following result.
Let and . Then
Theorem 1.6 discovers an interesting new phenomenon. Comparing (1.12) with (1.13), we see that the entropy numbers of the class in the space have different rate of decay in cases and . We note that in the proof of the upper bounds in this new phenomenon we use the Riesz products for the hyperbolic crosses. This technique works well in the case but we do not know how to extend it to the general case . This difficulty is of the same nature as the corresponding difficulty in generalizing the Small Ball Inequality from to (see , Ch. 3, for further discussion). We already mentioned above that in studying the entropy numbers of function classes the discretization technique is useful. Classically, the Marcinkiewicz theorem serves as a powerful tool for discretizing the -norm of a trigonometric polynomial. It works well in the multivariate case for trigonometric polynomials with frequencies from a parallelepiped. However, there is no analog of Marcinkiewicz’ theorem for hyperbolic cross polynomials (see  and , Section 2.5, for a discussion). Thus, in Sections 5–7 we develop a new technique for estimating the entropy numbers of the unit balls of the hyperbolic cross polynomials. The most interesting results are obtained in the dimension . It would be very interesting to extend these results to the case . It is a challenging open problem.
2 Known results
2.1 General inequalities
For the reader’s convenience we collect in this section known results, which will be used in this paper. The reader can find results of this subsection, except Theorem 2.3, and their proofs in , Chapter 3.
Let , and let be a subspace of . Then
Let us consider the space equipped with different norms, say, norms and . For a Lebesgue measurable set we denote its Lebesgue measure by .
For any two norms and and any we have
Let us formulate one immediate corollary of Theorem 2.1.
For any -dimensional real Banach space we have
Let denote the norm and let be a unit ball in . Denote the boundary of . We define by the normalized -dimensional measure on . Consider another norm on and denote by the equipped with .
Let be equipped with and
Then we have
The following Nikol’skii-type inequalities are known (see , Chapter 1, Section 2).
Let . For any (see Subsection 2.2 below for the definition of ) we have
2.2 Volume estimates
Denote for a natural number
with for . We call a set hyperbolic layer. For a set denote
For a finite set we assign to each a vector
where denotes the cardinality of and define
For any we have
with constants in that may depend only on .
We note that the most difficult part of Theorem 2.4 is the lower estimate for . The corresponding estimate was proved in the case in  and in the general case in  and  by a method different from the one in . The upper estimate for in Theorem 2.4 can be easily reduced to the volume estimate for an octahedron (see, for instance ). In the case Theorem 2.4 is a direct corollary of the well known estimates of the volume of the Euclidean unit ball.
For any finite set and any we have
The following result was obtained in .
Let have the form , is a finite set. Then for any we have
In particular, Theorem 2.6 implies for and that
The following result was obtained in . Denote .
In the case we have
The following lemma from  is an important ingredient of analysis in this paper. For the reader’s convenience we give a proof of this lemma here.
Let and . Then
We use the following result of E. Gluskin .
Let , , and
Consider the following lattice on the :
It is clear that . It is well known (see , Ch.2, Theorem 2.4) that for any one has
Thus, for any we have
We associate with each point two vectors and from :
It is clear that the condition is satisfied if
Then and by Theorem 2.8
Using that the condition
is equivalent to the condition
This completes the proof of Lemma 2.1 ∎
3 New lower bounds. The volumes technique
Proof of lower bounds in Theorems 1.3 and 1.7. The lower bound in Theorem 1.3 follows from the lower bound in Theorem 1.7 with . We prove the lower bounds for the with and any . This lower bound is derived from the well known simple inequality (see Corollary 2.1 above)
for any -dimensional real Banach space . Consider as a Banach space the with norm. Clearly, it can be seen as a -dimensional real Banach space with . It follows from the definition of that
Take . Then (3.1) implies that
Proof of lower bounds in Theorem 1.4. We prove the lower bound for . This proof is somewhat similar to the proof of lower bounds in Theorem 1.3. Instead of (3.1) we now use the inequality (see Theorem 2.1 above)
with and . It follows from the definition of that
The lower bounds in Theorem 1.4 are proved.
The lower bounds in Theorem 1.5 are proved.
4 Upper bounds. A general scheme
From finite dimensional to infinite dimensional. Let and be two Banach spaces. We discuss a problem of estimating the entropy numbers of an approximation class, defined in the space , in the norm of the space . Suppose a sequence of finite dimensional subspaces , , is given. Define the following class
Denote and assume that for the unit balls we have the following upper bounds for the entropy numbers: there exist real and nonnegative and such that