On the Search for Tight Frames of Low Coherence

02/15/2020 ∙ by Xuemei Chen, et al. ∙ New Mexico State University Vanderbilt University 0

We introduce a projective Riesz s-kernel for the unit sphere S^d-1 and investigate properties of N-point energy minimizing configurations for such a kernel. We show that these configurations, for s and N sufficiently large, form frames that are well-separated (have low coherence) and are nearly tight. Our results suggest an algorithm for computing well-separated tight frames which is illustrated with numerical examples.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

A set of vectors

is a frame111Depending on the context, we either consider to be a multiset, allowing for repetition, or as an ordered list. for a separable Hilbert space if there exist such that for every ,

The constant (, resp.) is called the lower (upper, resp.) frame bound. When , is called a tight frame, which generalizes the concept of an orthonormal basis in the sense that the recovery formula holds for every . For the finite dimensional space , where or , is a frame of if and only if spans . We shall also use to denote the matrix whose th column is and therefore we have

where is the identity matrix.

The notion of frames was introduced by Duffin and Schaeffer [35]. Since the work of [33] by Daubechies et al., there has been a significant amount of work on the theory and application of frames for signal processing, tomography, biomedical imaging [61], X-ray crystallography via phaseless reconstruction [2], and compressed sensing [20, 27, 52]. Tight frames are preferred in many of these applications because they give stable signal representations especially in noisy environments, and allow fast reconstruction and convergence.

Let be the collection of all -point configurations on , the unit sphere of , where denotes the norm. If we have a unit norm tight frame , then it is well known that the frame bound has to be since

(1.1)

Benedetto and Fickus [3]

have classified unit norm tight frames as minimizers of a certain energy. Given a frame

, its frame potential energy is defined as . It is shown in [3] that frames that attain

(1.2)

are precisely the unit norm tight frames. We will call the function the frame potential kernel. Ehler and Okoudjou [37] generalized this result to the -frame potential kernel , see also [6] for recent results on -frames.

Among tight frames, some may be more desirable than others. For example, let , where , and let , where . Both and are tight, and hence minimizers of (1.2). However, as a frame, is less desirable because it is the concatenation of an orthonormal basis and its negative copy. Indeed, designing frames is not about distributing points on a sphere, but rather about distributing lines in space. The question arises whether we can design a reliable scheme that generates tight frames that are better separated, like . For this purpose, we must first define what we mean by separation. Separation is quantified by the coherence of a frame ; that is,

The smaller the coherence, the better separated the frame is.

A straightforward method to find well-separated frames is to solve

(1.3)

which has been studied in several works including Welch [67], Conway et al. [31], Strohmer and Heath [62], and more recently [40, 9]. The problem (1.3) is often referred as the best line-packing problem because it asks how to arrange lines in so that they are as far apart as possible. Conway et al [31] made extensive computations on this problem from a more general perspective: how to best pack -dimensional subspaces in ? There are also many other contributions using tools in geometry and combinatorics [34]. A minimizer of (1.3) is called a Grassmannian frame by [62] and we shall use this terminology as well.

Well separated tight frames are desirable in many applications including quantum physics [68] and the design of spreading sequences for CDMA [50]. Recently, it has been argued that such frames exhibit faster convergence in the randomized Kaczmarz algorithm [63] for solving a linear system where the rows of form a frame [25, 32]. We will list two more applications in detail below:

Robustness to erasure. Frames are used in signal representation for several reasons including resilience to additive noise, resilience to quantization and erasure. Consider, for example the following communication scheme:

where coefficients are erased during transmission. It is shown in [43, 51] that a unit norm tight frame is 1-erasure (one arbitrary frame coefficient is erased) optimal in terms of reconstruction error. Furthermore, 2-erasure optimal frames are those having the smallest coherence among all unit norm tight frames.

Measurement matrix for compressed sensing. Compressed sensing involves solving an underdetermined system given that the solution has only nonzero entries. It has gained much attention in the recent decade for its application in imaging and data analysis in general, and for its connections to many other branches of mathematics. The effectiveness of many compressed sensing algorithms calls for low coherence of the measurement matrix  [65, 8, 14] so that any columns of behave like a partial isometry. We demonstrate in Section 7.4 numerically that optimal configurations arising from minimizing certain energies appear to be good sensing matrices. See the work [66] for related results. More applications can be found in [40, Section 1.2].

In this paper we study certain minimal energy problems on the projective space , where consists of all lines in through the origin; namely, sets of the form

(1.4)

for some . We endow with the metric

(1.5)

which is the ‘chordal’ distance.222The chordal distance between the lines and is given by . Note that is well-defined since is independent of the choice of representatives of the two lines. This suggests the use of energy methods for a kernel on of the form:

(1.6)

which can equivalently be regarded as a kernel on The energy of with respect to the kernel is given by

(1.7)

One seeks the infimum of (1.7) over all possible point configurations on . Assuming is lower semi-continuous on so the infimum is attained, we define the -point minimal energy of kernel as

(1.8)

An -point configuration that achieves the minimum (1.8) will be denoted by (or when there is no ambiguity). So far the minimal energy and optimal configuration have been confined to the sphere and generalizes to any compact set , and will be denoted as respectively. Note that the frame potential is of the form (1.6) and that the energy minimizers are precisely the unit norm tight frames. However, in general for , these minimizers may not consist of well-separated lines. Indeed, as the previous example of shows, a minimizing configuration of four lines may collapse to two lines with coherence equal to one.

To achieve well separation of lines our approach is to first consider a class of kernels that are more strongly repulsive and analyze the approximate tightness of their energy minimizers relative to their frame potential energy. Specifically, we introduce the Riesz projective -kernel

(1.9)

for and seek solutions to the problem

(1.10)

The kernel is a modification of the classical Riesz -kernel defined for in a normed linear space as

(1.11)

In fact, as we will show in (3.6), the projective Riesz -kernel can also be represented in terms of for an appropriate subspace of matrices with the Frobenius norm.

Notice that minimizers of (1.10) will avoid antipodal points since the energy in that case would be infinite. The connection between projective Riesz -kernels and Riesz -kernels is more immediate in the real case ; since , we have

Thus the projective Riesz kernel is just the Riesz kernel with the multiplicative factor to account for antipodal points.

A major focus of this paper is to exploit connections between and and reduce solving the projective Riesz -kernel minimization problem (1.10) to solving

(1.12)

where we take to be the projective space, but embedded in a higher dimensional real vector space (see Section 3.2). There are well established theorems available in the minimal energy literature for Riesz -kernels (see e.g. [13]) and we shall review some of them in Section 2.

The projective Riesz -kernel for defined by is also interesting. For such we will be solving

(1.13)

This coincides with (1.2) when . This paper shall focus on the case in the analysis, but our numerical experiments will include optimal configurations of (1.13).

The contributions of this paper are two-fold:

I. Minimal energy results for the projective Riesz kernel: We list both continuous and discrete results regarding solving (1.10) in Sections 4 and 5. The continuous result Theorem 4.1 determines the equilibrium measure for the projective Riesz -kernel on . The discrete results are for a type of kernel more general than the projective Riesz kernels. Theorem 5.1 states that on , the equally spaced points on the projective space is the optimal configuration. Theorem 5.4 states that equiangular tight frames are optimal configuration whenever they exist. These minimal energy results are of independent interest and can provide means for constructing well-separated antipodal points on the sphere.

II. Construction of nearly tight and well-separated frames: We justify that projective Riesz minimizing frames, i.e., frames as minimizers of (1.10) are well-separated in the sense that its coherence has asymptotic optimal order. This is stated in Theorem 6.3. Theorem 6.4

states that projective Riesz minimizing frames are nearly tight. Finally, we provide a heuristic way to obtain well-separated and exactly tight frames in Section

7.3.

For the rest of the paper, Section 2 states some necessary background and notation on both discrete and continuous minimal energy problems, especially the ones related to the classical Riesz kernel. Section 3 explains the main technique, which is to convert (1.10) to minimal energy problems over the Riesz kernel. Sections 4, 5, and 6 contain the main results. Numerical experiments are provided in Section 7.

2. Minimal Energy Background

In this section we will introduce some necessary background on minimizing discrete energy and its relation to the continuous energy.

The discrete minimal energy problem is known to be challenging, and we have very limited knowledge about the optimal configuration even for the classical Riesz kernel case (1.12) on the 2-dimensional sphere. The following theorem settles the case when points are on a circle of a real vector space for a large class of kernels that includes Riesz kernels.

Theorem 2.1 (Fejes-Tóth).

If and is a non-increasing convex function defined at 0 by the (possibly infinite) value , then any equally spaced points on a circle of radius (in ) minimizes the discrete energy for the kernel . If in addition, is strictly convex, then no other -point configuration on this circle is optimal.

The proof of Theorem 2.1 is a standard “winding number argument” that can be traced back to the work of Fejes-Tóth [39].

We know very little of the optimal configurations of (1.12) beyond . For , the minimal Riesz energy configuration for is given by two antipodal points, for by the vertices of an equilateral triangle that lie on an equator, and for by the vertices of a regular tetrahedron inscribed in . But we are still not able to rigorously prove what is the optimal configuration for all Riesz kernels for . Numerical experiments suggest that the optimal configuration is either the bipyramid (North pole, South pole, and equilateral triangle on the equator), or a square-base pyramid. The latest work on the 5-point problem, by Schwartz [60], shows in over 150 pages, computer assisted, that the bipyramid is optimal for all up to the “magic number” which is approximately 15.04. The problem is still open for greater than this magic number plus a small constant. For , the optimal configuration is the octahedral vertices , where is an orthonormal basis of . The work [29] shows that is the optimal configuration for a wide range of kernels that include the Riesz kernel. Much less is known for for unless , in which case the simplex is the optimal configuration. On the other hand, many asymptotic results (as ) for optimal configurations on the sphere as well as on are known (for examples of recent results, see [5], [47]).

For a set of points , the separation distance of is defined as

The best-packing problem is to find the -point configuration on that maximizes the separation distance:

(2.1)

For , one trivially has . It is immediate, for example, that the best -point packing of consists of equally spaced points on the circle.

When , the minimization problem with respect to the Riesz kernel

turns into the best-packing problem (2.1); more precisely,

Theorem 2.2 ([13]).

If and is a compact set of cardinality at least , then

where is the Riesz kernel defined in (1.11). Furthermore, if is an optimal configuration that achieves , then every cluster point as of the set on is an -point best-packing configuration on .

This discrete minimal energy problem is related to the continuous one as we next describe. Let

be the set of probability measures supported on

. For a general kernel , the potential function of a measure with respect to is defined as

provided the integral exists as an extended real number. The energy of is defined as

and the Wiener constant is

(2.2)

Likewise this infimum can be achieved, and the probability measure that optimizes the above problem is called the -equilibrium measure. The -capacity of the set is defined by

A set has zero capacity means that , which makes the problem (2.2) trivial since every probabilistic measure generates energy.

We now present a classical theorem connecting the discrete minimal energy problem to the continuous one. Before that we introduce the weak* limit of measures. A sequence of measures converges weak* to if for every continuous function on ,

We also define to be the point mass probability measure on the point . Moreover, given a finite collection of points , its normalized counting measure is defined as

Theorem 2.3 ([28], [13]).

If is a kernel on , where is an infinite compact set, then

(2.3)

Moreover, every weak* limit measure (as ) of the sequence of normalized counting measures is a -equilibrium measure.

The proof of Theorem 2.3 for the case of a Riesz kernel can also be found in the book by Landkof [55, Eq. (2.3.4)].

We now review two important facts concerning Riesz kernels.

Theorem 2.4 ([59], [12], [13]).

Let be a compact infinite subset of an -dimensional -manifold with of positive -dimensional Hausdorff measure.

  1. If , then the -equilibrium measure on is unique. Moreover, if the potential function is constant on , then is the -equilibrium measure on .

  2. If , then has -capacity zero. Moreover, if denotes an -energy optimal -point configuration for , then the sequence of normalized counting measures converges to the uniform measure (normalized Hausdorff measure) on in the weak* sense as (this is a special case of the so-called Poppy-seed bagel theorem).

3. An overview of the problem on the sphere

3.1. Projectively equivalent configurations.

Note that for a kernel of the form (1.6), the energy , , is invariant under any of the following operations on :

(3.1)

Any configuration obtained from by applying these operations is said to be projectively equivalent to . For example, is projectively equivalent to .

Theorem 5.1 below states that the configuration of equally spaced points on the half-circle,

(3.2)

is optimal for (1.8) for a certain class of kernels . For , three equivalent optimal configurations are shown in Figure 1.

Figure 1. Optimal configurations for the unit circle with kernels given in Theorem 5.1. Left is . Middle is a rotation of the left. Right is a sign change of the middle.

3.2. From sphere to the projective space

The projective space can be embedded isometrically into the space of Hermitian matrices, denoted by as we next describe. Note that is a real vector space for both and with inner product in defined as . This inner product induces the Frobenius norm on . We further note that with the Frobenius norm can be identified with the Euclidean space where when and when (e.g., when and we take any ordering of the numbers for and for ).

Recalling (1.4), we define as with and . Clearly, is well defined (i.e., independent of the choice of the representative of the line). We denote the range of by ; that is, where .

For , the following well known equality (see, e.g. [30]) establishes that is an isometry:

(3.3)

It is used, for example, in works on phase retrieval, see e.g. [21, 44]. For the reader’s convenience we give the following derivation of the middle equality in (3.3) using the cyclic property of the trace:

(3.4)

from which we also get

(3.5)

Note that (3.3) shows that is an isometric embedding of in , and so we identify with . We remark that is a real analytic manifold whose dimension is in the case and in the case (see [9] or [56]).

Now we are able to consider a kernel of the form on as a kernel on . Specifically the projective Riesz -kernel (see (1.9)) can be reexpressed as

(3.6)

This allows us to reformulate the minimal projective energy problem in terms of the Riesz minimal energy problem on the set . This technique was also employed in [26]. We will apply the results presented in Section 2 on the continuous problem

(3.7)

and the discrete problem

(3.8)

in the next two sections.

Similarly, the Grassmannian problem (1.3) is equivalent to the best-packing problem (2.1) on the projective space, which is to maximize the smallest pairwise distance between all the lines (frame vectors). Let . For any point , we can find such that . By (3.3),

(3.9)

So

(3.10)

The last equality is from the definition (1.3).

For any Borel probability measure on the sphere, this embedding also induces the pushforward (probability) measure on . By definition of a pushforward measure,

(3.11)

We shall also write for .

To better understand , we further consider the symmetrization of a measure defined as

(3.12)

for Borel measurable

It is not difficult to show that if and only if the pullback measures and agree. The injectivity of then shows

(3.13)

Let be the uniform measure (normalized surface measure) on . Then , the pushforward measure of under , is the uniform measure on . In fact, is the Haar invariant measure induced by the unitary group (see [26, Section 4.2]).

4. Large behavior of optimal configurations

We first focus on the continuous problem

(4.1)

The results are of independent interest, and will be used in Section 6.

For future reference, we set

and

As previously discussed, the projective space embedded in is a smooth () compact manifold, so Theorem 2.4 applies with and .

Theorem 4.1.

For the projective Riesz kernel , the following properties hold.

  1. If , then is a -equilibrium measure on if and only if its symmetrized measure is the normalized surface measure on .

  2. If , then has -capacity 0.

  3. Let and let be a -optimal -point configuration on for . Then the sequence of normalized counting measures converges weak* to the uniform measure on as .

Proof.

(1) When , by (3.6) and the definition of a pushforward measure,

(4.2)

A similar equality holds for the log case : .

Thus for , the uniform measure produces a constant potential function with the kernel , so also produces a constant potential function with the Riesz kernel . By Theorem 2.4(1), must be the unique minimizer of (3.7).

On the other hand, similar to (4.2),

(4.3)

Again a similar equality holds for the log case. This implies that is a minimizer of (4.1) if and only if is a minimizer of (3.7), which has to be . By (3.13), this is equivalent to . This proves that is an equilibrium measure if and only if its symmetrized measure is .

(2) With the relation (4.3), this is a direct consequence of Theorem 2.4(2).

(3) For the discrete case, similar to (4.3), we have . So be a -optimal -point configuration on if and only if is an optimal configuration for the Riesz kernel on .

By Theorem 2.3, we conclude that the normalized counting measure converges to the -equilibrium measure on in the weak* sense. As shown in part (1), this unique equilibrium measure is , when . When , by Theorem 2.4(2), we also have converges to .

5. Discrete minimal energy problem

In this section we consider discrete extremal energy problems, for a general class of projective kernels of the form (1.6). Once again, the optimal configuration is an equivalent class in the sense of (3.1). Theorem 5.1 is for the 1-dimensional sphere in the real vector space while Theorem 5.4 is a general result over . Corollary 5.6 addresses the special projective Riesz kernel case (1.10).

Theorem 5.1.

If is a non-increasing convex function defined at zero by the (possibly infinite) value , then given in (3.2) is an optimal configuration on for the problem (1.8) where is as in (1.6). If, in addition, is strictly convex, then up to the equivalence relation in (3.1), no other -point configuration is optimal.

Proof.

By (3.3), , so we need to consider the minimal energy problem (1.8) with the kernel function to be on the compact set . The map is precisely

As mentioned at the beginning of Section 3.2, is identified with using the mapping . This way, is a circle in with radius .

With , the function satisfies the assumptions of Theorem 2.1, so

is minimized if are equally spaced on the circle . One can easily show that maps equally spaced points on half to equally spaced points on . So minimizers of (1.8) are precisely the equivalent class of equally spaced points on half of . ∎

Remark 5.2.

It is well known that is a compact Riemannian manifold. However, is topologically equivalent to a sphere only when .

Remark 5.3.

The frame potential kernel can be written as , where is not convex on . As a consequence, Theorem 5.1 cannot be applied to the frame potential kernel. The conclusion of Theorem 5.1 is however true, but there is no uniqueness (see [3]).

The discrete minimal energy problem for Riesz -kernel is in general very hard as mentioned previously. The situation is slightly better for kernels that are a function of absolute value of inner product, as we have the following general characterization when an equiangular tight frame (ETF) exists. A frame is equiangular if is a constant for all . An ETF is a frame that is equiangular and tight. For frames in , a necessary condition for the existence of ETF is for and for . The coherence has the famous Welch bound

(5.1)

and is achieved by ETFs. This can be easily derived from the relation (cf. [30])

(5.2)

The Welch bound also coincides with the simplex bound of the chordal distance in [31]. We refer interested readers to [64] for more details and [41] for a table on existing ETFs. The second inequality in (5.2) also shows that the frame potential is minimized when the frame is tight.

The second theorem is for both the real and complex case.

Theorem 5.4.

Let be a strictly convex and decreasing function defined at by the (possibly infinite) value , and be a strictly convex and decreasing function defined at by the (possibly infinite) value . If and are such that an ETF exists, then

  1. it is the unique optimal configuration of (1.8) for the kernel
    ;

  2. it is also the unique optimal configuration of (1.8) for the kernel .

Proof.

(i) From (3.3), . Let be an arbitrary configuration on the sphere and set . Then, by (3.5),