On the minimal ranks of matrix pencils and the existence of a best approximate block-term tensor decomposition

by   José Henrique de Morais Goulart, et al.

Under the action of the general linear group, the ranks of matrices A and B forming a m × n pencil A + λ B can change, but in a restricted manner. Specifically, to every pencil one can associate a pair of minimal ranks, which is unique up to a permutation. This notion can be defined for matrix pencils and, more generally, also for matrix polynomials of arbitrary degree. The natural hierarchy it induces in a pencil space is discussed. Then, a characterization of the minimal ranks of a pencil in terms of its Kronecker canonical form is provided. We classify the orbits according to their minimal ranks - under the action of the general linear group - in the case of real pencils with m, n < 4. By relying on this classification, we show that no real regular 4 × 4 pencil having only complex-valued eigenvalues admits a best approximation (in the norm topology) on the set of real pencils whose minimal ranks are bounded by 3. These non-approximable pencils form an open set, which is therefore of positive volume. Our results can be interpreted from a tensor viewpoint, where the minimal ranks of a degree-(d-1) matrix polynomial characterize the minimal ranks of matrices constituting a block-term decomposition of a m × n × d tensor into a sum of matrix-vector tensor products.



There are no comments yet.


page 1

page 2

page 3

page 4


Minimal obstructions for a matrix partition problem in chordal graphs

If M is an m × m matrix over { 0, 1, ∗}, an M-partition of a graph G is ...

On computing root polynomials and minimal bases of matrix pencils

We revisit the notion of root polynomials, thoroughly studied in [F. Dop...

Structured inversion of the Bernstein-Vandermonde Matrix

Bernstein polynomials, long a staple of approximation theory and computa...

Modelling hidden structure of signals in group data analysis with modified (Lr, 1) and block-term decompositions

This work is devoted to elaboration on the idea to use block term decomp...

Local Minimizers of the Crouzeix Ratio: A Nonsmooth Optimization Case Study

Given a square matrix A and a polynomial p, the Crouzeix ratio is the no...

Dictionary Learning and Tensor Decomposition via the Sum-of-Squares Method

We give a new approach to the dictionary learning (also known as "sparse...

On Some Properties of Calibrated Trifocal Tensors

In two-view geometry, the essential matrix describes the relative positi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The Kronecker-Weierstrass theory of matrix pencils provides a complete classification in terms of -orbits, which are equivalence classes under the action of :

where the pencil is expressed in homogeneous coordinates. Here, denotes a field, usually or . These orbits are represented by Kronecker canonical forms, which are characterized by unique minimal indices describing the singular part of the pencil along with elementary divisors associated with its regular part [7]. It follows then that these elementary divisors are -invariant.

This theory has been extended by Ja’Ja’ and Atkinson [10, 1], who have characterized the orbits of the larger group of tensor equivalence transformations , acting on pencils via


For simplicity of notation, we use the shorthands and . Ja’Ja’ [10] has shown that the Kronecker minimal indices of a pencil are -invariant, and so the singular part of a pencil is preserved by as well. However, the elementary divisors of its regular part are not. Nevertheless, their powers still remain the same, which motivates the terminology “invariant powers,” used by Ja’Ja’ [10]. Atkinson [1] went on to prove that, for an algebraically closed field , the equivalence classes of regular pencils are characterized by those powers and also by certain ratios which completely describe the elementary divisors. Specifically, recalling that over such a field all elementary divisors are powers of linear factors of the form

these ratios are defined as .

When viewed as a tensor, the (tensor) rank111This is not to be confused with the normal rank of the pencil , simply defined as [7, 8]. of a pencil is defined as the minimal number of rank-one matrices such that [9]. Equivalently, it is given by the minimal number such that one can find vectors , and satisfying

where denotes the canonical basis vector of . Under the action of , this expression is transformed into the tensor

from which it is visible that the tensor rank is -invariant. The rank of tensors can thus be studied by considering -orbits and associated representatives (see, e.g., the classification of -orbits undertaken by De Silva and Lim [6]).

One application of the study of -orbits is in algebraic complexity theory, since the tensor rank of quantifies the minimal number of multiplications needed to simultaneously evaluate a pair of bilinear forms and [9]. In the case where is algebraically closed, Ja’Ja’ [9] has derived results which allow determining the tensor rank of any pencil based on its Kronecker canonical form. Sumi et al. [14] have extended these results to pencils over any field .

Tensor equivalence transformations can also be employed to avoid so-called infinite elementary divisors of regular pencils [8]. These arise when matrix is singular (note that both and can be singular but still satisfy ). In non-homogeneous coordinates, the polynomial of an pencil has degree , and its characteristic polynomial is said to have infinite elementary divisors of combined degree . In this case, the tensor equivalence transformation can be employed for any such that , yielding a pencil having only finite elementary divisors, including some of the form induced by the infinite elementary divisors of . The existence of such an is guaranteed by definition, since is regular. In other words, every regular pencil is -equivalent to another pencil such that is nonsingular. In fact, it is always -equivalent to some with nonsingular matrices and .

On the other hand, not every regular pencil constituted by nonsingular matrices and is -equivalent to some other pencil such that either or are singular (or both). Take, for instance, and

No tensor equivalence transformation in of this pencil can yield such that either or is singular. Obviously, this property is -invariant. It turns out that each -orbit of a matrix pencil space can be classified on the basis of its associated minimal ranks and , with , which are such that every pencil with satisfies and . This notion of intrinsic complexity of a matrix pencil is complementary to its tensor rank, in the sense that pencils of same tensor rank do not necessarily have the same minimal ranks and vice-versa. For simplicity, we will compactly denote the minimal ranks of a pencil by .

It turns out that there is a direct connection between the minimal ranks of a pencil and the decomposition of its associated third-order tensor in block terms consisting of matrix-vector outer products, as introduced by De Lathauwer [4]. Namely, the components of are the minimal numbers and satisfying

where and forms a basis for . The theoretical properties of such block-term decompositions (henceforth abbreviated as BTD) of tensors are therefore related to properties of matrix pencils via this notion of minimal ranks.

In this paper, we will more generally define the minimal ranks of matrix polynomials , which include matrix pencils as a special case. This property of matrix polynomials is directly related to the BTD of tensors. In particular, similarly to the tensor rank, it induces a hierarchy of matrix polynomials, albeit a more involved one. We derive results which determine the minimal ranks of any matrix pencil in Kronecker canonical form. A classification of -orbits of real pencils in terms of their minimal ranks is then carried out for . On the basis of these results, we proceed to show that:

  1. The set of real pencils which are -equivalent to some with and is not closed in the norm topology for any positive integer .

  2. No real pencil having minimal ranks admits a best approximation in the set above described.

The first above result is analogous to the fact that a set of tensors having rank bounded by some number is generally not closed. Similarly, the second one parallels the fact that no element of certain sets of real rank- tensors admits a best approximation of a certain rank222For instance, no real tensor of rank admits a best rank- approximation. in the norm topology [13, 6]. This second result is of consequence to applications relying on the BTD, since the set of real pencils having minimal ranks is open in the norm topology, thus having positive volume. For complex-valued pencils, the results of [12] imply that such a non-existence phenomenon can only happen over sets of zero volume. We shall give a template of examples of (possibly complex) pencils having no best approximation on a given set of pencils with strictly lower minimal ranks.

It should be noted that the fact that a tensor might not admit an approximate BTD with a certain prescribed structure (referring to the number of blocks and their multilinear ranks [4]) is already known. Specifically, De Lathauwer [5] has provided an example relying on a construction similar to that of De Silva and Lim [6] concerning the case of low-rank tensor approximation. Our example given in Section 3.1 is in the same spirit. Nonetheless, to our knowledge ours is the first work showing the existence of a positive-volume set of tensors having no approximate BTD of a given structure, a phenomenon which is known to happen for low-rank tensor approximation [13, 6].

2 Minimal ranks of pencils

For brevity, we will henceforth express matrix pencils only in non-homogeneous coordinates.

2.1 Definition and basic results

Given its prominent role in what follows, the -orbit of a pencil deserves a special notation:

It will also be helpful to introduce the sets


Clearly, , and thus we shall assume that without loss of generality. For simplicity, when we can also write instead of . According to this convention and to definition (1), we have, for instance, . Furthermore, is by definition -invariant, because the relation defined as is reflexive and transitive, i.e., it defines an equivalence class. Hence:

Lemma 1.

if and only if .

As far as the question of whether is in for some is concerned, all that matters is the action of . Indeed, if are such that and , then333Note that we denote the identity of by or, when no ambiguity arises, simply by . satisfies and . We have shown the following.

Lemma 2.

if and only if there exists such that satisfies and . In other words, can be equivalently defined as the set of all pencils which are -equivalent to some satisfying and .

Let us now formally define the minimal ranks in terms of the introduced notation.

Definition 3.

Let be an pencil over . The minimal ranks of , denoted as , are defined as


where is any minimizer444Observe that such a minimizer is not unique. Besides the obvious family of minimizers of the form , there may be also multiple non-collinear minimizers. For instance, for the pencil , with , there are two non-collinear minimizers: and . of (2). We obviously have . When denoting a pencil as , we shall also use the notation .

The first thing to note is that is well-defined, i.e., its value is always the same regardless of the minimizer picked in the definition (3). For different collinear minimizers of (2), this is immediately clear. Now if two non-collinear minimizers and exist for (2), then must hold. It is also clear from (2) and (3) that the minimal ranks of a pencil are the ranks of matrices and of some pencil in the -orbit of . Indeed, taking and , where and are the minimizers of (2) and (3), respectively, then , with . Moreover, the minima in (2) and (3) are unchanged under a transformation from , implying that the value of is an invariant of action. Summarizing, we have:

Proposition 4.

Every satisfies . In particular, if satisfies , we say that attains the minimal ranks of .

From the above discussion, implies . However, the converse is not true. For instance, but . In general, if or for some (including the possibility ), then with . Conversely, only if and are proportional. We thus have the following result.

Lemma 5.

A pencil satisfies if and only if and are proportional. Furthermore, .

The terminology “minimal ranks” is motivated by the fact that, if and for some , then both and must hold. The definitions in (2) and (3) clearly imply for any pair and or any pair and . It remains to show that also for . Suppose on the contrary that with . This implies there exists such that

But then, implies , where is a minimizer of (2). This contradicts the definition of given by (3).

Proposition 6.

If and , then if and only if and .

We consider now some examples.

Example 7.

A regular pencil can only belong to if . Indeed, implies and are proportional, say , and for any . As a concrete example, is clearly in but not in any with .

Example 8.

Regular pencils can also be in for some . For example, the regular pencil555The dimensions of the zero blocks in that expression should be clear from the context. is in . Yet, the constraint must be satisfied. Indeed, means is -equivalent to some , with and . If , then clearly , implying neither nor is regular.

The next two examples underline how the elementary divisors of a regular pencil determine its minimal ranks. A general result establishing this connection will be presented ahead.

Example 9.

The pencil

with has minimal ranks in , because

for any . However, this is not true in , because the transformation yields , where

has rank 1. This difference comes from the fact that has a single elementary divisor over , namely , which cannot be factored into powers of linear forms since its roots are complex. In fact, is diagonalizable over , since it is similar to

From creftypecap 4, we have , and it is not hard to see that .

Example 10.


with , we have , and . Note that the three considered pencils are regular and, in particular, the eigenvalues of the first two are the same but their elementary divisors are not. In fact, their invariant polynomials are for , for , and for .

2.2 Induced hierarchy of matrix pencils

The tensor rank induces a straightforward hierarchy in any tensor space, namely, , where contains all tensors of rank up to . Our definition of minimal ranks also induces a hierarchy which can be expressed by using the definition of the sets given in (1). However, such a hierarchy is more intricate, as now we have, for instance, and but and .

Figure 1 contains a diagram depicting the hierarchy of sets in the space of real pencils, denoted by . We assume without loss of generality, since and have identical structures. For even , the set

is always non-empty, but for odd

the set is non-empty if and only if . This is because an pencil has full minimal ranks if and only if it is regular and its elementary divisors cannot be written as powers of linear forms, as we shall prove in the next section. In , this means that a pencil satisfies if and only if it is -equivalent to some other pencil where no eigenvalue of is in . Since complex-valued eigenvalues of a real matrix necessarily arise in pairs, this can evidently only happen for even values of . For concreteness, three examples concerning , and are shown in Figure 1.

Figure 1: Hierarchy of sets of real pencils in according to their minimal ranks: illustration of the general form (left) and concrete examples for three pencil spaces (right). The notation stands for . See (1) for a definition of the sets .

2.3 Minimal ranks of Kronecker canonical forms

We now show how the minimal ranks of a pencil can be determined from its Kronecker canonical form. The notation , where , will be used for a Jordan block of size associated with the finite elementary divisor . In this definition, the vectors denote as usual the canonical basis vectors of their corresponding spaces. A canonical block associated with an infinite elementary divisor will be expressed as . Let us first consider regular pencils.

Lemma 11.

Let be a regular pencil and let

be its Kronecker canonical form, where the elementary divisors of cannot be factored into powers of linear forms and for some , . Let be the largest number of blocks whose elementary divisors share a common factor and be the second largest number of blocks whose elementary divisors share a common factor (with ). Then


where and are the first and second largest components of , respectively.


By virtue of creftypecap 4, we have . It thus suffices to show that . The steps are as follows.

  1. First, we claim that is nonsingular for any . This claim is trivially true if and . For (and possibly null), the argument is as follows. Suppose for a contradiction that is singular, with . Without loss of generality, we may take . Then, there exists such that

    where . But then, , implying has an elementary divisor of the form , which contradicts the hypothesis that the elementary divisors of cannot be written as powers of linear forms. As a consequence, . In particular, if (i.e., ), then (4) yields (because ), as required.

  2. Now, note that for if and only if for some , in which case . Moreover, for if and only if for some , implying . Hence, since by definition , we have three cases:

    1. If , then we can apply the following transformation

      to obtain an -equivalent pencil attaining the minimal ranks of . Indeed, is minimal among all linear combinations with . Hence, (2) must equal . Furthermore, is minimal among all linear combinations with . So, (3) must equal .

    2. If , then using a similar argument we deduce that the -equivalent pencil attains the minimal ranks of , showing that .

    3. Finally, if , then following the same line of thought we have that attains the minimal ranks of , that is, .

For a singular canonical pencil having no regular part, computing the minimal ranks is straightforward, because and cannot be both nonzero for any given pair of indices . Indeed, the canonical block related to a minimal index associated with the columns is the pencil of the form

Any transformation applied to yields some pencil such that . In other words, has minimal ranks . By the same argument, the canonical block related to a minimal index associated with the rows, which is an pencil defined as , has minimal ranks . The special case (or ) also adheres to that rule, as its minimal ranks are . Now, adjoining blocks having these forms yields a singular pencil whose minimal ranks are clearly the sum of the minimal ranks of the blocks. Note that this is true even for the zero minimal indices , since they correspond to null columns and null rows, and so the minimal ranks must be bounded by . We have arrived at the following result.

Lemma 12.

Let be a singular pencil having the form , where are the minimal indices associated with its columns (henceforth called minimal column indices) and are the minimal indices associated with its rows (minimal row indices). Then,

For an arbitrary pencil , the block diagonal structure of its Kronecker canonical form allows a direct combination of the previous results, yielding the main theorem of this section.

Theorem 13.

Let be an arbitrary pencil with Kronecker canonical form , where is its singular part and is regular. Suppose has minimal column indices and minimal row indices . Define . Then, its minimal ranks are given by


where , whose components are given by creftypecap 11.

The above result implies that both minimal ranks of an singular pencil must be strictly smaller than . This is because the sum of the minimal indices of its singular part (which equals in (5)) can never attain the largest dimension of that part. Hence, if an pencil has minimal ranks or , then it is necessarily regular. In particular, pencils with full minimal ranks can be characterized as follows.

Corollary 14.

An pencil satisfies if and only if it is regular and its elementary divisors cannot be written as powers of linear forms.

2.4 Classification of -orbits for

Using creftypecap 13, a complete classification of all Kronecker canonical forms of pencils over is provided in Tables 4, 3, 2 and 1 for . Each such form is associated with a family of -orbits. We denote canonical blocks whose elementary divisors are powers of second-order irreducible polynomials by

where denotes the Kronecker product. Observe that, because we consider orbits of action, we can represent each family of orbits by a canonical form having no infinite elementary divisors (which can always be avoided by employing a transformation).

It can be checked that each described family with corresponds to a single orbit,666Atkinson [1] had already pointed out that, over an algebraically closed field , there are only finitely many -orbits for any . Thus, over this must be true of orbits whose elementary divisors are powers of linear forms. except for , which encompasses an infinite number of non-equivalent orbits. All families having dimensions and (or and ) also contain only one orbit each. These properties can be verified by inspecting the equivalent pencils of Table 5 shown ahead in Appendix A: for , the only family whose given canonical form depends on a parameter is that of . For , infinitely many non-equivalent orbits are contained by each family in general. To avoid redundancies, families of orbits having zero minimal indices are omitted in the tables, since each such family corresponds to some other one of lower dimensions. For instance for , if a singular pencil has minimal indices , then it can be expressed in the form , where both blocks in this decomposition have size . So, the canonical form of can be inspected to determine the properties of . Similarly, not all combinations of canonical blocks are included for the singular part, because the shown properties remain the same if we transpose these blocks. To exemplify, it can be checked that and have the same dimensions, tensor rank, multilinear rank and minimal ranks, because the roles played by column and row minimal indices are essentially the same, up to a transposition.

A family is denoted with the letter or the letter if it encompasses regular or singular pencils, respectively. The subscript indices of each family indicate its minimal ranks, and primes are used to distinguish among otherwise identically labeled families. The tensor rank of each canonical form was determined using Corollary 2.4.1 of Ja’Ja’ [9] and Theorem 4.6 of Sumi et al. [14], which requires taking into account the minimal indices of the pencil and also its elementary divisors. The values given in the column “multilinear rank” were determined by inspection; for a definition see [6]. Specifically, for an pencil viewed as an tensor , the multilinear rank is the triple satisfying

It should be noted that, in the above equation, denotes the subspace of spanned by the matrices and , whose dimension is at most two. Finally, in order to determine the minimal ranks (column labeled “”) of each family, creftypecap 13 was applied.

We point out that another classification of pencil orbits is given by Pervouchine [11], but his study is concerned with closures of orbits and pencil bundles, not with tensor rank or minimal ranks. Our list is therefore a complement to the one he provides. Furthermore, the hierarchy of closures of pencil bundles he has presented bears no direct connection with the hierarchy of sets we present in Section 2.2, which is easily determined by the numbers associated with each such set.

Family Canonical form tensor multilinear
rank rank
Table 1: Families of canonical forms of and real pencils having no zero minimal indices, with for .
Family Canonical form tensor multilinear
rank rank