# A Tensor Rank Theory and The Sub-Full-Rank Property

One fundamental property in matrix theory is that the rank of a matrix is always equal to the maximum value of all of its full rank submatrices. We call this property the sub-full-rank property. Matrix datasets are in general not of full rank. But we may always identify their full rank submatrices with maximum rank values. In this paper, we explore this property for tensors. We first present a theory for tensor ranks such that they are natural extension of matrix ranks. We present some axioms for tensor rank functions. Then we introduce strongly proper tensor rank functions. We define a partial order among tensor rank functions and show that there exists a unique smallest tensor rank function. We show that the smallest tensor rank function is strongly proper and has the sub-full-rank property. We also show that the closure of a strongly proper tensor rank function is a strongly proper tensor rank function with the sub-full-rank property. An example of a strongly proper tensor rank function, which is easily computable, is the submax-Tucker rank function, which is associated with the Tucker decomposition.

## Authors

• 20 publications
• 12 publications
• 10 publications
• ### A Tensor Rank Theory, Full-Rank Tensors and Base Subtensors

A matrix always has a full-rank submatrix such that the rank of this mat...
04/22/2020 ∙ by Liqun Qi, et al. ∙ 0

• ### A Tensor Rank Theory, Full Rank Tensors and The Sub-Full-Rank Property

A matrix always has a full rank submatrix such that the rank of this mat...
04/22/2020 ∙ by Liqun Qi, et al. ∙ 0

• ### A Tensor Rank Theory and Maximum Full Rank Subtensors

A matrix always has a full rank submatrix such that the rank of this mat...
04/22/2020 ∙ by Liqun Qi, et al. ∙ 0

• ### A Unified Theory for Tensor Ranks and its Application

In this paper, we present a unified theory for tensor ranks such that th...
04/22/2020 ∙ by Liqun Qi, et al. ∙ 0

• ### Bilinear Complexity of 3-Tensors Linked to Coding Theory

A well studied problem in algebraic complexity theory is the determinati...
03/15/2021 ∙ by Eimear Byrne, et al. ∙ 0

• ### Balancing Interpretability and Predictive Accuracy for Unsupervised Tensor Mining

The PARAFAC tensor decomposition has enjoyed an increasing success in ex...
09/04/2017 ∙ by Ishmam Zabir, et al. ∙ 0

• ### A Solution for Large Scale Nonlinear Regression with High Rank and Degree at Constant Memory Complexity via Latent Tensor Reconstruction

This paper proposes a novel method for learning highly nonlinear, multiv...
05/04/2020 ∙ by Sandor Szedmak, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

A matrix always has a full-rank submatrix such that the rank of this matrix is equal to the rank of that submatrix. Furthermore, the rows (columns) of this matrix are linear combinations of the rows (columns) of this matrix, corresponding to that submatrix. These two properties form the corner stones of matrix theory. We call that submatrix a base submatrix of this matrix, and the two properties the sub-full-rank property and the base submatrix property, respectively. In this paper, we explore these two properties for tensors.

We now arrive the era of big data and tensors. Are there similar theory for tensors? We explore this possibility in this paper.

We first present a theory for tensor ranks such that they are natural extension of matrix ranks. We propose some axioms for tensor rank functions. Then we introduce proper and strongly proper tensor rank functions. We define a partial order among tensor rank functions and show that there exists a unique smallest tensor rank function. We introduce the concepts of full-rank tensors and base subtensors, and show that the max-Tucker tensor rank function always has the sub-full-rank property and the base subtensor property. We then define the closure for a proper tensor rank function, and show that the closure of a proper tensor rank function is a proper tensor rank function with the sub-full-rank property. We also show that the smallest tensor rank function is strongly proper and has the sub-full-rank property.

The set of all nonnegative integers is denoted by . The set of all positive integers is denoted by . Let . Denote the set of all real th order tensors of dimension by . If , then we denote it by . Here “CT” means cubic tensors. Denote the set of all real tensors by

. Thus, scalars, vectors, matrices are a part of

. Let . We call a rank-one tensor if and only if there are nonzero vectors for , such that

 X=x(1)∘⋯∘x(m).

Here, is the tensor outer product. Then, nonzero vectors and scalars are all rank-one tensors in this sense.

Suppose that and . Let and for . Suppose that for , with if for . Then we say that is a subtensor of . If , then we say that is a proper subtensor of . If all the entries of , which are not in , are zero, then we say that is an essential subtensor of .

In the next section, we present a set of axioms for tensor rank functions. We list six properties which are essential for tensor ranks. In particular, we define a partial order “” among tensor rank functions, and show that there exists a unique smallest tensor rank function . We also introduce proper and strongly proper tensor rank functions in that section.

We study the CP rank and the Tucker rank in Section 3. The Tucker rank is a vector rank. We derive two tensor rank functions from this, and call them the max-Tucker rank and the submax-Tucker rank respectively. We show that the CP rank, the max-Tucker rank and the submax-Tucker rank are all tensor rank functions. The CP rank is subadditive but not proper. The max-Tucker rank is proper, subadditive but not strongly proper. The submax-Tucker rank is strongly proper but not subadditive.

We introduce the concepts of full rank tensors and base subtensors, and define the sub-full-rank property and the base subtensor property in Section 4. We show that the max-Tucker rank function has these two properties.

In Section 5, we define the closure of a proper tensor rank function, and show that the closure of a proper tensor rank function is a proper tensor rank function with the sub-full-rank property. We show that is strongly proper and has the sub-full-rank property.

We present an application of the submax-Tucker rank in internet traffic data approximation in Section 6.

Some final remarks are made in Section 7.

We use small letters to denote scalars, small bold letters to denote vectors, capital letters to denote matrices, and calligraphic letters to denote tensors.

## 2 Axioms and Properties of Tensor Rank Functions

Let . Consider . Suppose . An entry is called a diagonal entry of if . Otherwise, is called an off-diagonal entry of . If all the off-diagonal entries of are zero, then is called a diagonal tensor. If is diagonal, and all the diagonal entries of is , then is called the identity tensor of , and denoted as . Clearly, the identity tensor is unique to .

###### Definition 2.1

Suppose that . If satisfies the following six properties, then is called a tensor rank function.

Property 1 Suppose that . Then if and only if is a zero tensor, and if and only if is a rank-one tensor.

Property 2 For with , .

Property 3 Let , . Then is equal to the matrix rank of the matrix corresponding to .

Property 4 Let , , and is a real nonzero number. Then .

Property 5 Let , , and is a permutation on . Then , where , .

Property 6 Let . Suppose that , and is a subtensor of . Then . If is an essential subtensor of , then .

These six properties are essential for tensor ranks. Property 1 specifies rank zero tensors and rank-one tensors. Though the tensor rank theory is not matured, there are no arguments in rank zero and rank-one tensors in the literature. Property 2 fixes the value of the tensor rank for identity tensors. Property 3 justifies the tensor rank is an extension of the matrix rank. Property 4 claims that the tensor rank is not changed when a tensor is multiplied by a nonzero real number. Property 5 says that the roles of the modes are balanced. Property 6 justifies the subtensor rank relation.

Suppose that are two tensor rank functions. If for any we always have , then we say that the tensor rank function is not greater than the tensor rank function and denote this relation as .

###### Theorem 2.2

Suppose that are two tensor rank functions. Define by

 r(X)=min{r1(X),r2(X)},

for any . Then is a tensor rank function, and .

Proof For any , let . Then Properties 1, 2, 3 and 4 hold clearly from the definition of tensor rank functions.

To show Property 5, we assume that is a permutated tensor of . Then and . Hence, and Property 5 is obtained.

Now we assume that is a subtensor of . Then and . Hence since and . For an essential subtensor , and . Hence and Property 6 holds.

Thus, we conclude that is a tensor rank function.

clearly, and .

.

###### Theorem 2.3

There exists a unique tensor rank function , such that for any tensor rank function , we have .

Proof For any , define . This is well-defined as tensor rank functions take values on . Now we show that is a tensor rank function.

1) Suppose is a zero tensor in . Then for any tensor rank function , . This implies that by the definition of . On the other hand, suppose that for some . Then for some tensor rank function , . Hence, is a zero tensor from Property 1 of the tensor rank function . Similarly, we may show that if and only if is a rank-one tensor.

2) For any , for all tensor rank functions . Thus .

3) Let . Let be the corresponding matrix in . Then for any tensor rank function , reduces to the matrix rank of . Hence all of are equal. Hence, will be the matrix rank of and Property 3 holds.

4) For any and any tensor rank function , for any . Thus, .

5) We have Properties 5 and 6 in a similar way as in the proof of Theorem 2.2 and omit the details here.

By the definition, for any tensor rank function .

Suppose that and are two tensor rank functions with the property that and for any tensor rank function. Then . We see that . Thus, such a tensor rank function is unique.

.

We call the smallest tensor rank function. In Section 4, we will show that has the sub-full-rank property.

The six properties in Definition 2.1 are essential to tensor rank functions. There are some other properties which are satisfied by some tensor rank functions.

###### Definition 2.4

Suppose that is a tensor rank function. We say that is a proper tensor rank function if for any and , we have .

###### Definition 2.5

Suppose that is a tensor rank function. We say that is a subadditive tensor rank function if for any , and , we have

 r(X+Y)≤r(X)+r(Y).
###### Proposition 2.6

Suppose that is a proper tensor rank function. Let with and . Then we have

 r(X)≤max{n1,⋯,nm}. (2.1)

Proof Let and with a subtensor . Then from Property 6. Together with since is proper, the result is arrived.

.

For with , we define submax as the second largest value of .

###### Definition 2.7

Suppose that is a tensor rank function. We say that is a strongly proper tensor rank function if for any with , and , we have

 r(X)≤submax{n1,⋯,nm}. (2.2)

We will show that is strongly proper in Section 4.

## 3 CP Rank, Max-Tucker Rank and Submax-Tucker Rank

As we stated in the introduction, our motivation to introduce the axiom system for tensor ranks is to find some tensor ranks which have the sub-full-rank property. The six properties of Definition 2.1 are not satisfied by some tensor ranks in the literature. For example, the tubal rank of third order tensors was introduced in [5]. For , . Thus, it is not a tensor rank function even for third order tensors. It is still very useful in applications [11, 12, 14, 13, 16].

However, the six properties of Definition 2.1 are satisfied by tensor ranks arising from two most important tensor decompositions – the CP decomposition and the Tucker decomposition.

We now study the CP rank [6].

###### Definition 3.1

Suppose that and . Suppose that there are for and such that

 X=r∑p=1a(1,p)∘⋯∘a(m,p), (3.3)

then we say that has a CP decomposition (3.3). The smallest integer such that (3.3) holds is called the CP rank of , and denoted as CPRank.

###### Theorem 3.2

The CP rank is a subadditive tensor rank function. It is not a proper tensor rank function.

Proof In this proof, is the CP rank. We first show that the CP rank is a tensor rank function. Properties 1, 3 and 4 hold clearly from the definition of the CP rank. Before we show Property 2, we can assert that for all since , where with the unique nonzero entry . In the following, we show Property 2 by induction for . We fix here.

For , reduces to the identity matrix and hence Property 2 is true for such a case. Now we assume that . Then we show .

Assume that with . Then

 Im,n=Im+1,n⋅e≡r∑p=1((e)Ta(m+1,p))a(1,p)∘⋯∘a(m,p).

Here, is the all one vector in . This indicates that since . This contradicts the assumption that and hence .

Hence, Property 2 holds.

For Property 5, we have that if when is a permutation of with . Hence we have Property 5.

For property 6, assume that is a subtensor of . For , let and be a subtensor of by a similar way of from . Then we have that and since are rank-one tensors for . This means that .

Furthermore, let be an essential subtensor of . Clearly, since is a subtensor of . Now we assume that . Suppose that and let

 ¯a(i,p)={ali,i,if  (i,p)=(li,i)∈Tl,0,otherwise,

where is the index set related to subtensor . Then and hence . Hence and Property 6 is satisfied and the CP rank is a tensor rank function.

Hence, the CP rank is a tensor rank function.

Suppose that with and . Let

 X=r1∑p=1a(1,p)∘⋯∘a(m,p),Y=r2∑q=1b(1,q)∘⋯∘b(m,q).

It holds that

 X+Y=r1∑p=1a(1,p)∘⋯∘a(m,p)+r2∑q=1b(1,q)∘⋯∘b(m,q).

Hence, . This shows that it is subadditive. By [6], the CP rank of a tensor given by Kruskal is between and . Thus, the CP rank is not a proper tensor rank function. .

We now study the Tucker rank. In some papers such as [4], the -rank is called the Tucker rank.

Suppose that and . We may unfold to a matrix for . Denote the matrix ranks of as for . Then the vector is called the -rank of [6].

The -rank is a vector rank. Hence it does not satisfy Definition 2.1. However, if we define

 r=max{r1,⋯,rm}, (3.4)

then we have the following proposition.

###### Theorem 3.3

The function defined by (3.4) is a proper, subadditive tensor rank function. But it is not strongly proper.

Proof We first show that rank function defined by (3.4) is a tensor rank function. To see this, it suffices to show that Property 1-6 are all satisfied.

1) Suppose that for is a zero tensor. Then are zero matrices for . This implies that for . By (3.4), we have . On the other hand, assume that for some with . This means that for , which means that and hence is a zero tensor.

Suppose that , then for all . This can be seen as follows. Assume that there exists such that , then since . From above analysis, if and only if . This contradicts with .

Let . Then From , we have that is rank-one. From and , we have that for all is also rank-one.

Similarly, we have that for any , is rank-one.

Thus, for some and hence is a rank-one tensor.

Conversely, if is a rank-one tensor, then for some nonzero vectors . Then and for all . Thus .

Based on the above analysis, Property 1 is satisfied.

2) Denote Then is a rectangular matrix which can be partitioned to an -dimensional identity matrix and an zero matrix, for , and hence . Thus, .

3) When , then , and for any . Clearly, and hence .

4) Suppose that . For any , and any , and hence . Hence .

5) Suppose that and is any permuted tensor of . Then will be for . So . Hence and the result holds.

6) Suppose that is a subtensor of . Then for all , will be a submatrix of and since is matrix rank. So . Suppose that is an essential subtensor of , then is an essential subtensor of and hence . So we can assert that .

Now we conclude that defined by (3.4) is a tensor rank function.

It is clear that such a tensor rank function is proper from its definition. Furthermore, we have that such rank is also subadditive since matrix rank is subadditive.

In addition, we consider with where is the identity matrix of three dimension. Hence . Hence we conclude that such a tensor rank function is not strongly proper.

.

Thus, we call this tensor rank function the max-Tucker rank in this paper, and denote it as max-TucRank.

Note that the max-Tucker rank naturally arises from applications of the Tucker decomposition when people assume that for and fix the value of [2, 10]. Then this means that tensors of max-Tucker ranks not greater than are used. In the following, we introduce a new tensor rank function, which is also associated with the Tucker decomposition, but is different from the max-Tucker rank. We may replace (3.4) by

 r=submax{r1,⋯,rm}. (3.5)

Then we have the following theorem.

###### Theorem 3.4

The function defined by (3.5) is a strongly proper tensor rank function. But it is not subadditive.

Proof We first show that function defined by (3.5) is a tensor rank function. It suffices to show that Property 1-6 are all satisfied.

1) Suppose that for is a zero tensor. Then are zero matrices for all . This implies that , which means that . By (3.5), we have . On the other hand, assume that for some with . This means that for some , , and hence , . Therefore, if and only if

Suppose that , then is not zero tensor and hence then for all . Since is the submax-Tucker rank, there exists such that . Without loss of generality, we assume that for . Let . Similar to discussion in proof of Theorem 3.3, is rank-one for all . Thus

 X=a(1,p)∘⋯∘(a(m,1)+λ2a(m,2)+λ3a(m,3)+⋯+λ¯ra(m,¯r)),

for some . Clearly, such is a rank-one tensor.

Conversely, if is a rank-one tensor, then for some nonzero vectors . Then and for all . Thus .

Based on the above analysis, Property 1 is satisfied.

2) Denote Then is a rectangular matrix which can be partitioned to an -dimensional identity matrix and an zero matrix, for , and hence . Thus, .

3) When , then , and for any . Clearly, and when . Hence is the same as matrix rank.

4) Suppose that . For any , and any , and hence . Hence .

5) Suppose that and is any permuted tensor of . Then will be for . So . Hence and the result holds.

6) Suppose that is a subtensor of . Then for all , will be a submatrix of and since is matrix rank. So . Suppose that is an essential subtensor of , then is an essential subtensor of and hence . So we can assert that .

Now we conclude that defined by (3.5) is a tensor rank function.

The strongly proper property of such a tensor rank function is clear and hence it suffices to show that it is not subadditive.

Let with and

 yijk=0  if  i>n1, j>n2, k>n3,zpqs=0  if  p≤n1, q≤n2,s≤n3.

It is assumed that , and , . Then for and . So since and . Therefore, we conclude that such a tensor rank function is not subadditive.

.

Thus, we call this tensor rank function the submax-Tucker rank in this paper, and denote it as submax-TucRank.

###### Proposition 3.5

We have submax-TucRank max-TucRank and submax-TucRank max-TucRank. Thus, max-TucRank .

Proof Clearly, submax-TucRank max-TucRank. To see submax-TucRank max-TucRank, we consider the following counterexample.

Consider the tensor with its nonzeros entries . By observation, we have that and , which implies that submax-TucRankmax-TucRank and the result is arrived here.

.

We cannot replace submax in (3.5) by the third largest value in , as this will violate Property 1 of Definition 2.1.

## 4 Full-Rank Tensors and Base Subtensors

In this section, we introduce the concepts of full-rank tensors and base subtensors.

We first define the full rank concept for a tensor rank function.

###### Definition 4.1

Suppose that is a tensor rank function. Let with , and . If we have

 r(X)=ni (4.6)

for one integer satisfying , then we say that is of full rank. In particular, zero tensors and rank-one tensors are regarded as full of rank.

We then define the sub-full-rank property for a tensor rank function.

###### Definition 4.2

Suppose that is a tensor rank function. We say that is of the sub-full-rank property if for any with , and , either is of full rank, or has a subtensor such that and is of full rank.

Now, the question is if any tensor rank function has such a sub-full-rank property. We have the following theorem.

###### Theorem 4.3

The max-Tucker rank function has the sub-full-rank property.

Proof Let with , and . Denote as the max-Tucker rank. Assume that the matrix rank of