 # ST-SVD Factorization and s-Diagonal Tensors

A third order real tensor is mapped to a special f-diagonal tensor by going through Discrete Fourier Transform (DFT), standard matrix SVD and inverse DFT. We call such an f-diagonal tensor an s-diagonal tensor. An f-diagonal tensor is an s-diagonal tensor if and only if it is mapped to itself in the above process. The third order tensor space is partitioned to orthogonal equivalence classes. Each orthogonal equivalence class has a unique s-diagonal tensor. Two s-diagonal tensors are equal if they are orthogonally equivalent. Third order tensors in an orthogonal equivalence class have the same tensor tubal rank and T-singular values. Four meaningful necessary conditions for s-diagonal tensors are presented. Then we present a set of sufficient and necessary conditions for s-diagonal tensors. Such conditions involve a special complex number. In the cases that the dimension of the third mode is 2, 3 and 4, we present direct sufficient and necessary conditions which do not involve such a complex number.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Matrix SVD factorization is a fundamental tool in numerical linear algebra . For a real matrix , it is decomposed to the product of an orthogonal matrix , an diagonal matrix , and an orthogonal matrix :

 A=USV⊤. (1.1)

Here, needs to be a nonnegative diagonal matrix. Then the diagonal entries of are singular values of . If the diagonal entries of is in the standard nonincreasing order, then we may regard (1.1) as a standard SVD of .

The matrix SVD factorization (1.1) was extended to third order tensors as T-SVD factorization by Kilmer and Martin [4, 5]. The T-SVD factorization has been found wide applications in engineering and tensor computation [1, 3, 6, 7, 8, 10, 11, 13, 14, 15, 16, 17, 18, 19, 21, 20, 22, 23]. In T-SVD factorization, an third order real tensor is decomposed to the tensor product of an orthogonal tensor , an real f-diagonal tensor , and an orthogonal tensor :

 A=U∗S∗V⊤, (1.2)

where is the tensor-tensor product, or t-product for short. An tensor is called an f-diagonal tensor if all of its frontal slices are diagonal matrices, for . In the next section, we will review the knowledge about t-product, orthogonal tensors, f-diagonal tensors, etc.

The T-SVD factorization (1.2) is used in low rank tensor approximation, where the rank is the tensor tubal rank . To define the tensor tubal rank, the third order tensor must go through Discrete Fourier transform (DFT), standard matrix SVD and inverse DFT. The f-diagonal tensor , thus obtained, may be denoted as , where is the Kilmer-Martin mapping to represent the above processing. In Section 3, we will state the Kilmer-Martin mapping in details.

The f-diagonal tensor in this way obtained is a special f-diagonal tensor. We call such an f-diagonal tensor an s-diagonal tensor. In Section 4, we formally define s-diagonal tensors. Actually, an f-diagonal tensor is an s-diagonal tensor if there is a tensor such that . Two tensors and are called orthogonally equivalent  if there are an orthogonal tensor and an orthogonal tensor such that

 A=U∗B∗V⊤.

It was proved in  that if and are orthogonally equivalent, then . Based upon this result, in Section 4, we derive a checkable result for s-diagonal tensors: an f-diagonal tensor is an s-diagonal tensor if and only if . Then we have an orthogonal partition theorem for . The third order tensor space is partitioned into orthogonal equivalence classes. Each orthogonal equivalence class has a unique s-diagonal tensor. Any pair of two nonzero orthogonal equivalence classes have the same cardinality in the sense that a one-to-one relation can be established between these two classes. Two s-diagonal tensors are equal if they are orthogonally equivalent. Third order tensors in an orthogonal equivalence class have the same tensor tubal rank and T-singular values, which can be calculated from the entries of the s-diagonal tensor in that class. The T-singular values of that class are linked with some extremal values of third order tensors in that class.

In Section 5, we present four necessary conditions for an f-diagonal tensor to be an s-diagonal tensor. These four necessary conditions are the tubal 2-norm decay property, the first frontal slice decay property, the third mode symmetry property and the tubal leading entry maximum property. We state the meanings of these four necessary conditions in that section.

Then, in Section 6, we present a set of sufficient and necessary conditions for an f-diagonal tensor to be an s-diagonal tensor. Such conditions involve a special complex number, hence are not so direct. From these conditions, we conclude that the set of the s-diagonal tensors is a closed convex cone.

In Section 7, for , we present direct sufficient and necessary conditions for an f-diagonal tensor to be an s-diagonal tensor. No complex numbers are involved in these conditions. An s-diagonal tensor may not be nonnegative. This can be seen from these sufficient and necessary conditions.

Some final remarks are made in Section 8.

## 2 Preliminaries

In this paper, and are positive integers, and . Denote real matrices by capital Roman letters , complex matrices by capital Greek letters , and tensors by Euler script letters . We use to denote the real number field, to denote the complex number field. The set of all real tensors is denoted as . Then is a linear space. For a third order tensor , its -th element is represented by . The frontal slice is denoted by .

A tensor is called an f-diagonal tensor if all of its frontal slices are diagonal matrices for . The set of all real f-diagonal tensors is denoted as , which is also a linear space.

For a third order tensor , define

 bcirc(A):=⎛⎜ ⎜ ⎜ ⎜ ⎜ ⎜⎝A(1)A(p)A(p−1)⋯A(2)A(2)A(1)A(p)⋯A(3)⋅ ⋅⋅⋯⋅⋅ ⋅⋅⋯⋅A(p)A(p−1)A(p−2)⋯A(1)⎞⎟ ⎟ ⎟ ⎟ ⎟ ⎟⎠,

and bcircbcirc.

For , its transpose is defined as

 A⊤=bcirc−1[(bcirc(A))⊤].

The identity tensor is defined as

 Innp=bcirc−1(Inp),

where

is the identity matrix in

.

For , define

 unfold(A):=⎛⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜⎝A(1)A(2)⋅⋅⋅A(p)⎞⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟⎠∈Rmp×n,

and foldunfold. For and , the T-product of and is defined as

 A∗B:=fold(bcirc(A)unfold(B))∈T(m,n,p).

For , if , then is called an orthogonal tensor. The set of all orthogonal tensors is denoted as , which is a group with respect to T-product.

## 3 The Kilmer-Martin Mapping

In this paper, we denote

 ω=e2π√−1p.

This is the special complex number which will be involved in some of our discussion. The Discrete Fourier Transform (DFT) matrix has the form

 Fp=1√p⎡⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢⎣11⋯111ω⋯ωp−2ωp−1⋮⋮⋱⋮⋮1ωp−2⋯ω(p−2)(p−2)ω(p−2)(p−1)1ωp−1⋯ω(p−2)(p−1)ω(p−1)(p−1)⎤⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥⎦.

The conjugate transpose of is denoted as .

For , we may block-diagonalize bcirc as

 Δ(A):=(Fp⊗Im)bcirc(A)(F∗p⊗In)=⎡⎢ ⎢ ⎢ ⎢ ⎢⎣Δ(1)Δ(2)⋱Δ(p)⎤⎥ ⎥ ⎥ ⎥ ⎥⎦,

where denotes the Kronecker product, for . Then we have

 Δ(k)≡Δ(k)(A)=p∑l=1ω(l−1)(k−1)A(l), (3.3)

for .

For each matrix , compute its SVD

 Δ(k)=Φ(k)Σ(k)Ψ(k)⊤,

where and are unitary matrices, is a diagonal matrix, the singular values of follow the standard nonincreasing order. Denote

 Σ(A):=⎡⎢ ⎢ ⎢ ⎢ ⎢⎣Σ(1)Σ(2)⋱Σ(p)⎤⎥ ⎥ ⎥ ⎥ ⎥⎦.

We also denote the th diagonal entry of as .

Let

 S=G(A):=bcirc−1((F∗p⊗Im)Σ(A)(Fp⊗In)).

Then . In particular, by the above equality, we have

 S(i,i,k)=1pp∑l=1¯ω(k−1)(l−1)Σ(i,i,l), (3.4)

for , .

We call the Kilmer-Martin mapping.

Let

 Φ(A):=⎡⎢ ⎢ ⎢ ⎢ ⎢⎣Φ(1)Φ(2)⋱Φ(p)⎤⎥ ⎥ ⎥ ⎥ ⎥⎦,
 Ψ(A):=⎡⎢ ⎢ ⎢ ⎢ ⎢⎣Ψ(1)Ψ(2)⋱Ψ(p)⎤⎥ ⎥ ⎥ ⎥ ⎥⎦,
 U=U(A)=bcirc−1((F∗p⊗Im)Φ(A)(Fp⊗In)),
 V=V(A)=bcirc−1((F∗p⊗Im)Ψ(A)(Fp⊗In)).

Then and , and we have

 A=U∗S∗V⊤. (3.5)

By [4, 5], we have

 p∑k=1S(1,1,k)2≥p∑k=1S(2,2,k)2≥⋯≥p∑k=1S(min{m,n},min{m,n},k)2. (3.6)

In , the th largest T-singular value of is defined as

 σi:= ⎷p∑k=1S(i,i,k)2,

for

. T-singular values are used there to define the tail energy for the error estimate of a proposed tensor sketching algorithm.

## 4 s-Diagonal Tensors

###### Definition 4.1

Denote the Kilmer-Martin mapping as . Let . If there is such that , then we say that is an s-diagonal tensor. The set of all s-diagonal tensors is denoted as . Let . If we have , and such that

 A=U∗S∗V⊤, (4.7)

then we say that has a ST-SVD (Standard T-SVD) factorization (4.7).

Note that in (4.7), we do not require that and are resulted from the Kilmer-Martin process, while (3.5) shows that such and always exist as long as .

The following theorem is Theorem 3.2 of .

###### Theorem 4.2

Suppose that and they are orthogonally equivalent. Then .

From this theorem, we have a checkable condition to determine a given tensor in is in or not.

###### Theorem 4.3

Suppose that . Then if and only if .

Proof If , then by definition, . On the other hand, suppose . Then there is such that . We also have and such that (3.5) holds. Then and are orthogonally equivalent. By Theorem 4.2, we have . .

The orthogonal equivalence is an equivalence relation of . We have the following orthogonal partition theorem.

###### Theorem 4.4

The linear space is partitioned to equivalence classes by the orthogonal equivalence relation. Each orthogonal equivalence class has a unique s-diagonal tensor. Any pair of two nonzero orthogonal equivalence classes have the same cardinality in the sense that a one-to-one relation can be established between these two classes. Two s-diagonal tensors are equal if they are orthogonally equivalent. Third order tensors in an orthogonal equivalence class have the same tensor tubal rank and T-singular values, which can be calculated from the entries of the s-diagonal tensor in that class.

Proof It is easy to see that the orthogonal equivalence is an equivalence relation of . Then we have the first conclusion. By Theorem 4.2, two s-diagonal tensors are equal if they are orthogonally equivalent. Then each orthogonal equivalence class has a unique s-diagonal tensor. Suppose two orthogonal equivalence classes of nonzero tensors have their s-diagonal tensors and . Then any tensor in these two classes have the forms

 A1=U∗S1∗V⊤

and

 A2=U∗S2∗V⊤,

respectively, where and . Then there is a one-to-one relation between these two classes, and the two classes have the same cardinality. By the definitions of tensor tubal rank and T-singular values, we have the remaining conclusions. .

The following proposition was proved in .

###### Proposition 4.5

Suppose that has T-singular values

 σ1≥σ2≥⋯≥σmin{m,n},

and . Then

 s∑l=1σ2l≥s∑l=1p∑k=1A(il,jl,k)2,

where , , are distinct pairs of indices.

By Theorems 4.2 and 4.4, Proposition 4.5 indicates that the T-singular values of that class are linked with some extremal values of third order tensors in that class.

###### Proposition 4.6

If there are , , and , such that

 A=U∗S∗V⊤, (4.8)

then and (4.8) is an ST-SVD of .

Proof By (4.8), and are orthogonally equivalent. Then as . .

Thus, if we have (4.8) and some sufficient conditions to show that , then we may conclude without calculate . In Sections 6 and 7, we will present some sufficient and necessary conditions to determine if , for a given f-diagonal tensor .

## 5 Necessary Conditions for s-Diagonal Tensors

There are four major features of s-diagonal tensors. Suppose that . Then has the following four properties:

(1) Tubal 2-Norm Decay Property, namely (3.6

). The 2-norms of the tubal vectors

decay with . It was used in  to define T-singular values and tail energy for error estimate of tensor sketching algorithms. This property can be found in . A proof for it can be found in .

(2) First Frontal Slice Decay Property, namely

 S(1,1,1)≥S(2,2,1)≥⋯≥S(min{m,n},min{m,n},1)≥0. (5.9)

Though some negative entries may appear in , the entries of its first frontal slice must be nonnegative and have a decay property. This property can be found in . A proof for it can be found in .

(3) Third Mode Symmetry Property, namely

 S(i,i,k)=S(i,i,p−k+2), (5.10)

for and . The tensor is symmetric for the third mode in the above sense. We identify this property in this paper and will prove it in the following theorem.

(4) Tubal Leading Entry Maximum Property, namely

 S(i,i,1)≥|S(i,i,k)|, (5.11)

for and . For each tube of , its first entry takes the maximum value. We identify this property in this paper and will prove it in the following theorem.

We have the following theorem.

###### Theorem 5.1

Suppose that . Then has the tubal 2-norm decay property (3.6), the first frontal slice decay property (5.9), the third mode symmetry property (5.10), and the tubal leading entry maximum property (5.11).

Proof As stated above, the tubal 2-norm decay property (3.6) and the first frontal slice decay property (5.9) have been identified before and proved elsewhere.

We now prove the third mode symmetry property (5.10). Since , there is such that . Let . Then by (3.4),

 S(i,i,k)=1pp∑l=1¯ω(k−1)(l−1)Σ(i,i,l), (5.12)

for , . Then for ,

 S(i,i,p−k+2) = 1pp∑l=1¯ω(p−k+2−1)(l−1)Σ(i,i,l) = 1pp∑l=1¯ω(1−k)(l−1)Σ(i,i,l).

Since and are conjugate to each other, they have the same real part. Thus, and have the same real part. However, and are real, thus, they are equal. We have (5.10).

We then prove the tubal leading entry maximum property (5.11). By (5.12), for , , we have

 |S(i,i,k)| = 1p∣∣ ∣∣p∑l=1¯ω(k−1)(l−1)Σ(i,i,l)∣∣ ∣∣ ≤ 1pp∑l=1|¯ω(k−1)(l−1)||Σ(i,i,l)| ≤ 1pp∑l=1|Σ(i,i,l)| = 1pp∑l=1Σ(i,i,l) = S(i,i,1).

This proves (5.11). .

However, the tubal 2-norm decay property (3.6), the first frontal slice decay property (5.9), the third mode symmetry property (5.10), and the tubal leading entry maximum property (5.11) are only some necessary conditions for s-diagonal tensors. For example, let , , , , , the other entries of are zero. Then the four properties (3.6) and (5.9-5.11) are satisfied. But we have . By Theorem 4.3, . Thus, the four properties (3.6) and (5.9-5.11) are only necessary conditions for s-diagonal tensors.

## 6 Sufficient and Necessary Conditions for s-Diagonal Tensors

We now present a set of sufficient conditions for s-diagonal tensors.

###### Theorem 6.1

Suppose that satisfies the third mode symmetry property (5.10). Then is real and diagonal. This implies that for and ,

 p∑l=1ω(l−1)(k−1)S(i,i,l)≡Δ(k)ii∈R. (6.13)

Then if and only if for and , we have

 p∑l=1ω(l−1)(k−1)S(i,i,l)≥0, (6.14)

and

 p∑l=1ω(l−1)(k−1)[S(i,i,l)−S(i+1,i+1,l)]≥0, (6.15)

where for .

Proof Since , is diagonal for . By (3.3),

 Δ(k)≡Δ(k)(S)=p∑l=1ω(l−1)(k−1)S(l).

Then, is diagonal for . We now show that is real for . For and , by (3.4), we have

 Δ(k)ii = p∑l=1ω(l−1)(k−1)S(i,i,l) = S(i,i,1)+p∑l=2ω(l−1)(k−1)S(i,i,l),

where is real. By the third mode symmetry property (5.10),

 ω(p−l+2−1)(k−1)S(i,i,p−l+2)=ω(1−l)(k−1)S(i,i,l)

is conjugate with . Thus,

 ω(l−1)(k−1)S(i,i,l)+ω(p−l+2−1)(k−1)S(i,i,p−l+2)∈R.

If

is odd, we have

 Δ(k)ii=S(i,i,1)+p+12∑l=2[ω(l−1)(k−1)S(i,i,l)+ω(p−l+2−1)(k−1)S(i,i,p−l+2)]∈R.

If is even, note that . Thus,

 Δ(k)ii=S(i,i,1)−S(i,i,p2+1)+p2∑l=2[ω(l−1)(k−1)S(i,i,l)+ω(p−l+2−1)(k−1)S(i,i,p−l+2)]∈R.

Thus, is real. Hence is real and diagonal.

Now, if (6.14) and (6.15) hold, then for , we have

 Δ(k)11≥Δ(k)22≥⋯≥Δ(k)min{m,n},min{m,n}≥0.

Since is diagonal, this implies that for , i.e., . Thus, .

On the other hand, if , then . Thus, for ,

 Δ(k)=p∑l=1ω(l−1)(k−1)S(l)=Σ(k).

Hence, is real nonnegative diagonal matrix, whose diagonal entries follow a nonincreasing order. This implies (6.14) and (6.15). .

By this theorem, we may conclude that is a closed convex cone. The question is what is its dual cone?

The conditions (6.14) and (6.15) involves the complex number . Their meanings are not so direct comparing with conditions (3.6) and (5.9-5.11). Thus, we still wish to find direct sufficient and necessary conditions for s-diagonal tensors.

## 7 Direct Sufficient and Necessary Conditions In the Cases that p=2,3,4

In this section, we present direct sufficient and necessary conditions for s-diagonal tensors in the cases that .

### 7.1 The Case that p=2

When , the third mode symmetry property (5.10) does not exist, and the tubal leading entry maximum property (5.11) is still needed. For , the tubal leading entry maximum property (5.11) has the form

 S(i,i,1)≥|S(i,i,2)|, (7.16)

for . Then we may replace the tubal 2-norm decay property (3.6) and the first frontal slice decay property (5.9) by a strong condition

(5) Strong First Frontal Slice Decay Property, namely

 S(i,i,1)−S(i+1,i+1,1)≥|S(i,i,2)−S(i+1,i+1,2)|, (7.17)

for . We call this property the strong first frontal slice decay property, as it is stronger than the first frontal slice decay property (5.9) under the tubal leading entry maximum property (7.16). Actually, for , (7.16) and (7.17) imply (3.6) and (5.9).

We have the following theorem.

###### Theorem 7.1

Let . Then if and only if the tubal leading entry maximum property (7.16) and the strong first frontal slice decay property (7.17) hold.

Proof For , . Since ,

 Δ(1)ii≡Δ(1)ii(S)=2∑l=1ω0S(i,i,l)=S(i,i,1)+S(i,i,2)∈R,
 Δ(2)ii≡Δ(2)ii(S)=2∑l=1ωl−1S(i,i,l)=S(i,i,1)−S(i,i,2)∈R.

The equality (7.16) is equivalent to

 S(i,i,1)≥max{S(i,i,2),−S(i,i,2)},

i.e.,

 S(i,i,1)+S(i,i,2)≥0, and  S(i,i,1)−S(i,i,2)≥0,

for . This is (6.14) for .

The equality (7.17) is equivalent to

i.e.,

 [S(i,i,1)−S(i+1,i+1,1)]+[S(i,i,2)−S(i+1,i+1,2)]≥0

and

 [S(i,i,1)−S(i+1,i+1,1)]−[S(i,i,2)−S(i+1,i+1,2)]≥0,

for . This is (6.15) for .

The conclusion follows from Theorem 6.1 now. .

Theorem 7.1 indicates that an s-diagonal tensor may not be nonnegative.

### 7.2 The Case that p=3

When , the third mode symmetry property (5.10) has the form

 S(i,i,2)=S(i,i,3), (7.18)

for . Then we need two more conditions.

(6) Strong Tubal Leading Entry Maximum Property, namely

 S(i,i,1)≥max{−2S(i,i,2),S(i,i,2)} (7.19)

for and . For , this condition is stronger than the tubal leading entry maximum property (5.11). Thus, we call it the strong tubal leading entry maximum property.

(7) Strong First Frontal Slice Decay Property for , namely

 S(i,i,1)−S(i+1,i+1,1)≥max{−2[S(i,i,2)−S(i+1,i+1,2)],S(i,i,2)−S(i+1,i+1,2)}, (7.20)

for . We call this property the strong first frontal slice decay property for , as it is stronger than the first frontal slice decay property (5.9) under the strong tubal leading entry maximum property (7.19).

We have the following theorem.

###### Theorem 7.2

Let . Then if and only if the third mode symmetry property (7.18), the strong tubal leading entry maximum property (7.19) and the strong first frontal slice decay property for (7.20) hold.

Proof For , we have and . Then for ,

 Δ(1)ii≡Δ(1)ii(S)=3∑l=1S(i,i,l)=S(i,i,1)+2S(i,i,2),

as by (7.18), ,

 Δ(2)ii≡Δ(2)ii(S) = 3∑l=1ωl−1S(i,i,l) = S(i,i,1)+ωS(i,i,2)+ω2S(i,i,3) = S(i,i,1)−S(i,i,2),
 Δ(3)ii≡Δ(3)ii(S) = 3∑l=1ω2(l−1)S(i,i,l) = S(i,i,1)+ω2S(i,i,2)+ωS(i,i,3) = S(i,i,1)−S(i,i,2).

Under the condition (7.18), the inequalities (7.19) and (7.20) are (6.14) and (6.15), respectively, for . Together with the third mode symmetry property (7.18), we have the conclusion by Theorem 6.1. .

### 7.3 The Case that p=4

When , the third mode symmetry property (5.10) has the form

 S(i,i,2)=S(i,i,4), (7.21)

for . Again, we need two more conditions:

 (7.22)

for , and

 [S(i,i,1)−S(i+1,i+1,1)]+[S(i,i,3)−S(i+1,i+1,3)]
 ≥max{2|S(i,i,2)−S(i+1,i+1,2)|,2[S(i,i,3)−S(i+1,i+1,3)]}, (7.23)

for . We will see that under (7.21), (7.22) and (7.23) are (6.14) and (6.15), respectively, for .

We have the following theorem.

###### Theorem 7.3

Let . Then if and only if (7.21), (7.22) and (7.23) hold.

Proof For , we have , , and . Then, for , we have

 Δ(1)ii≡Δ(1)ii(S)=4∑l=1S(i,i,l)=S(i,i,1)+2S(i,i,2)+S(i,i,3),
 Δ(2)ii≡Δ(2)ii(S) = 4∑l=1ωl−1S(i,i,l) = S(i,i,1)+ωS(i,i,2)+ω2S(i,i,3)+ω3S(i,i,4) = S(i,i,1)−S(i,i,3),
 Δ(3)ii≡Δ(3)ii(S) = 4∑l=1ω2(l−1)S(i,i,l) =