## Authors

• 20 publications
• 7 publications
• 12 publications
• ### Nuclear Norm and Spectral Norm of Tensor Product

We show that the nuclear norm of the tensor product of two tensors is no...
09/24/2019 ∙ by Liqun Qi, et al. ∙ 0

• ### Error Preserving Correction for CPD and Bounded-Norm CPD

In CANDECOMP/PARAFAC tensor decomposition, degeneracy often occurs in so...
09/25/2017 ∙ by Anh-Huy Phan, et al. ∙ 0

• ### Spectral norm of random tensors

We show that the spectral norm of a random n_1× n_2×...× n_K tensor (or ...
07/07/2014 ∙ by Ryota Tomioka, et al. ∙ 0

• ### On orthogonal tensors and best rank-one approximation ratio

As is well known, the smallest possible ratio between the spectral norm ...
07/09/2017 ∙ by Zhening Li, et al. ∙ 0

• ### Exact nuclear norm, completion and decomposition for random overcomplete tensors via degree-4 SOS

In this paper we show that simple semidefinite programs inspired by degr...
11/18/2020 ∙ by Bohdan Kivva, et al. ∙ 0

• ### Tensor denoising with trend filtering

We extend the notion of trend filtering to tensors by considering the k^...
01/26/2021 ∙ by Francesco Ortelli, et al. ∙ 0

• ### Norm of Tensor Product, Tensor Norm, Cubic Power and Gelfand Limit

We establish two inequalities for the nuclear norm and the spectral norm...
09/24/2019 ∙ by Liqun Qi, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

In this paper, unless otherwise stated, all the discussions will be carried out in the field of real numbers.

Suppose that and are positive integers. Without loss of generality, we may assume that .

As in [7], we use to denote the operation of tensor outer product. Then for and , is a fourth order rank-one tensor in . By the following definition, it is actually a biquadratic rank-one tensor.

###### Definition 1.1

Let be the space of fourth order tensors of dimension . Let . The tensor is called biquadratic if for all and , we have

 ai1j1i2j2=ai2j1i1j2=ai1j2i2j1.

The tensor is called positive semi-definite if for any and ,

 ⟨A,x∘y∘x∘y⟩≡m∑i1,i2=1n∑j1,j2=1ai1j1i2j2xi1yj1xi2yj2≥0.

The tensor is called positive definite if for any and ,

 ⟨A,x∘y∘x∘y⟩≡m∑i1,i2=1n∑j1,j2=1ai1j1i2j2xi1yj1xi2yj2>0.

Denote the set of all biquadratic tensors in by . Then is a linear space.

Biquadratic tensors play a central role in many areas of science. Examples include the elasticity tensor and the Eshelby tensor in solid mechanics, and the Riemann curvature tensor in relativity theory. The elasticity tensor may be the most well-known tensor in solid mechanics and engineering [9]. The Eshelby inclusion problem is one of the hottest topics in modern solid mechanics [18]. Furthermore, the Riemann curvature tensor is the backbone of Einstein’s general relativity theory [13].

Biquadratic tensors have very special structures. The tensor product of two biquadratic tensors are still a biquadratic tensor. This makes them very special. Biquadratic tensors also have an M-eigenvalue structure. An important problem in solid mechanics is if strong ellipticity condition holds or not [6, 14]. In 2009, M-eigenvalues were introduced for the elastic tensor to characterize the strong ellipticity condition in [10]. An algorithm for computing the largest M-eigenvalue was presented in [15]. The biquadratic optimization problem was studied in [8]. The M-eigenvalue structure was further extended to the Riemann curvature tensor [16]. As the big data era arrived, the tensor completion problem came to the stage. It was shown that the nuclear norm of tensors plays an important role in the tensor completion problem [17]. A typical model in the tensor completion problem for higher order models is a general third order tensor [4, 17]. The nuclear norm is the dual norm of the spectral norm [3, 4, 17]. The spectral norm of a tensor is its largest singular value. In [11], it was shown that if we make contraction of a third order tensor with itself on one index, then we get a positive semi-definite biquadratic tensor. A real number is a singular value of that third order tensor if and only if it is the square root of an M-eigenvalue of that positive semi-definite biquadratic tensor. Thus, the spectral norm of that third order tensor is the square root of the spectral norm of that positive semi-definite biquadratic tensor.

We use small letters , etc., to denote scalars, small bold letters

, etc., to denote vectors, capital letters

, etc., to denote matrices, and calligraphic letters , etc., to denote tensors.

## 2 Norms and M-Eigenvalues of Biquadratic Tensors

For a vector , we use to denote its 2-norm. Thus,

 ∥u∥2:=√u21+⋯+u2m.

For a tensor , its spectral norm is defined as [3, 4, 5, 17]

 ∥A∥S:=max{|⟨A,x∘y∘u∘v⟩|:∥x∥2=∥y∥2=∥u∥2=∥v∥2=1,x,u∈Rm,y,v∈Rn}. (2.1)

We have the following theorem.

###### Theorem 2.1

Suppose that . Then

 ∥A∥S=max{|⟨A,x∘y∘x∘y⟩|:∥x∥2=∥y∥2=1,x∈Rm,y∈Rn}. (2.2)

Proof Suppose that the maximum of (2.1) is attained at and . Then

 ∥A∥S=max{|⟨A,x∘¯y∘u∘¯v⟩|:∥x∥2=∥u∥2=1,x,u∈Rm}.

Note that this is a homogeneous quadratic optimization. Then there is a such that and

 ∥A∥S=|⟨A,^x∘¯y∘^x∘¯v⟩|.

Then

 ∥A∥S=max{|⟨A,^x∘y∘^x∘v⟩|:∥y∥2=∥v∥2=1,y,v∈Rn}.

Again, this is a homogeneous quadratic optimization. Then there is a such that and

 ∥A∥S=|⟨A,^x∘^y∘^x∘^v⟩|.

This proves (2.2). .

In this way, also defines a norm in .

Recall that the nuclear norm of is defined as

 ∥A∥∗=inf{r∑j=1|λj|:A=r∑j=1λjx(j)∘y(j)∘u(j)∘v(j),∥x(j)∥2=∥y(j)∥2=∥u(j)∥2=∥v(j)∥2=1,x(j),u(j)∈Rm,y(j),v(j)∈Rn,r∈N}. (2.3)

By Corollary 5.4 of [3], we have

 ∥A∥∗=min{r∑j=1|λj|:A=r∑j=1λjx(j)∘y(j)∘x(j)∘y(j),∥x(j)∥2=∥y(j)∥2=1,x(j)∈Rm,y(j)∈Rn,r∈N}. (2.4)

It can be calculated as [3, 4, 5, 17]

 ∥A∥∗:=max{|⟨A,B⟩|:∥B∥S=1,B∈Rm×n×m×n}. (2.5)

For a biquadratic tensor, we have the following theorem.

###### Theorem 2.2

Suppose that . Then

 ∥A∥∗=max{|⟨A,B⟩|:∥B∥S=1,B∈BQ(m,n)}. (2.6)

Proof Without loss of generality, assume that is nonzero. Suppose that the maximum of (2.5) is attained at with . Let with

 ^bi1j1i2j2=14(¯bi1j1i2j2+¯bi2j1i1j2+¯bi1j2i2j1+¯bi2j2i1j1).

Then , , and

 ∥A∥∗=∣∣⟨A,¯B⟩∣∣=∣∣⟨A,^B⟩∣∣.

Since is not a zero tensor. This implies that is also not a zero tensor. Then . Let

 ~B=^B∥^B∥S.

We have , , and

 ∣∣⟨A,~B⟩∣∣≥∣∣⟨A,¯B⟩∣∣.

Since is a maximizer of (2.5), we have

 ∣∣⟨A,~B⟩∣∣=∣∣⟨A,¯B⟩∣∣.

This proves (2.6). .

These two theorems show that we may compute the spectral norm and nuclear norm for a biquadratic tensor by using its biquadratic structure. Then, either the number of variables is reduced, or the feasible region of the maximization problem can be reduced. Furthermore, a biquadratic tensor has its own M-eigenvalue structure which is closely related to its spectral norm.

###### Definition 2.3

Suppose that . A real number is called an M-eigenvalue of if there are vectors such that the following equations are satisfied: For ,

 m∑i2=1n∑j1,j2=1ai1j1i2j2yj1xi2yj2=λxi1; (2.7)

For ,

 m∑i1,i2=1n∑j2=1ai1j1i2j2xi1xi2yj2=λyj1; (2.8)

and

 x⊤x=y⊤y=1. (2.9)

Then and

are called the corresponding M-eigenvectors.

###### Theorem 2.4

Suppose that . Then its M-eigenvalues always exist. The spectral norm of is equal to the largest absolute value of its M-eigenvalues. Furthermore, is positive semi-definite if and only if all of its M-eigenvalues are nonnegative; is positive definite if and only if all of its M-eigenvalues are positive. If is positive semi-definite, then its spectral norm is equal to its largest M-eigenvalue.

This theorem was proved in [11].

For , the elastic tensor in solid mechanics falls in the category of biquadratic tensors, with one additional symmetric properties between indices and . Then, the positive definiteness condition of corresponds the strong ellipticity condition in solid mechanics.

Let . Then has a rank-one decomposition in the form

 A=r∑k=1x(k)∘y(k)∘s(k)∘w(k),

where for . The smallest for such a rank-one decomposition is called the rank of .

On the other hand, if we have

 A=r∑k=1x(k)∘y(k)∘x(k)∘y(k), (3.10)

where for for some positive integer , then we say that has a biquadratic rank-one decomposition. The smallest for such a biquadratic rank-one decomposition is called the biquadratic rank of . Denote it by . The question is if such a biquadratic rank-one decomposition always exists. We may following the approach in [1] to show this by introducing biquadratic polynomials and using them as a tool for the proof. Corollary 5.4 of [3] also implies this. Here, we give a constructive proof. This also gives an upper bound of the biquadratic rank of a biquadratic tensor.

###### Theorem 3.1

For , such a biquadratic rank-one decomposition always exists. We also have

 BR(A)≤mnmin{m(m+1)2,n(n+1)2}.

Proof For , define a matrix

 P=(pst)∈Rm(m+1)2×n(n+1)2

by

 pst=ai1j1i2j2

for

 s=i1(i1−1)2+i2

and

 t=j1(j1−1)2+j2,

with , and . Then

 M=q∑k=1σku(k)(v(k))⊤,

where , , , , for , and

 q≤min{m(m+1)2,n(n+1)2}.

For , we may fold it to a symmetric matrix . Similarly, for , we may fold it to a symmetric matrix . Suppose that has an eigenvalue decomposition

 U(k)=m∑lu=1λlux(k,lu)(x(k,lu))⊤,

where , , for , . Similarly, suppose that has an eigenvalue decomposition

 V(k)=m∑lv=1μlvy(k,lv)(y(k,lv))⊤,

where , , for , . Then we have

 A=q∑k=1m∑lu=1n∑lv=1σkλluμlvx(k,lu)∘y(k,lv)∘x(k,lu)∘y(k,lv).

We have the conclusions. .

Clearly, the biquadratic rank of a biquadratic tensor is always not less than its rank. In which cases are these two ranks equal? We do not go to further discussion on this in this paper.

Let . Fix and , then we have an -vector . Denote by the matrix whose column vectors are such -vectors for and . Then is the matrix flattening of by the first index. Here we do not specify the order of such column vectors in as this is not related. Denote the rank of by . We may define , and , respectively. They are the Tucker ranks of [5, 7]. Then we have and . Hence, only and are independent. We also have and .

Suppose that has a biquadratic rank-one decomposition as (3.10). Denote as an matrix, whose column vectors are , and as an matrix, whose column vectors are . Then, as in [7], we may denote (3.10) as

 A=[[X,Y]]BQ. (3.11)

Let . Suppose that has a biquadratic rank-one decomposition (3.11). Denote the ranks of and by and respectively. Then we have

 R(X)=R1(A), R(Y)=R2(A). (3.12)

We may also extend Tucker decomposition [2, 5, 7] to biquadratic Tucker decomposition. Denote as the mode- (matrix) product [2, 5, 7].

###### Definition 4.1

Let . Suppose that there are , and and such that

 A=B×1P×2Q×3P×4Q:=[[B;P,Q]]BQ. (4.13)

Then (4.13) is called a biquadratic Tucker decomposition of . The tensor is called a biquadratic Tucker core of . The matrices and are called the factor matrices of this decomposition. A biquadratic Tucker decomposition is said to be independent if and have full column rank. A biquadratic Tucker decomposition is said to be orthonormal if and have orthonormal columns.

Note that if the biquadratic Tucker decomposition (4.13) is independent, then and .

De Lathauwer, De Moor and Vandewalle [2] proposed an algorithm to compute Tucker decomposition (HOSVD) for a given tensor. If we apply their algorithm to a biquadratic tensor, since the first and the third matrix flattenings are the same, the second and the fourth flattenings are the same, we obtain an orthonormal biquadratic Tucker decomposition.

A biquadratic Tucker decomposition is a Tucker decomposition [5, 7]. Thus, a biquadratic Tucker core has the properties of a Tucker core. For example, the rank of a biquadratic Tucker core is the same as the rank of , if the biquadratic Tucker decomposition is independent [5]. Similarly, the Tucker ranks will also be preserved by an independent biquadratic Tucker decomposition. The problem is if some biquadratic properties, such as the biquadratic rank, and M-eigenvalues will be preserved or not.

We now prove the following theorem.

###### Theorem 4.2

Suppose that has a biquadratic Tucker decomposition (4.13) and it is independent. Then

 BR(A)=BR(B).

We first prove a lemma.

###### Lemma 4.3

If the biquadratic Tucker decomposition (4.13) is independent, then there are and such that

 B=A×1^P×2^Q×3^P×4^Q:=[[A;^P,^Q]]BQ. (4.14)

Proof Let and . The conclusion follows. .

Proof of Theorem 4.2. Suppose that has a biquadratic rank-one decomposition

 B=r∑k=1^x(k)∘^y(k)∘^x(k)∘^y(k), (4.15)

where , for . Then has a biquadratic rank-one decomposition (3.10) with

 x(k)=P^x(k), y(k)=Q^y(k),

for . This shows that

 BR(A)≤BR(B).

Since the biquadratic Tucker decomposition (4.13) is independent, by Lemma 4.3, we have (4.14). Thus, if has a biquadratic rank-one decomposition (3.10), then has a biquadratic rank-one decomposition (4.15), with

 ^x(k)=^Px(k), ^y(k)=^Qy(k),

for . This shows that

 BR(A)≥BR(B).

Hence, we have

 BR(A)=BR(B).

.

Jiang, Yang and Zhang [5] proved the following theorem (Theorem 7 of [5]).

###### Theorem 4.4

Suppose that has a biquadratic Tucker decomposition (4.13) and it is orthonormal. Then an M-eigenvalue of is an M-eigenvalue of , and a nonzero M-eigenvalue of is also an M-eigenvalue of .

By Theorem 2.4, this shows that the spectral norm is preserved under an orthonormal biquadratic Tucker decomposition.

Thus, biquadratic Tucker decomposition has better properties. It only involves two factor matrices and . This makes it much simple.

## 5 Lower and Upper Bounds of the Nuclear Norm of a Biquadratic Tensor

Let . If we regard as an index from to , and regard as another index from to , then we have a matrix flattening . Then there is a one to one relation between and . Hence, we may also write for . If is diagonal, then we also say that is diagonal. In particular, if

is the identity matrix

, then we denote as and call it the identity tensor in .

In the other words, for a fourth order tensor , an entry is called a diagonal entry if and . Otherwise, it is called an off-diagonal entry. Then a diagonal tensor in is a biquadratic tensor in such that all of its off-diagonal entries are , while the identity tensor is the diagonal biquadratic tensor in such that all of its diagonal entries are .

Denote the Frobenius norm of a fourth order tensor by , and the Frobenius norm of by . For , we use to denote the inner product of matrices and . Then for , we have

 (5.16)

and

 ∥A∥2=∥M(A)∥2. (5.17)

We first prove a proposition.

###### Proposition 5.1

Let . Then

 ∥M(A)∥S≥∥A∥S.

Proof Suppose that . Let be the Kronecker product of and . Then . If , then .

By Theorem 2.1, we have

 ∥A∥S =max{|⟨A,x∘y∘u∘v⟩|:∥x∥2=∥y∥2=∥u∥2=∥v∥2=1,x,u∈Rm,y,v∈Rn} =max{|⟨M(A),(x⊗y)∘(u⊗v)⟩|:∥x∥2=∥y∥2=∥u∥2=∥v∥2=1,x,u∈Rm,y,v∈Rn} ≤max{|⟨M(A),z∘w⟩|:∥z∥2=∥x∥2=1,z,w∈Rmn} =∥M(A)∥S.

This proves the proposition. .

If , then its matrix flattening is symmetric. We now have the following theorem.

###### Theorem 5.2

Suppose that and is its symmetric matrix flattening. Then

 ∥M∥∗≤∥A∥∗≤min{m,n}∥M∥∗. (5.18)

In particular, if is diagonal, we have

 ∥A∥∗=∥M∥∗=m∑i=1n∑j=1|aijij|. (5.19)

Proof We first prove the first inequality of (5.18). By (2.5), we have

 ∥A∥∗ =max{|⟨M(A),M(B)⟩|:∥B∥S≤1,B∈Rm×n×m×n} ≥max{|⟨M(A),M(B)⟩|:∥M(B)∥S≤1,B∈Rm×n×m×n} ≥max{|⟨M(A),B⟩|:∥B∥S≤1,B∈Rmn×mn} =∥M(A)∥∗,

where the third equality is due to (5.16), the first inequality is by Proposition 5.1, and the last equality is by the definition of the nuclear norm of a matrix.

We now prove the second inequality of (5.18). Since is symmetric, we may assume that has an eigenvalue decomposition

 M(A)=mn∑k=1λkz(k)(z(k))⊤,

where and , for . For each , corresponds to an matrix . Since , we have , where is the Frobenius norm. Then [4], we have

 ∥¯M(z(k))∥∗≤√min{m,n}.

On the other hand,

 ∥¯M(z(k))∥∗=min{m,n}∑l=1|σk,l|,

where for , are singular values of for . Then has a singular value decomposition

 ¯M(z(k))=min{m,n}∑l=1σk,lx(k,l)(y(k,l))⊤,

where , and , , for and . This implies

 min{m,n}∑l=1|σk,l|≤√min{m,n},

for . Then we have

 A =mn∑k=1λk¯M(z(k))∘¯M(z(k)) =mn∑k=1λk⎛⎝min{m,n}∑l=1σk,lx(k,l)∘y(k,l)⎞⎠∘⎛⎝min{m,n}∑l=1σk,lx(k,l)∘y(k,l)⎞⎠ =mn∑k=1λkmin{m,n}∑l,s=1σk,lσk,sx(k,l)∘y(k,l)∘x(k,s)∘y(k,s).

Thus,

 ∥A∥∗≤mn∑k=1min{m,n}∑l,s=1|λkσk,lσk,s|≤mn∑k=1|λk|min{m,n}=min{m,n}∥M(A)∥∗.

Finally, assume that is diagonal. Then

 A=m∑i=1n∑j=1aijije(i)∘¯e(j)∘e(i)∘¯e(j),

where for are the unit vectors in , while for are the unit vectors in . This implies that

 ∥A∥∗≤m∑i=1n∑j=1|aijij|=∥M(A)∥∗.

Then, by (5.18), we have (5.19).

This proves the theorem. .

Comparing this theorem with Theorem 5.2 of [4], our theorem is somewhat stronger.

This theorem says that the equality in the first inequality of (5.18) may hold. How about the second inequality of (5.18)?

###### Corollary 5.3
 ∥Im,n∥∗=mn

.

## 6 Norms of Tensor Products of Biquadratic Tensors

We may define products of two biquadratic tensors. Let , then we have , defined by

 ci1j1i2j2=m∑i3=1n∑j3=1ai1j1i3j3bi3j3i