Tensor-Ring Nuclear Norm Minimization and Application for Visual Data Completion

by   Jinshi Yu, et al.

Tensor ring (TR) decomposition has been successfully used to obtain the state-of-the-art performance in the visual data completion problem. However, the existing TR-based completion methods are severely non-convex and computationally demanding. In addition, the determination of the optimal TR rank is a tough work in practice. To overcome these drawbacks, we first introduce a class of new tensor nuclear norms by using tensor circular unfolding. Then we theoretically establish connection between the rank of the circularly-unfolded matrices and the TR ranks. We also develop an efficient tensor completion algorithm by minimizing the proposed tensor nuclear norm. Extensive experimental results demonstrate that our proposed tensor completion method outperforms the conventional tensor completion methods in the image/video in-painting problem with striped missing values.



There are no comments yet.


page 3

page 4


Tensor completion via nonconvex tensor ring rank minimization with guaranteed convergence

In recent studies, the tensor ring (TR) rank has shown high effectivenes...

Scaled Nuclear Norm Minimization for Low-Rank Tensor Completion

Minimizing the nuclear norm of a matrix has been shown to be very effici...

An Efficient Tensor Completion Method via New Latent Nuclear Norm

In tensor completion, the latent nuclear norm is commonly used to induce...

Parts for the Whole: The DCT Norm for Extreme Visual Recovery

Here we study the extreme visual recovery problem, in which over 90% of ...

Non-local Patch-based Low-rank Tensor Ring Completion for Visual Data

Tensor completion is the problem of estimating the missing entries of a ...

Exact 3D seismic data reconstruction using Tubal-Alt-Min algorithm

Data missing is an common issue in seismic data, and many methods have b...

Simultaneous Tensor Completion and Denoising by Noise Inequality Constrained Convex Optimization

Tensor completion is a technique of filling missing elements of the inco...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Low rank tensor completion (LRTC) problem aims to recover the incomplete tensor from the observed entries by assuming different low-rank tensor structures, and it has attracted a lot of attentions in the past decades [1, 2, 3, 4, 5]. Most recently, Zhao et al. proposed tensor ring decomposition [6], which achieves the state-of-the-art performance in the LRTC problem [3, 5, 4].

However, the drawbacks limit the application of the existing TR-based methods in practice. One of the drawbacks is that the existing TR-based methods are quite time-consuming. For example, TR-ALS [3] has the computational complexity of and the one of tensor ring low-rank factors (TRLRF)[5] equals , where denotes the order of the tensor, denotes the dimension of each mode, represents the rank of the model and is a constant between 0 and 1. It can be seen that the computational cost of the two methods increase with the sixth power of TR rank . It implies that a large TR-rank chosen in practice leads to terribly low efficiency, and sometimes with the memory-exploration problem. Though tensor ring weighted optimization (TR-WOPT) [4]

applies gradient descent algorithm to find the latent core tensors, the convergence rate of this method is low. Beside the computational complexity problem, the determination of the optimal TR rank is also a tough work in the completion problem. It is because that TR rank is defined as a vector, whose dimension equals to the order of the tensor. This fact makes that the computational complexity for rank selection exponentially increases with the dimension of the rank by using cross validation. Furthermore, TR decomposition is non-convex, so there is no theoretical guarantee to obtain the global minimum solution.

Figure 1: Illustration of TR representation of a th-order tensor and its tensor circular unfolding’s. The nodes for [6] represent tensors whose order is denoted by the number of edges. Each dimension of the tensor is specified by the number beside the edges. The connection line between two nodes denotes the multi-linear product operation of these two nodes along a specific mode. represents the tensor circular unfolding along modes and , specified by a red arc. Note that tensor circular unfolding’s are obtained by circularly shifting along the tensor ring.

To overcome these drawbacks, we develop a novel convex completion method by minimizing tensor ring nuclear norm defined in this paper. Specifically, we first define a new circular unfolding operation on higher-order tensor, and theoretically prove that ranks of the circularly-unfolded matrices bound their corresponding TR rank. After that, the tensor ring nuclear norm is defined as a sum of the matrix nuclear norm of the circularly-unfolded matrices. As a convex surrogate of TR rank, the proposed completion method not only has lower computational complexity than the conventional TR-based methods, but also avoids choosing the optimal TR rank manually. To sum up, our contributions of this paper are listed below:

  • We theoretically prove the relationship between the TR-rank and the rank of the circularly-unfolded matrix.

  • To our best knowledge, this is the first paper to introduce tensor ring nuclear norm, and it is demonstrated to obtain the state-of-the-art performance in the image and video completion problem.

  • An alternating direction method of multipliers (ADMM) based algorithm is developed to optimize the proposed model.

2 Tensor Ring Nuclear Norm

To introduce the TR nuclear norm formulation, we first define the tensor circular unfolding and then theoretically reveal its connection to TR rank.

Definition 1.

(Tensor circular unfolding) Let be a th-order tensor, its tensor circular unfolding is a matrix, denoted by of size whose elements are defined by




continuous indices (including th index) enumerate the rows of , and the rest indices for its columns. is the positive integer and named step-length in our paper.

Theorem 1.

Assume is th-order tensor with TR-format, then for each unfolding matrix ,


where .


The tensor with TR-format is expressed in element-wise form given by


denotes the th lateral slice matrix of the latent tensor , which is of size . By employing the property of the trace operation, the element of tensor can be expressed in tensor circular unfolding format, i.e.,


where with , with
We can also rewrite (2) in the index form, which is


Therefore, we can get

Definition 2.

(TR nuclear norm) Assume the tensor with TR-form, its TR nuclear norm is defined by



is defined as the sum of singular values of a matrix.

Note that TR nuclear norm is combined by a series of tensor circular folding’s of a tensor. In the case of , is obtained by circularly shifting along the tensor ring, shown in Fig. 1.

3 Tensor Ring Nuclear Norm Minimization

By utilizing the relationship between the TR-rank and the rank of the circularly-unfolded matrices, a novel convex model named TR nuclear norm minimization (TRNNM) is proposed for LRTC problem, i.e.,


where denotes the index set of observed entries of

The problem (3) is difficult to solve due to share the same entries and can’t be optimized independently. To simplify the optimization, we introduce additional tensors and thus obtain the equivalent formulation:

0:   Missing entry zero filled tensor data with observed index set ,
0:  , , .
1:   Initialize: zero filled , with , .
2:  for  to  do
4:     for  to  do
6:     end for
8:     for  to  do
10:     end for
11:     if  then
12:        break
13:     end if
14:  end for
Algorithm 1 The TRNNM algorithm.

ADMM is developed to solve problem (3) due to its efficient in solving optimization problem with multiple nonsmooth terms in the objective function [7]. We define the augmented Lagrangian function as follows:


According to the framework of ADMM, we can update ’s, and ’s as follows.

Update . It is easy to note that problem (10) can be converted to an equivalent formulation:


To optimize is equivalent to solve the subproblem:


The above problem has been proven to lead to a closed form in [8, 9, 10]. Thus the optimal can be given by:


where and denotes the thresholding SVD operation [9]. If the SVD of ,



is an identity matrix with the same size of

Update . The optimal with all other variables fixed is given by solving the following subproblem of (11):


It is easy to check that the solution of (3) is given by:


Update . The Lagrangian multiplier is updated by:


The TRNNM algorithm is summarized in Algorithm 1.

3.1 Computational complexity of algorithm

For a tensor with the computational complexity of our proposed method is where . In contrast to TRALS and TRLRF with computational complexity of and respectively, the computational complexity of our method is may much smaller when . In practice, as shown in [3, 5], the suitable TR-rank is always much higher than the dimension in the high-order form of visual data, which is also found in our experiment. In addition, due to the immense TR-rank selection possibilities, the computational complexity of TR-ALS and TRLRF exponentially increases by using cross validation.


Figure 2: Lena image with 30% of stripes missing and its restorations by completion methods.
Figure 3: The 10th and 20th frames (from top to bottom) of the ocean video with 70% of stripes missing and its restorations by tensor completion methods.
Figure 4: RSE under varying ratio of stripes missing from Lena image. (a) is the investigation of TRNNM with various , and (b) is the comparison of TRNNM with other methods.
RSE RunTime RSE RunTime RSE RunTime
FBCP 0.207 298.41 0.107 302.40 0.099 227.91
HaLRTC 0.207 41.65 0.125 41.43 0.093 32.77
SiLRTC-TT 0.303 81.47 0.287 43.48 0.268 40.63
tSVD 0.461 506.71 0.286 498.24 0.126 453.92
TR-ALS 0.116 568.42 0.113 851.37 0.103 1.54e3
TRNNM 0.098 235.52 0.066 233.71 0.048 215.01
Table 1: RSE and runtime (seconds) for 80%, 70% and 60% of stripes randomly missing from the ocean video.

4 Experiments

To validate our proposed method TRNNM, both image and video completion are used to compare our method and state-of-the-art methods, i.e., FBCP [11], HaLRTC [1], SiLRTC-TT [12], tSVD [13], TR-ALS [3]. We conduct each experiment 10 times and record the average relative square error (RSE) and its runtime, where

4.1 Image Completion

In this section, Lena image is used to evaluate the performance of TRNNM and its compared methods. The image is initially presented by 3rd-order tensor with size of . We directly reshape the image into 9th-order tensor with size of for the methods assuming the data with TT/TR structure, i.e. SiLRTC-TT, TR-ALS and TRNNM, due to the high-order reshape operation is often used to improve the performance in classification [14] and completions [12, 15, 3]. We conduct experiments under varying ratio of stripes missing from the image. In the case of th-order tensor, the TRNNM with is equivalent to that with , we thus only consider the step-length varying from 1 to where denotes the operation.

Fig. 4(a) shows that our proposed method obtains better results as step-length increases from 1 to 4, which indicates that closer to the step-length , stronger the ability of TRNNM to capture information. Seen from Fig. 4(b), TRNNM with significantly outperforms other methods in RSE under various missing ratios. The image restorations are shown in Fig. 2, which demonstrates that TRNNM with

performs best on estimating the stripes missing values.

Due to the superiority of TRNNM when step-length , we set in default in our later experiments.

4.2 Video Completion

The ocean video [2] with size is used in this experiment. As done before, the ocean video is reshaped into 7th-order tensor of size for TT/TR-based methods. We conduct experiment under varying ratio of stripes randomly missing from the video.

Table 1 shows that TRNNM significantly performs better than other methods in RSE under our considered missing ratios, i.e. , with an acceptable time-cost. TR-ALS obtains rather well performance following TRNNM, however with significantly time-cost, which is not applicable in practice. For the case of , the restorations of some frames are shown in Fig. 3. Observe that TRNNM obtains the detail information of the frames with a better resolution, which demonstrates the superiority of TRNNM on capturing the information of the video with stripes missing.

5 Conclusions

We propose a convex completion method by minimizing tensor ring nuclear norm which is first introduced in our paper. The proposed method not only has lower computational complexity than the previous TR-based methods, but also avoids choosing the optimal TR rank. Extensive experimental results demonstrate that the proposed method outperforms the conventional tensor completion methods in image/video completion problem.