1 Introduction
Low rank tensor completion (LRTC) problem aims to recover the incomplete tensor from the observed entries by assuming different lowrank tensor structures, and it has attracted a lot of attentions in the past decades [1, 2, 3, 4, 5]. Most recently, Zhao et al. proposed tensor ring decomposition [6], which achieves the stateoftheart performance in the LRTC problem [3, 5, 4].
However, the drawbacks limit the application of the existing TRbased methods in practice. One of the drawbacks is that the existing TRbased methods are quite timeconsuming. For example, TRALS [3] has the computational complexity of and the one of tensor ring lowrank factors (TRLRF)[5] equals , where denotes the order of the tensor, denotes the dimension of each mode, represents the rank of the model and is a constant between 0 and 1. It can be seen that the computational cost of the two methods increase with the sixth power of TR rank . It implies that a large TRrank chosen in practice leads to terribly low efficiency, and sometimes with the memoryexploration problem. Though tensor ring weighted optimization (TRWOPT) [4]
applies gradient descent algorithm to find the latent core tensors, the convergence rate of this method is low. Beside the computational complexity problem, the determination of the optimal TR rank is also a tough work in the completion problem. It is because that TR rank is defined as a vector, whose dimension equals to the order of the tensor. This fact makes that the computational complexity for rank selection exponentially increases with the dimension of the rank by using cross validation. Furthermore, TR decomposition is nonconvex, so there is no theoretical guarantee to obtain the global minimum solution.
To overcome these drawbacks, we develop a novel convex completion method by minimizing tensor ring nuclear norm defined in this paper. Specifically, we first define a new circular unfolding operation on higherorder tensor, and theoretically prove that ranks of the circularlyunfolded matrices bound their corresponding TR rank. After that, the tensor ring nuclear norm is defined as a sum of the matrix nuclear norm of the circularlyunfolded matrices. As a convex surrogate of TR rank, the proposed completion method not only has lower computational complexity than the conventional TRbased methods, but also avoids choosing the optimal TR rank manually. To sum up, our contributions of this paper are listed below:

We theoretically prove the relationship between the TRrank and the rank of the circularlyunfolded matrix.

To our best knowledge, this is the first paper to introduce tensor ring nuclear norm, and it is demonstrated to obtain the stateoftheart performance in the image and video completion problem.

An alternating direction method of multipliers (ADMM) based algorithm is developed to optimize the proposed model.
2 Tensor Ring Nuclear Norm
To introduce the TR nuclear norm formulation, we first define the tensor circular unfolding and then theoretically reveal its connection to TR rank.
Definition 1.
(Tensor circular unfolding) Let be a thorder tensor, its tensor circular unfolding is a matrix, denoted by of size whose elements are defined by
(1) 
where
(2) 
continuous indices (including th index) enumerate the rows of , and the rest indices for its columns. is the positive integer and named steplength in our paper.
Theorem 1.
Assume is thorder tensor with TRformat, then for each unfolding matrix ,
(3) 
where .
Proof.
The tensor with TRformat is expressed in elementwise form given by
(4) 
denotes the th lateral slice matrix of the latent tensor , which is of size . By employing the property of the trace operation, the element of tensor can be expressed in tensor circular unfolding format, i.e.,
(5) 
where with , with
We can also rewrite (2) in the index form, which is
(6)  
Therefore, we can get ∎
Definition 2.
(TR nuclear norm) Assume the tensor with TRform, its TR nuclear norm is defined by
(7) 
where
is defined as the sum of singular values of a matrix.
Note that TR nuclear norm is combined by a series of tensor circular folding’s of a tensor. In the case of , is obtained by circularly shifting along the tensor ring, shown in Fig. 1.
3 Tensor Ring Nuclear Norm Minimization
By utilizing the relationship between the TRrank and the rank of the circularlyunfolded matrices, a novel convex model named TR nuclear norm minimization (TRNNM) is proposed for LRTC problem, i.e.,
(8) 
where denotes the index set of observed entries of
The problem (3) is difficult to solve due to share the same entries and can’t be optimized independently. To simplify the optimization, we introduce additional tensors and thus obtain the equivalent formulation:
(9)  
ADMM is developed to solve problem (3) due to its efficient in solving optimization problem with multiple nonsmooth terms in the objective function [7]. We define the augmented Lagrangian function as follows:
(10) 
According to the framework of ADMM, we can update ’s, and ’s as follows.
Update . It is easy to note that problem (10) can be converted to an equivalent formulation:
(11) 
To optimize is equivalent to solve the subproblem:
(12) 
The above problem has been proven to lead to a closed form in [8, 9, 10]. Thus the optimal can be given by:
(13) 
where and denotes the thresholding SVD operation [9]. If the SVD of ,
(14) 
where
is an identity matrix with the same size of
Update . The optimal with all other variables fixed is given by solving the following subproblem of (11):
(15) 
It is easy to check that the solution of (3) is given by:
(16) 
where
Update . The Lagrangian multiplier is updated by:
(17) 
The TRNNM algorithm is summarized in Algorithm 1.
3.1 Computational complexity of algorithm
For a tensor with the computational complexity of our proposed method is where . In contrast to TRALS and TRLRF with computational complexity of and respectively, the computational complexity of our method is may much smaller when . In practice, as shown in [3, 5], the suitable TRrank is always much higher than the dimension in the highorder form of visual data, which is also found in our experiment. In addition, due to the immense TRrank selection possibilities, the computational complexity of TRALS and TRLRF exponentially increases by using cross validation.
RSE  RunTime  RSE  RunTime  RSE  RunTime  

FBCP  0.207  298.41  0.107  302.40  0.099  227.91 
HaLRTC  0.207  41.65  0.125  41.43  0.093  32.77 
SiLRTCTT  0.303  81.47  0.287  43.48  0.268  40.63 
tSVD  0.461  506.71  0.286  498.24  0.126  453.92 
TRALS  0.116  568.42  0.113  851.37  0.103  1.54e3 
TRNNM  0.098  235.52  0.066  233.71  0.048  215.01 
4 Experiments
To validate our proposed method TRNNM, both image and video completion are used to compare our method and stateoftheart methods, i.e., FBCP [11], HaLRTC [1], SiLRTCTT [12], tSVD [13], TRALS [3]. We conduct each experiment 10 times and record the average relative square error (RSE) and its runtime, where
4.1 Image Completion
In this section, Lena image is used to evaluate the performance of TRNNM and its compared methods. The image is initially presented by 3rdorder tensor with size of . We directly reshape the image into 9thorder tensor with size of for the methods assuming the data with TT/TR structure, i.e. SiLRTCTT, TRALS and TRNNM, due to the highorder reshape operation is often used to improve the performance in classification [14] and completions [12, 15, 3]. We conduct experiments under varying ratio of stripes missing from the image. In the case of thorder tensor, the TRNNM with is equivalent to that with , we thus only consider the steplength varying from 1 to where denotes the operation.
Fig. 4(a) shows that our proposed method obtains better results as steplength increases from 1 to 4, which indicates that closer to the steplength , stronger the ability of TRNNM to capture information. Seen from Fig. 4(b), TRNNM with significantly outperforms other methods in RSE under various missing ratios. The image restorations are shown in Fig. 2, which demonstrates that TRNNM with
performs best on estimating the stripes missing values.
Due to the superiority of TRNNM when steplength , we set in default in our later experiments.
4.2 Video Completion
The ocean video [2] with size is used in this experiment. As done before, the ocean video is reshaped into 7thorder tensor of size for TT/TRbased methods. We conduct experiment under varying ratio of stripes randomly missing from the video.
Table 1 shows that TRNNM significantly performs better than other methods in RSE under our considered missing ratios, i.e. , with an acceptable timecost. TRALS obtains rather well performance following TRNNM, however with significantly timecost, which is not applicable in practice. For the case of , the restorations of some frames are shown in Fig. 3. Observe that TRNNM obtains the detail information of the frames with a better resolution, which demonstrates the superiority of TRNNM on capturing the information of the video with stripes missing.
5 Conclusions
We propose a convex completion method by minimizing tensor ring nuclear norm which is first introduced in our paper. The proposed method not only has lower computational complexity than the previous TRbased methods, but also avoids choosing the optimal TR rank. Extensive experimental results demonstrate that the proposed method outperforms the conventional tensor completion methods in image/video completion problem.
References
 [1] Ji Liu, Przemyslaw Musialski, Peter Wonka, and Jieping Ye, “Tensor completion for estimating missing values in visual data,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 1, pp. 208–220, 2013.

[2]
Ji Liu, P. Musialski, P. Wonka, and Jieping Ye,
“Tensor completion for estimating missing values in visual data,”
in
2009 IEEE 12th International Conference on Computer Vision
, Sept 2009, pp. 2114–2121.  [3] Wenqi Wang, Vaneet Aggarwal, and Shuchin Aeron, “Efficient low rank tensor ring completion,” in 2017 IEEE International Conference on Computer Vision (ICCV), Oct 2017, pp. 5698–5706.
 [4] Longhao Yuan, Jianting Cao, Qiang Wu, and Qibin Zhao, “Higherdimension tensor completion via lowrank tensor ring decomposition,” arXiv preprint arXiv:1807.01589, 2018.
 [5] Longhao Yuan, Chao Li, Danilo Mandic, Jianting Cao, and Qibin Zhao, “Tensor ring decomposition with rank minimization on latent space: An efficient approach for tensor completion,” arXiv preprint arXiv:1809.02288, 2018.
 [6] Qibin Zhao, Guoxu Zhou, Shengli Xie, Liqing Zhang, and Andrzej Cichocki, “Tensor ring decomposition,” CoRR, vol. abs/1606.05535, 2016.
 [7] Zhouchen Lin, Minming Chen, and Yi Ma, “The augmented lagrange multiplier method for exact recovery of corrupted lowrank matrices,” arXiv preprint arXiv:1009.5055, 2010.
 [8] Shiqian Ma, Donald Goldfarb, and Lifeng Chen, “Fixed point and bregman iterative methods for matrix rank minimization,” Mathematical Programming, vol. 128, no. 12, pp. 321–353, 2011.
 [9] JianFeng Cai, Emmanuel J Candès, and Zuowei Shen, “A singular value thresholding algorithm for matrix completion,” SIAM Journal on Optimization, vol. 20, no. 4, pp. 1956–1982, 2010.
 [10] Benjamin Recht, Maryam Fazel, and Pablo A Parrilo, “Guaranteed minimumrank solutions of linear matrix equations via nuclear norm minimization,” SIAM review, vol. 52, no. 3, pp. 471–501, 2010.
 [11] Qibin Zhao, Liqing Zhang, and Andrzej Cichocki, “Bayesian cp factorization of incomplete tensors with automatic rank determination,” IEEE transactions on pattern analysis and machine intelligence, vol. 37, no. 9, pp. 1751–1763, 2015.
 [12] Johann A Bengua, Ho N Phien, Hoang Duong Tuan, and Minh N Do, “Efficient tensor completion for color image and video recovery: Lowrank tensor train,” IEEE Transactions on Image Processing, vol. 26, no. 5, pp. 2466–2479, 2017.
 [13] Zemin Zhang and Shuchin Aeron, “Exact tensor completion using tSVD,” arXiv preprint arXiv:1502.04689, 2015.

[14]
Alexander Novikov, Dmitrii Podoprikhin, Anton Osokin, and Dmitry P Vetrov,
“Tensorizing neural networks,”
in Advances in Neural Information Processing Systems, 2015, pp. 442–450.  [15] Longhao Yuan, Qibin Zhao, and Jianting Cao, “Highorder tensor completion for data recovery via sparse tensortrain optimization,” CoRR, vol. abs/1711.02271, 2017.
Comments
There are no comments yet.