A General Model for Robust Tensor Factorization with Unknown Noise

05/18/2017 ∙ by Xi'ai Chen, et al. ∙ Xi'an Jiaotong University 0

Because of the limitations of matrix factorization, such as losing spatial structure information, the concept of low-rank tensor factorization (LRTF) has been applied for the recovery of a low dimensional subspace from high dimensional visual data. The low-rank tensor recovery is generally achieved by minimizing the loss function between the observed data and the factorization representation. The loss function is designed in various forms under different noise distribution assumptions, like L_1 norm for Laplacian distribution and L_2 norm for Gaussian distribution. However, they often fail to tackle the real data which are corrupted by the noise with unknown distribution. In this paper, we propose a generalized weighted low-rank tensor factorization method (GWLRTF) integrated with the idea of noise modelling. This procedure treats the target data as high-order tensor directly and models the noise by a Mixture of Gaussians, which is called MoG GWLRTF. The parameters in the model are estimated under the EM framework and through a new developed algorithm of weighted low-rank tensor factorization. We provide two versions of the algorithm with different tensor factorization operations, i.e., CP factorization and Tucker factorization. Extensive experiments indicate the respective advantages of this two versions in different applications and also demonstrate the effectiveness of MoG GWLRTF compared with other competing methods.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 7

page 8

page 9

page 10

page 12

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

The problem of recovering a low dimensional linear subspace from high dimensional visual data naturally arises in the fields of computer vision, machine learning and statistics, and has drawn increasing attention in the recent years. Typical examples include representation and recognition of faces

[1, 2, 3, 4], structure from motion [5], recognition of 3D objects under varying pose [6], motion segmentation [7]

. In such contexts, the data to be analyzed usually can be formulated as high-order tensors, which are natural generalization of vectors and matrices. Existing approaches, including LRMF and RPCA, proceed by matricizing tensors into matrices and then applying common matrix techniques to deal with tensor problems. However, as shown in

[8], such matricization procedure fails to exploit the essential tensor structure and often leads to suboptimal procedure. Figure 1 illustrates the difference between the matrix based method and tensor based method in dealing with the high-order tensor data. The upper row is the matrix based factorization method, which needs to preliminarily matricize the tensor at the cost of losing data structure information; the lower row is our tensor based method which directly factorizes the tensor without destroying the spatial structures.

Fig. 1: Framework of high-order data processing by matrix techniques and robust tensor factorization method (take CP factorization for illustration). In the initial high-order data representation stage, tensorization better preserves the essential data structure compared with matricization. Furthermore, the integrated MoG model make the tensor factorization more robust to unknown complex noise.

Given a high-order tensor data, an efficient way to extract the underlying useful information is low-rank tensor factorization (LRTF), which aims to extract low-rank subspaces underlying those vector spaces such that the original tensor can be suitably expressed through reasonably affiliating these subspaces.

Traditionally, there are two definitions of tensor factorizations, i.e., CP factorization and Tucker factorization.

The CP factorization can be viewed as a higher-order generalization of the matrix singular value decomposition

[9, 10, 11, 12]

and has been widely applied to many fields, including image inpainting 

[13, 14], collaborative filtering [15], and data mining [16], etc. The idea of CP is to express a tensor as the sum of a finite number of rank-1 tensors. Mathematically, an -order tensor , with the integer indicating the dimension of along the -th order, is represented in the CP factorization form as

(1)

here, denotes the vector outer product and is assumed to be the rank of the tensor , denoted as rank(), which is defined as the smallest number of rank-1 tensors [17, 18].

Each element of the tensor has the following form:

(2)

where The mode (or factor) matrices refer to the combination of the vectors from the rank-1 components, i.e., and likewise for others.

The Tucker factorization is a form of higher-order principal component analysis 

[9, 10, 11, 19] and has been widely used in many applications [20, 21, 22, 23, 24, 25, 26]. It decomposes a tensor into a core tensor multiplied (or transformed) by a matrix along each mode, written as

(3)

where denotes the -mode matrix product and is the core tensor controlling the interaction between the mode matrices .

Elementwise, the Tucker decomposition in Eq. (3) is

(4)

Here the defined -rank of , denoted as rank, considers the mode- rank of tensors. Accordingly, we call a rank- tensor.

As mentioned in [27], there exist a number of other tensor factorizations but they are all related to the CP and the Tucker factorization. On account of this, our work for robust low rank tensor factorization is designed based on this two typical factorizations.

It is known that the canonical fit function for the LRTF is based on the Frobenius norm function which assumes the noise to follow a Gaussian distribution. However, for many real data, such as the fMRI neuroimaging data [28] and the video surveillance data [29], a relative large perturbation in magnitude only affects a relatively small fraction of data points, which often violates the Gaussian assumption and follows a Laplacian distribution instead.

Therefore, it is necessary to consider other loss function that is robust to Laplacian noise. To alleviate this problem, one commonly used strategy is to replace the Frobenius norm function (say, norm) by the -type norm [30, 31], which is known to be robust to gross Laplacian perturbations. Unfortunately, in many real applications, the noise often exhibits very complex statistical distributions rather than a single purely Gaussian or Laplacian noise [32]. This motivates us to consider more flexible modeling strategies to tackle such complex noise cases.

Under the framework of low-rank matrix factorization (LRMF), Meng and De la Torre [33] firstly proposed to model the noise as Mixture of Gaussians (MoG). They showed that the MoG model is a universal approximator to any continuous distribution, and hence could be capable of modeling a wider range of noise distributions. Along this line, Zhao et al. [34] further extended the MoG model to deal with robust PCA (RPCA) problem. Extensive experiments on synthetic data, face modeling and background subtraction demonstrated the merits of MoG model.

As such, to share the same light of matrix MoG model, we aim to introduce a novel MoG model to the tensor case for the LRTF task to overcome the drawbacks of existing models, which are only optimal for simple Gaussian or Laplacian noise.

The contributions of this paper can be summarized as follows: (1) As an extension of our last work [35], we propose a generalized low-rank subspace learning approach called generalized weighted low-rank tensor factorization (GWLRTF), i.e., the GWLRTF-CP and the GWLRTF-Tucker, which both can preserve the essential tensor structure; (2) For modelling complex noise, MoG is applied to the proposed GWLRTF model called generalized weighted low-rank tensor factorization integrated with MoG (MoG GWLRTF), i.e., the MoG GWLRTF-CP and the MoG GWLRTF-Tucker; (3) For solving the proposed model, we propose efficient algorithms to estimate the parameters under the EM framework and through the proposed algorithm of GWLRTF. Our strategy is different from not only the traditional EM algorithm for solving matrix/tensor decomposition models, but also conventional alternative least squares (ALS) techniques for solving other tensor factorization problems; (4) To further compare the performance between CP factorization and Tucker factorization in different real application problems, a series of synthetic and real data experiments are conducted. The source codes of our algorithm are published online: http://vision.sia.cn/our%20team/Hanzhi-homepage/vision-ZhiHan(English).html.

The rest of this paper is organized as follows. Section 2 describes the notation and common operations used in this paper. In Section 3, a generalized model of tensor factorization integrated with MoG is proposed for reconstructing low-rank tensor from high-order tensor with unknown noise. In Section 4, the established problem model is solved under the EM framework with new proposed tensor factorization algorithms. Extensive experiments are conducted on both synthetic data and real image data in Section 5.

Ii Notation and preliminaries

The notations and common operations used throughout the paper are defined as follows. Scalars are denoted by lowercase letters and vectors are denoted by bold lowercase letters with elements . Matrices are represented by uppercase letters with column vectors and elements . The calligraphic letters stand for the the high-order tensors. We denote an -order tensor as , where is a positive integer. Each element in it is represented as , where .

If tensor is a rank-1 tensor, then it can be written as the outer product of vectors, i.e.,

(5)

Each element of the tensor is the product of the corresponding vector elements which can be represented as:

(6)

The slice of an -order tensor is a matrix defined by fixing every index but two. For instance, the slice of a 3-order tensor has the form: frontal slices , lateral slices , horizontal slices . Meanwhile, each order of a tensor is associated with a ‘mode’ and the unfolding matrix of a tensor in each mode is obtained by unfolding the tensor along its corresponding mode. For example, the mode- unfolding matrix of , denoted as unfold. The inverse operation of the mode- unfolding is the mode- folding, represented as fold. The mode- rank of is defined as the rank of the mode- unfolding matrix rank.

The operation of mode- product of a tensor and a matrix forms a new tensor. Given tensor and matrix , their mode- product is calculated by with element

(7)

Given two same-sized tensors , their inner product is defined as:

(8)

The Frobenius norm is . The norm is to calculate the number of non-zero entries in , and the norm . To be noted, ,, and for any .

Iii MoG GWLRTF Model

In this section, a generalized weighted low-rank tensor factorization model integrated with MoG (MoG GWLRTF) for modelling complex noise is proposed. By applying MoG to model the noise elements of the input tensor, we obtain the log-likelihood optimization objective.

Firstly, taking the noise part (denoted as ) into consideration, the input tensor can be represented as:

(9)

where denotes the low-rank tensor and the corresponding elementwise form is

(10)

As MoG has the ability to universally approximate any hybrids of continuous distributions, it is adopted for modeling the unknown noise in the original data. Hence every follows an MoG and the distribution is defined as:

(11)

where is the mixing proportion with and . denotes the Gaussian distribution with mean

and variance

.

Therefore, the element in Eq. (10) follows a MoG distribution with mean and variance

. The probability of each element

in the input tensor can be represented as:

(12)

where . Correspondingly, the likelihood of can be written as

(13)

where is the index set of the non-missing entries of .

Our goal is to maximize the logarithmic form of Eq. (13) with respect to the parameters , i.e.

(14)

Note that the original problem can be viewed as a Gaussian Scale Mixtures (GSM) with assumed to be 0, which has been widely used in previous works [36, 37]. Therefore, Eq. (14) can be rewritten as:

(15)

Iv Algorithms under EM framework

In this section, through assuming a latent variable with higher dimension, we solve the above problem iteratively under the EM framework. We also design algorithms to solve the generalized weighted low-rank tensor factorization model for updating the low-rank tensor.

EM algorithm [38] is proven to be effective for solving the maximization problem of the log-likelihood function. Therefore, for solving Eq. (14), we assume a higher dimensional latent variable under the EM framework.

In the model, the factorized low-rank tensor components are shared by all the clusters of MoG and the mean for each cluster of the standard EM algorithm is represented by them. Thus our proposed algorithm will iterate between computing responsibilities of all Gaussian components (E Step) and maximizing the parameters and the low-rank tensor in the model (M Step).

E Step: A latent variable is assumed in the model, with and , representing the assigned value of the noise to each component of the mixture. Here we denote . The posterior responsibility of the -th mixture for generating the noise of can be calculated by

(16)

The M step maximizes the upper bound given by the E step with regard to :

(17)

This maximization problem can be solved by alternatively updating the MoG parameters and the factorized components of low-rank tensor as follows:

M Step to update : The closed-form updates for the MoG parameters are:

(18)

M Step to update : Re-write Eq. (17) only with regard to the unknown as follows:

(19)

Here denotes the Hadamard product (component-wise multiplication) and the element of is

(20)

The whole MoG GWLRTF optimization process is summarized in Algorithm 1.

0:     the original data represented in tensor form
0:  the recovered low-rank tensor
1:  Initialize , MoG number , small threshold .
2:  while not converged do
3:     E Step: Evaluate by Eq. (16).
4:     M Step for : Evaluate by Eq. (18)
5:     M Step for : Evaluate by solving the GWLRTF model , where is calculated by Eq. (20).
6:  end while
Algorithm 1 (Algorithm for MoG GWLRTF)

In the M Step, is evaluated by solving the GWLRTF model . Here, we introduce the two typical factorizations to the GWLRTF and the corresponding algorithms are given in detail in the following parts.

Iv-a Weighted low-rank tensor CP factorization (GWLRTF-CP)

The GWLRTF model of a 3-order tensor in the form of CP factorization can be written as

(21)

where

and are mode matrices with rank . is the weighted tensor which is composed by the standard variance of the input tensor elements.

Because of the effectiveness and implementation convenience of ALS, we adopt its idea to update of the tensor one at a time.

Suppose are data matrices. In order to stack each of the above matrix as a vector, we define the operator .

For each slice of the higher-order tensor, it can be viewed as a linear combination of the corresponding slices of all the rank-1 tensors. Different from other methods for solving the problem of LRTF, we stack each frontal slice of the higher-order tensor as a vector of a new matrix denoted as . Correspondingly, the vectorized horizontal slices and lateral slices are represented as and , respectively.

Firstly we have

(22)

Then taking term as an example, the vectorized frontal slice of the higher-order tensor can be written as follows:

(23)

For the -th frontal slice of the higher-order tensor, the vectorized corresponding slices of all the rank-1 tensors can be viewed as the -th element of the cell which can be represented as:

(24)

Then the -th vector of term can be updated as follows:

(25)

where represents the pseudo-inverse matrix of matrix , and denotes the transposed matrix of matrix .

Similarly, we have the term and updated as following:

(26)
(27)
(28)
(29)
(30)
(31)

The whole optimization process is summarized in Algorithm 2.

0:    The input tensor , initialized tensor factors , weighted tensor , number of iteration and the threshold .
0:  .
1:  while not converged do
2:     update with Eq. (23), (24), (25);
3:     update with Eq. (26), (27), (28);
4:     update with Eq. (29), (30), (31).
5:  end while
Algorithm 2 (GWLRTF-CP)

Iv-B Weighted low-rank tensor Tucker factorization (GWLRTF-Tucker)

By applying the Tucker factorization to the low-rank tensor in the GWLRTF model, we obtain the following GWLRTF-Tucker model

(32)

Through coordinate-wisely separating the original optimization problem into solving a sequence of scalar minimization subproblems, coordinate descent has exhibited its effectiveness in dealing with the convex optimization problems [39, 40, 41]. Based on this, we aim to coordinate-wisely optimize each entry of and in Eq. (32).

Update the mode matrices : Firstly, we reformulate Eq. (32) in the form of minimizing the function against only one of the unknown mode matrices (take for example) at a time with others fixed as

(33)

where

(34)

Unfolding the tensors along mode-, it can be reformulated as the following sub-problem

(35)

where

(36)

Then, the original problem Eq. (32) are separated into the following single-scalar parameter optimization sub-problems

(37)

Update the core tensor : Likewise, for the equivalent formulation of Eq. (32)

(38)

it can be rewritten as

(39)

where

Here we denote , , , then the optimization of Eq. (38) can be obtained by minimizing the following sub-problems

(40)

The solutions of Eqs. (37) and (40) can be exactly obtained by ALS. The whole optimization process is summarized in Algorithm 3.

0:    The input tensor , initialized tensor factors and , weighted tensor , number of iteration and the threshold .
0:  .
1:  while not converged do
2:     update the entries of mode matrices by solving Eq. (37);
3:     update each entry of the core tensor with Eq. (40);
4:  end while
Algorithm 3 (GWLRTF-Tucker)

V Experiments

MoG LRMF HaLRTC LRTA PARAFAC MSI DL CWM LRTF MoG GWLRTF-CP MoG GWLRTF-Tucker
E1 0.08 2.55e+02 4.40e+02 4.38e+02 2.98e+02 3.51e+02 1.61e-08 2.31e-12
E2 1.08e-04 6.50e+04 6.47e+02 6.47e+02 3.70e+02 3.93e+02 6.82e-19 1.43e-26
E3 0.09 2.68e+04 6.07e+02 5.95e+02 4.10e+02 4.63e+02 2.03e-08 2.92e-12
E4 0.57 3.76e+06 9.62e+02 9.48e+02 5.82e+02 5.34e+02 8.85e-19 1.85e-26
E1 1.24 2.55e+02 5.28e+02 5.30e+02 3.58e+02 4.38e+02 8.84e-09 1.10e-12
E2 0.02 6.50e+04 1.24e+03 1.23e+03 7.95e+02 8.07e+02 3.18e-19 3.82e-27
E3 7.36e+02 5.20e+04 9.93e+02 9.78e+02 6.54e+02 8.00e+02 1.90e-08 2.07e-12
E4 1.41e+02 7.06e+06 2.28e+03 2.22e+03 1.50e+03 1.56e+03 9.63e-19 8.95e-27
E1 5.01 2.55e+02 7.11e+02 6.63e+02 4.85e+02 5.57e+02 2.49e-07 8.84e-07
E2 0.66 6.50e+04 2.34e+03 2.21e+03 1.67e+03 1.90e+03 2.77e-16 3.59e-27
E3 1.51e+04 7.89e+04 1.86e+03 1.81e+03 1.25e+03 1.73e+03 1.12e-06 2.48e-12
E4 1.57e+03 1.09e+07 6.26e+03 6.21e+03 4.32e+03 6.22e+03 4.21e-15 1.24e-26
TABLE I: Predictive performance of competing methods with varied missing rate. In each experiments, the best performance is highlighted in bold and the second is underlined.
MoG LRMF HaLRTC LRTA PARAFAC MSI DL CWM LRTF MoG GWLRTF-CP MoG GWLRTF-Tucker
E1 32.3 2.55e+02 6.31e+02 6.41e+02 4.42e+02 4.69e+02 54.2 14.7
Gaussian E2 2.44 6.50e+04 1.55e+03 1.57e+03 1.17e+03 7.04e+02 5.84 1.62
Noise E3 11.4 1.05e+05 8.59e+02 8.53e+02 6.02e+02 6.09e+02 29.2 14.8
E4 72.0 1.41e+07 2.10e+03 2.07e+03 1.53e+03 8.81e+02 1.52 0.647
E1 4.20e+02 5.10e+02 1.05e+03 9.63e+02 7.70e+02 8.86e+02 6.93e+02 4.36e+02
Sparse E2 4.96e+02 1.30e+05 2.96e+03 2.63e+03 2.24e+03 2.47e+03 1.42e+03 1.25e+03
Noise E3 5.25e+03 1.02e+05 1.02e+03 1.05e+03 7.47e+02 8.41e+02 5.10e+02 6.82e+02
E4 1.04e+03 1.33e+07 2.17e+03 2.34e+03 1.64e+03 1.77e+03 4.33e+02 1.30e+03
E1 4.63e+02 5.10e+02 1.19e+03 1.15e+03 8.17e+02 1.07e+03 6.68e+02 4.46e+02
Mixture E2 6.05e+02 1.30e+05 3.90e+03 3.70e+03 2.58e+03 3.31e+03 1.37e+03 1.29e+03
Noise E3 1.23e+04 1.06e+05 1.13e+03 1.17e+03 7.85e+02 1.10e+03 4.59e+02 4.62e+02
E4 1.46e+03 1.50e+07 2.84e+03 2.94e+03 1.93e+03 2.91e+03 3.83e+02 5.72e+02
TABLE II: Reconstruction performance of competing methods with unknown noise. In each experiments, the best performance is highlighted in bold and the second is underlined.

In this section, we conduct extensive experiments on both synthetic data and real applications to validate the effectiveness of the proposed MoG GWLRTF, compared with MoG LRMF [33], HaLRTC [8], LRTA [42], PARAFAC [43], MSI DL [44], CWM LRTF [39]. Specifically, MoG GWLRTF-Tucker and MoG GWLRTF-CP are also demonstrated to further compare the performance of CP factorization and Tucker factorization in different applications. The synthetic experiments are designed to quantitatively assess our methods from: i) predictive performance over missing entries given an incomplete tensor data; ii) reconstruction performance given a both incomplete and noisy tensor data. Real data applications, i.e., single RGB image reconstruction, face modeling, multispectral image recovery and real hyperspectral image restoration, are further conducted to validate the effectiveness of the proposed algorithms.

V-a Synthetic Experiments

Fig. 2: Facade with small mixture noise. (a) Noisy image. (b)MoG LRMF. (c)HaLRTC. (d)LRTA. (e)PARAFAC. (f)MSI DL. (g)CWM LRTF. (h)MoG GWLRTF-CP. (i)MoG GWLRTF-Tucker. (j) Original image.
Fig. 3: Facade with mixture noise. (a) Noisy image. (b)MoG LRMF. (c)HaLRTC. (d)LRTA. (e)PARAFAC. (f)MSI DL. (g)CWM LRTF. (h)MoG GWLRTF-CP. (i)MoG GWLRTF-Tucker. (j) Original image.
Facade MoG LRMF HaLRTC LRTA PARAFAC MSI DL CWM LRTF MoG GWLRTF-CP MoG GWLRTF-Tucker
PSNR 24.34 23.43 13.59 13.37 13.53 24.80 25.65 25.67
small RSE 0.1169 0.1298 0.4026 0.4129 0.4062 0.1109 0.1005 0.1003
noise FSIM 0.8954 0.9407 0.8318 0.7402 0.8258 0.9435 0.9539 0.9454
PSNR 22.18 14.20 18.51 16.95 16.71 22.82 23.69 24.46
mixture RSE 0.1499 0.3755 0.2287 0.2737 0.2817 0.1393 0.1260 0.1127
noise FSIM 0.8525 0.6003 0.7667 0.7101 0.7310 0.9117 0.9268 0.9310
TABLE III: Facade reconstruction performance of competing methods with mixture noise. In each experiments, the best performance is highlighted in bold and the second is underlined.
Fig. 4: Sampling images of the first people under 9 illuminations.
Fig. 5: Face modeling results by different methods. (a)Original face images. (b)MoG LRMF. (c)HaLRTC. (d)LRTA. (e)PARAFAC. (f)MSI DL. (g)CWM LRTF. (h)MoG GWLRTF-CP. (i)MoG GWLRTF-Tucker.
Fig. 6: Ten randomly selected bands of strawberries. (a) Noisy bands. (b) Original bands. (c) Corresponding bands recovered by MoG GWLRTF-CP. (d) Corresponding bands recovered by MoG GWLRTF-Tucker.
Fig. 7: The 31st band of multispectral images. (a) Noisy band. (b)MoG LRMF. (c)HaLRTC. (d)LRTA. (e)PARAFAC. (f)MSI DL. (g)CWM LRTF. (h)MoG GWLRTF-CP. (i)MoG GWLRTF-Tucker. (j) Original band.
MoG LRMF HaLRTC LRTA PARAFAC MSI DL CWM LRTF MoG GWLRTF-CP MoG GWLRTF-Tucker
PSNR 19.26 8.444 16.51 15.84 18.47 19.94 22.17 22.45
Jelly RSE 0.5169 1.773 0.7003 0.7565 0.5588 0.4720 0.3652 0.3535
beans FSIM 0.8101 0.5312 0.7025 0.6487 0.8213 0.8506 0.8864 0.9004
PSNR 18.90 8.725 18.76 16.26 19.20 21.75 27.29 24.48
Paints RSE 0.4022 1.336 0.4209 0.5607 0.3998 0.2983 0.1576 0.2177
FSIM 0.8543 0.4750 0.7827 0.6367 0.8182 0.9165 0.9514 0.9427
PSNR 16.45 7.480 18.40 16.54 19.15 22.72 26.17 24.24
Flowers RSE 0.9073 2.522 0.7173 0.8888 0.6583 0.4364 0.2933 0.3511
FSIM 0.8018 0.4172 0.8073 0.5271 0.8220 0.9126 0.9153 0.9139
PSNR 15.32 7.109 18.59 17.01 19.43 23.20 25.22 23.58
Egyptian RSE 1.563 3.847 1.026 1.231 0.9308 0.6032 0.4779 0.5271
statue FSIM 0.7834 0.3696 0.8245 0.4243 0.8142 0.9187 0.9439 0.9233
PSNR 12.05 7.911 18.29 16.23 18.95 21.46 23.95 23.33
Chart RSE 0.8633 1.388 0.4201 0.5327 0.3895 0.2919 0.2191 0.2353
FSIM 0.8069 0.4534 0.7729 0.5366 0.8092 0.8901 0.9221 0.9402
PSNR 22.22 11.56 19.92 17.14 21.33 21.92 25.07 20.98
Beers RSE 0.2389 0.8150 0.3114 0.4287 0.2646 0.2474 0.1722 0.2756
FSIM 0.7659 0.4155 0.7535 0.4423 0.8214 0.9430 0.9200 0.8748
PSNR 19.02 8.380 18.74 16.49 18.89 21.66 26.66 20.79
Glass RSE 0.6314 2.025 0.6140 0.7961 0.6038 0.4389 0.2467 0.4541
FSIM 0.6990 0.4690 0.7726 0.5738 0.7287 0.9075 0.9475 0.7946
PSNR 17.06 7.762 19.22 16.78 19.54 18.99 24.80 23.41
Strawberries RSE 0.7630 2.149 0.5745 0.7607 0.5534 0.5900 0.3021 0.3652
FSIM 0.7906 0.3997 0.8059 0.4932 0.8128 0.9229 0.9283 0.9137
TABLE IV: Multispectral image restoration performance of competing methods with mixture noise. In each experiments, the best performance is highlighted in bold and the second is underlined.

The synthetic tensor is generated as follows: firstly, matrices

are drawn from a standard normal distribution, i.e.,

, the vectors of the matrices comply with a standard normal distribution ; Secondly, construct the true tensor by , and set the size to and CP rank . Then we conduct two synthetic experiments: i) for validating the predictive performance, we vary the true tensor missing entries rate (, , ) ; ii) for verifying the reconstruction performance, we randomly choose missing entries of the true tensor and further add certain type of noise to it as the following procedure: (1) Gaussian noise ; (2) Sparse noise:

of the non-missing entries with the uniformly distribution over [-5,5]; (3) Mixture noise:

of the non-missing elements with the uniformly distribution over [-5,5], and of the rest non-missing with Gaussian noise and the rest with . The performance of each method is quantitatively assessed by the following measurements as used in [33]:

where and are used to denote the noisy tensor and the recovered tensor, respectively. As mentioned in [33], and are the optimization objectives of existing methods, which assess how the reconstruction complies with the noisy input, but and are more meaningful for evaluating the correctness of the clean subspace recoveries. Therefore, we pay more attention to the quantitative indices of and . In the tables, the first and second best performances are marked out with bold and underline, respectively.

The performance of each method in the synthetic experiments are summarized in Table I and Table II, respectively. From Table I we can see that, in the case of varying data missing rate, our methods always have a relative better predictive performance in all evaluation terms. When the data is only disturbed by a single distribution noise, i.e., Gaussian noise or sparse noise, other methods can also obtain a fairly well results. However, when the noise becomes complex, our methods still have a good reconstruction performance in this case, as shown in Table II.

V-B Single RGB Image Reconstruction

The benchmark colorful building facade image is used in this section to evaluate the performance of different methods in image reconstruction. Note that the colorful image can be viewed as a 3-order tensor of size . Two groups of experiments are considered here.

Firstly, the facade image rescaled to [0,255] is randomly sampled with missing entries and then added with a relative small scale mixture noise: of the non-missing pixels with the uniformly distribution over [-35,35], of the rest non-missing pixels with Gaussian noise and the rest with another uniformly distribution .

The visual effect of each methods are demonstrated in Figure 2. For better visual comparison, we have also provided a zoom-in version of a local region in Figure 2. From the results we can see that the MoG GWLRTF-Tucker method has a better reconstruction performance than the matrix based methods and the traditional tensor based methods in reconstructing the image details.

Besides the visual effect, quantitative assessments are also reported. Three quantitative image quality indices are adopted to evaluate the performance of each method: peak signal-to-noise ratio (PSNR), relative standard error (RSE) and feature similarity (FSIM)

[45]. Larger values of PSNR and FSIM and smaller values of RSE mean a better restoration results.

The quantitative results obtained by each method are given in Table III (the upper row), and it shows that the MoG GWLRTF-Tucker method is superior to all the other methods except the MoG GWLRTF-CP method which performs nearly as well in this small scale mixture noise case.

Secondly, in order to further compare the reconstruction ability of each method, we add a larger mixture noise to the facade image. The image is first rescaled to [0,1] and then a larger mixture noise added as in the synthetic experiments: missing entries, of the non-missing pixels with the uniformly distribution over [-5,5], of the rest non-missing pixels with Gaussian noise and the rest with another uniformly distribution .

Both the visual and the quantitative results are provided, as in Figure 3 and Table III (the lower row). In Figure 3, the zoom-in version of a local region for comparison is given. From the results, we can see that with the increasing of the mixture noise scale, the MoG GWLRTF-Tucker method performs much better than the MoG GWLRTF-CP method in reconstructing the image details. And they are both superior to the other methods which lose a lot of image structure information.

V-C Face Modeling

In this section, we assess the effectiveness of the proposed methods in face modeling with different objects under different illuminations. It is different from the traditional methods in dealing with face modeling problem which always only focus on one kind of object under different illuminations. The dataset is the ensemble subset of the Extended Yale B database [46], containing 45 faces of 5 objects and illuminations with size . The original tensor is thus generated with size . All the competing methods used above are also compared here. Considering that some tensor methods are only designed to dealing with 3-order tensors and the matrix methods are originally designed to solve matrix data, in this case the original 4-order tensor is vectorized into the 3-order tensor with size and matrix data with size before processing, respectively. The sampling images from this data subset are plotted in Figure 4.

Typical