Graph Regularized Non-negative Matrix Factorization By Maximizing Correntropy

Non-negative matrix factorization (NMF) has proved effective in many clustering and classification tasks. The classic ways to measure the errors between the original and the reconstructed matrix are l_2 distance or Kullback-Leibler (KL) divergence. However, nonlinear cases are not properly handled when we use these error measures. As a consequence, alternative measures based on nonlinear kernels, such as correntropy, are proposed. However, the current correntropy-based NMF only targets on the low-level features without considering the intrinsic geometrical distribution of data. In this paper, we propose a new NMF algorithm that preserves local invariance by adding graph regularization into the process of max-correntropy-based matrix factorization. Meanwhile, each feature can learn corresponding kernel from the data. The experiment results of Caltech101 and Caltech256 show the benefits of such combination against other NMF algorithms for the unsupervised image clustering.

READ FULL TEXT VIEW PDF

Authors

page 1

page 2

page 3

page 4

09/29/2017

A Nonlinear Orthogonal Non-Negative Matrix Factorization Approach to Subspace Clustering

A recent theoretical analysis shows the equivalence between non-negative...
10/25/2021

Fast Rank-1 NMF for Missing Data with KL Divergence

We propose a fast non-gradient based method of rank-1 non-negative matri...
08/19/2012

Adaptive Graph via Multiple Kernel Learning for Nonnegative Matrix Factorization

Nonnegative Matrix Factorization (NMF) has been continuously evolving in...
09/07/2019

A Non-Negative Factorization approach to node pooling in Graph Convolutional Neural Networks

The paper discusses a pooling mechanism to induce subsampling in graph s...
09/10/2015

A deep matrix factorization method for learning attribute representations

Semi-Non-negative Matrix Factorization is a technique that learns a low-...
11/29/2020

Clustering US States by Time Series of COVID-19 New Case Counts with Non-negative Matrix Factorization

The spreading pattern of COVID-19 differ a lot across the US states unde...
01/08/2016

Song Recommendation with Non-Negative Matrix Factorization and Graph Total Variation

This work formulates a novel song recommender system as a matrix complet...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Given a collection of images, the clustering algorithms attempt to group the dataset into multiple clusters such that images in the same cluster are similar to each other in terms of the semantics information. In this process, a good feature extraction method is vital to the clustering performance. Essentially, clustering (e.g.

k

-means) or classification algorithms (e.g. support vector machines

[1, 2, 3, 4]) map the low-level image features to semantic information. These algorithms have a variety of applications in different areas[5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]. If the extracted image features can reflect the latent semantic concepts, we believe it can somehow better boost the clustering/classification performance.

Recently, non-negative matrix factorization (NMF) has proven to be a powerful matrix factorization tool for data representation. Matrix factorization decomposes the original matrix into multiple low-rank matrices, such that their product approximates . In NMF, is decomposed into two non-negative matrices (basis matrix) and (coefficient matrix). The vectors in

spans a latent semantic space where each basis vector defines a semantic topic. By doing so, unlike other matrix factorization methods, such as singular value decomposition, that interpret the data as both additive and subtractive combination of semantics NMF only allows additive relationship. This constraint has proved to be closer to the way how humans perceive and understand the data

[20, 21, 22]. Based on this methodology, we can map the low-level image features into the additive combination of latent semantics for the clustering.

Extensive work has been done to investigate the NMF algorithm for different clustering tasks. Generally, the matrix decomposition is done by minimizing the errors between original and reconstructed matrix using distance or KL divergence [23, 24, 25]. One of the concerns is that they are linear similarity measures, which may not be suitable for data with nonlinear structure, like images [26]. A possible solution is to use nonlinear similarity measure to model the error (e.g. kernlized nonlinear mapping [27, 28]). Among these nonlinear methods, we are especially interested in the NMF based on maximizing correntropy criterion (MCC). Correntropy is a generalized nonlinear measure between two variables. MCC-based methods have proved effective in many areas, e.g. cancer clustering [29]

, face recognition

[30] and software defect prediction[31]. Another approach is preserving the geometric structure of data based on the manifold assumption of data distribution. To be more specific, the authors in [32, 33] exploit the local invariance and encode such geometrical information by constructing a nearest neighbor graph. Thus, at least two factors are included in modeling this process: the distance between and based on distance (or KL divergence) at low-level feature space, and the graph regularization. Furthermore, the authors in [34] incorporate such intrinsic geometric information in multiple manifolds.

In this paper, we propose a graph regularized NMF algorithm based on maximizing correntropy criterion for unsupervised image clustering. We can leverage MCC to properly model the errors in low-level feature space. Furthermore, the graph regularization can keep the correct geometric information in our factorization process. To our knowledge, this is the first work that combines MCC and graph regularization in NMF. Meanwhile, our proposed algorithm can self-learn the kernels from the data to evolve for clustering task. The results on two datasets (Caltech101 and Caltech256) shows the supremacy of our proposed algorithm over other variants of NMF algorithms.

The paper is organized into six sections: related work is discussed in Section II. In section III, our proposed model and the convergence proof are explained. The experiment settings are mentioned in Section IV. The results and further discussions are shown in Section V. The last section is the conclusion.

Ii Related Work

K

-means is perhaps one of the most popular clusteirng algorithms. This algorithm finds the match cluster for each data point by minimizing distance between the data point and existing clusters’ centroids. The clusters’ centroids are updated at each iteration when a new cluster member is introduced. Similar to Naive Bayes and Gaussian mixture model

[35, 36] that have their own assumption of data distribution, K-means algorithm assumes that data within the same class should also be close to each other in the feature space. One potential concern is that if the dataset properties don’t follow such assumptions, the accuracy of the algorithm may be at risk.

To solve the above issue, some methods (e.g. Latent Semantics Indexing (LSI) [37]) try to map the data into the latent semantics space where each basis axis in such space represents one type of semantic information of the dataset. Thus, we represent each image as a combination of multiple semantics information. Then we can use any clustering or classification algorithm to directly work on these semantic features. In LSI, the coefficients of the combination could be either positive or negative. However, a negative coefficient could be difficult to interpret. Meanwhile, LSI algorithm requires the basis in semantic space to be orthogonal from each other. This property, on one hand, ensures that we will have a unique solution [38]; on the other hand, it indicates that the semantics basis are distinguished from each other, which, however, is not always the case.

NMF is similar to LSI algorithm in the way that they both map the dataset into the latent feature space. However, the basis in latent feature space are not necessarily orthogonal from each other in NMF. Meanwhile, each basis now corresponds to one topic of the dataset. The key benefit is that we can easily find the category of the image by simply investigating the largest components in the latent space. Furthermore, every element in the two decomposed low-rank matrices are non-negative. This additive combination makes it easier to interpret an image intuitively.

NMF has been investigated heavily for image clustering [23, 24, 39, 40]. The classic NMF mainly targets on minimizing the distance or KL divergence. One issue with it is that it may not be able to handle the nonlinear data. Different algorithms have been proposed to solve this problem. One way is to use nonlinear distance measure, such as correntropy [29, 30], kernelized distance [28], etc. Another approach is preserving the geometric structure of data based on the manifold assumption. For example, the authors in [32] exploit the local invariance and encode such geometrical information by constructing a nearest neighbor graph. In a similar way, the authors in [41, 42]

incorporate a portion of true labels into the graph-regularized method (i.e. semi-supervised learning). The key ideas are to map samples with same ground-truth labels onto the same point in the semantic feature space. Some other solutions can be found in

[43].

Iii Algorithms

Iii-a NMF algorithm

Assuming we have a matrix . NMF allows us to factorize into two non-negative matrices and , where the product of approximates original matrix .

Each column in is a feature vector of the image with elements. Thus, represents the whole dataset with images. Conventionally, we name as basis matrix such that each column forms a basis vector of the semantic feature space, and as coefficient matrix. Hence, an image is further represented as the additive combination of weighted basis vectors in semantic space. norm distance (Equation 1) and Kullback-Leibler (KL) divergence (Equation 2) are two commonly-used measures of the similarity between original matrix and the product of and , where norm distance is:

(1)

and Kullback-Leibler (KL) divergence is:

(2)

Based on different similarity measures, we are able to solve the factorization problem by minimizing the errors between and .

Iii-B MCC algorithm

The authors in [29]

propose a new method to quantify the NMF by maximum correntropy criteria (MCC) for cancer clustering. Correntropy measures the generalized similarity between two random variables,

and , as defined in Equation 3, where is the kernel function and is the expectation.

(3)

More precisely, correntropy models the expected differences between two random variables after mapping through kernel function. Unlike

distance and KL divergence, such a modelling method can handle the nonlinear cases properly. Without knowing the joint distribution of

and

, we can simply estimate the expectation by taking a average (shown in Equation 

4):

(4)

Instead of using distance or KL divergence as defined Equation 1 and 2, respectively, we try to find the basis matrix and coefficient matrix , whose product is a well approximation of , by maximizing their correntropy on a feature-by-feature basis to allow for weighting each feature differently. For each feature, the kernel function can be calculated as:

(5)

Hence, the correntropy maximization problem is expressed as:

(6)

To simplify the calculations without losing generality, they choose the Gaussian kernel function as :

(7)

After substituting Equation 7 back into Equation 6, the basis and coefficient matrices can be derived by solving:

(8)

Then the convex conjugate function and auxiliary variables are introduced to solve the above optimization problem based on the Proposition 1. According to the theory of convex conjugate functions, the above optimization problem is equivalent to:

Proposition 1. There exists a convex conjugate function of s.t.

(9)

and for a fixed , the supremum is reached at .

Iii-C GRNMF algorithm

The authors in [32] approach the nonlinear case based on the local invariance assumption [44, 45] that if two images belong to the same topic, then they should be close to each other in both original (denoted as and ) and mapped feature spaces (denoted as and ). To model this intrinsic geometric information, manifold learning theory [44] has proved to be effective by constructing a nearest neighbor graph on the data points. We can think the distribution of images as a graph, where each image is a node. If one is one of nearest neighbors of

and vice versa, then we can build an edge between these two images with a weight. Hence, we can derive an affinity matrix

where indicates the weight between image and . Then, the graph regularization is represented by Equation 10.

(10)

where is a diagonal matrix and , and is the graph Lapacian.

Iii-D MCCGR NMF algorithm

In this paper, we propose a new method that combines MCC algorithm and GRNMF. MCC algorithm shows good decomposition performance directly on the low-level features while the graph regularization can assist in this factorization process by considering the geometrical information of data. For convenience, we name our proposed method as MCCGR in this paper.

The MCCGR algorithm tries to maximize the problem as defined in Equation 11, where is the weight of graph regularizer.

(11)

The optimization problem can be solved by Expectation-Maximization-like method. Starting from initial value of

and , is computed in expectation step (E-step). Conditional on the value, we can update the and values in maximization step (M-step). The process is called one iteration. This iterative process stops until it converges.

Optimize : Starting from the estimated and from last M-step (or random values in the 1st iteration), of the t-th iteration is computed as:

(12)

where

Optimize , : Conditional on the new from last step, we compute the new basis and coefficient matrix, denoted as and respectively, by maximizing the object function:

(13)

where is the operator that converts a vector into a diagnal matrix.

According to the dual objective function, we have:

(14)

Thus, the dual problem of Equation 13 can be converted into following optimization problem:

(15)

We apply the Lagrange method to solve the optimization problem defined in Equation 15. Let the elements of matrices and be the corresponding Lagrange multipliers for the non-negative conditions of and . Then we can express the Lagrange optimization problem as:

(16)

The partial derivatives of w.r.t. and are shown in Equation 18:

(17)

Based on Karush-Kuhn-Tucker optimal conditions ( and ), we can have:

(18)

Hence, the basis matrix and coefficient matrix are updated as follows:

(19)
(20)

The overall iterative solution is summarized in the Algorithm 1.

INPUT: , Max_iteration
Initialize and
for  Max_iteration do
     Update the auxiliary variables from Equation 12
     Update from Equation 19
     Update from Equation 20
end for
RETURN ,
Algorithm 1 MCCGR NMF Algorithm.

Iii-E Proof of convergence

We now prove that the objective function in Equation 14 is non-increasing under the update rules defined in Equation 19 and 20.

Firstly, we introduce the auxiliary function.

Definition 1. is an auxiliary function for if the following conditions hold:

(21)

Next, we introduce a lemma that involves auxiliary function.

Lemma 1. If is an auxiliary function of , then is non-increasing under the following update rule from iteration to :

(22)

The proof of the Lemma 1 can be found in [25]. Thus, if we can find an auxiliary function for the objective function (Equation 14) and an update rule that satisfies Lemma 1, we can show that is non-increasing.

We use to represent any element in , and to represent the part of the that is only related to . We can derive following two equations:

(23)
(24)

Lemma 2. The following function is an auxiliary function of which is only relevant to variable :

(25)

Proof: Obviously, . Now we show how to prove . First, we expand the Taylor series of (as shown in Equation 26).

(26)

To show , we need to prove that the following relation holds:

(27)

We have

(28)

and

(29)

Hence, inequality 27 holds and .

If we replace in 22 by the results of 25, we have following update rule:

(30)

Thus, (shown in Equation 25) is an auxiliary function for , and is non-increasing under this update rule.

In a similar vein, we can show that is non-increasing under the update rule in Equation 19.

Iv Experiment Settings

We use two datasets for testing: Caltech101 [46] and Caltech256 [47]. We follow the way in [48] to extract features. First, SIFT algorithm [49] is used to extract the local invariant features of each image in grayscale. Then all SIFT descriptors of both datasets are gathered together to form a dictionary, which is used for vector quantization for all images. Two types of dictionaries are used with different numbers of codewords: 300 and 1000. Thus, each image is represented as an either 300- or 1000-dimension feature vector.

We randomly select categories and use all images from these categories to form the testing matrix . Intuitively, the value of controls the decomposition of the matrix. Fundamentally, it determines the number of basis vectors that span the latent semantic space. To fully investigate the efficiency of the proposed algorithm, the number of cluster ranges from 1 to 10 for both datasets. We repeat this process 50 times to reduce the potential effect of random errors in experiment results, since the performance of NMF algorithm is affected by the initial values of the iterative process. The and are initialized by random values. The output of each algorithm is derived from taking the average of 50 runs.

To evaluate the decomposition performance, we directly apply k-means clustering method to sub-group into clusters. Essentially, each column of with elements are the new feature vector of one image in the latent semantic spaces after matrix decomposition. K-means will assign each image with a label. We compare the label from k-means to the original ground-truth label to evaluate the clustering results. The performance is measured by accuracy (shown in Equation  31). To find the correspondence between the class from ground-truth data and the label by k-means, we use Kuhn-Munkres algorithm [50].

(31)

where returns 1 if ; otherwise 0.

Besides accuracy, we also use normalized mutual information (NMI) as a measure. Mutual information is to measure how close two clusters are. Given a randomly selected image and two clusters of images, and , we denote and

as the probability that this image belongs to

and , respectively, and as the joint probability that this image is in both clusters at the same time. Then, the mutual information is defined as follows:

(32)

Similar to [32], we adopt the normalized mutual information (defined in Equation. 33) by considering the entropy.

(33)

where is the entropy of .

V Results and Discussions

V-a Parameter selection

(a) Caltech101, 300 codewords
(b) Caltech256, 300 codewords
(c) Caltech101, 1000 codewords
(d) Caltech256, 1000 codewords
Fig. 1: The effects of of MCCGR algorithm on the accuracy.

One important parameter we need to highlight is the weight of graph regularization (). indicates the relative importance of graph regularization in our decomposition process. Thus it’s worthwhile to systematically investigate the effects of on the accuracy.

We vary the values of in the range of [1, 10000] and fix (i.e. two-class clustering). We randomly choose 2 classes of images for testing with different , and then repeat this process 50 times to reduce the potential effects of randomness in our experiment. The average of the 50 runs can be found in Figure 1.

In this figure, -axis in log-scale demonstrates the choice of , while -axis indicates the corresponding accuracy. As we can see, the MCCGR algorithm is robust enough to the change of . The accuracy fluctuates in a small range as increases from 1 to 10000.

V-B Convergence study of MCCGR NMF

(a) Caltech101, 300 codewords
(b) Caltech256, 300 codewords
(c) Caltech101, 1000 codewords
(d) Caltech256, 1000 codewords
Fig. 2: Convergence curves of MCCGR algorithm on Caltech101 and Caltech256 datasets, with 300- and 1000- codewords options.

We also plot the convergence curves (Figure 2) of our proposed MCCGR algorithm for both datasets with different codeword settings. The -axis shows the objective function value while -axis means the iteration number. As we can see from the figures, the proposed algorithm converges quickly within less than 100 iterations. In most cases, it needs less than 50 iterations.

V-C Comparison with other NMF algorithms

(a) Caltech101, 300 codewords
(b) Caltech256, 300 codewords
(c) Caltech101, 1000 codewords
(d) Caltech256, 1000 codewords
Fig. 3: Accuracies on Caltech101 and Caltech256 datasets, with 300- and 1000- codewords.
K
Caltech101 Caltech256
GRNMF L2 MCC MCCGR PG GRNMF L2 MCC MCCGR PG
2 0.735 0.739 0.723 0.770 0.728 0.691 0.702 0.702 0.710 0.690
3 0.584 0.590 0.580 0.625 0.590 0.546 0.551 0.547 0.572 0.550
4 0.537 0.535 0.497 0.579 0.503 0.458 0.469 0.462 0.493 0.447
5 0.459 0.456 0.434 0.477 0.429 0.396 0.414 0.421 0.435 0.397
6 0.439 0.419 0.401 0.468 0.376 0.372 0.385 0.382 0.400 0.361
7 0.402 0.409 0.397 0.416 0.369 0.337 0.353 0.342 0.363 0.322
8 0.379 0.375 0.359 0.383 0.333 0.310 0.325 0.315 0.339 0.297
9 0.361 0.352 0.355 0.375 0.313 0.291 0.301 0.305 0.316 0.280
10 0.339 0.325 0.324 0.343 0.278 0.280 0.297 0.297 0.289 0.249
TABLE I: Clustering accuracies on Caltech101 and Caltech256 datasets, with 300 codewords in dictionary
K
Caltech101 Caltech256
GRNMF L2 MCC MCCGR PG GRNMF L2 MCC MCCGR PG
2 0.703 0.696 0.697 0.754 0.687 0.661 0.659 0.690 0.712 0.628
3 0.593 0.569 0.580 0.621 0.551 0.529 0.538 0.563 0.577 0.507
4 0.502 0.488 0.484 0.520 0.450 0.453 0.456 0.462 0.512 0.413
5 0.461 0.439 0.423 0.491 0.382 0.411 0.417 0.422 0.449 0.382
6 0.438 0.423 0.414 0.458 0.374 0.366 0.376 0.391 0.397 0.337
7 0.409 0.402 0.402 0.423 0.328 0.344 0.352 0.350 0.374 0.274
8 0.396 0.392 0.389 0.415 0.259 0.318 0.330 0.339 0.353 0.235
9 0.348 0.349 0.346 0.367 0.207 0.289 0.303 0.303 0.302 0.200
10 0.356 0.329 0.333 0.358 0.187 0.283 0.289 0.297 0.291 0.146
TABLE II: Clustering accuracies on Caltech101 and Caltech256 datasets, with 1000 codewords in dictionary
(a) Caltech101, 300 codewords
(b) Caltech256, 300 codewords
(c) Caltech101, 1000 codewords
(d) Caltech256, 1000 codewords
Fig. 4: MIhat on Caltech101 and Caltech256 datasets, with 300- and 1000- codewords.
K
Caltech101 Caltech256
GRNMF L2 MCC MCCGR PG GRNMF L2 MCC MCCGR PG
2 0.247 0.250 0.232 0.260 0.235 0.149 0.166 0.185 0.145 0.142
3 0.244 0.249 0.233 0.259 0.241 0.162 0.170 0.167 0.188 0.167
4 0.244 0.266 0.227 0.276 0.242 0.171 0.181 0.177 0.209 0.160
5 0.251 0.250 0.220 0.263 0.224 0.177 0.181 0.184 0.203 0.159
6 0.269 0.259 0.232 0.279 0.206 0.187 0.189 0.190 0.206 0.163
7 0.272 0.268 0.248 0.281 0.215 0.171 0.178 0.174 0.194 0.147
8 0.267 0.261 0.241 0.285 0.220 0.164 0.177 0.170 0.190 0.153
9 0.267 0.259 0.255 0.275 0.223 0.175 0.177 0.181 0.188 0.155
10 0.267 0.262 0.251 0.271 0.205 0.173 0.181 0.189 0.186 0.134
TABLE III: Clustering NMI on Caltech101 and Caltech256 datasets, with 300 codewords in dictionary
K
Caltech101 Caltech256
GRNMF L2 MCC MCCGR PG GRNMF L2 MCC MCCGR PG
2 0.202 0.192 0.189 0.228 0.185 0.113 0.115 0.174 0.167 0.079
3 0.264 0.236 0.241 0.285 0.217 0.168 0.169 0.195 0.204 0.129
4 0.237 0.224 0.221 0.258 0.165 0.161 0.165 0.174 0.180 0.109
5 0.226 0.224 0.212 0.241 0.158 0.184 0.186 0.191 0.194 0.139
6 0.267 0.265 0.246 0.276 0.199 0.177 0.182 0.194 0.196 0.137
7 0.264 0.263 0.262 0.279 0.176 0.175 0.181 0.190 0.198 0.095
8 0.288 0.291 0.278 0.301 0.130 0.180 0.186 0.197 0.205 0.082
9 0.251 0.259 0.253 0.267 0.096 0.169 0.179 0.184 0.181 0.068
10 0.267 0.269 0.258 0.277 0.085 0.181 0.186 0.198 0.190 0.023
TABLE IV: Clustering NMI on Caltech101 and Caltech256 datasets, with 1000 codewords in dictionary

We compare our proposed method against 5 other NMF algorithms: the original NMF method with distance [25], gradient descent-constrained least squares (GS) [24], Projected Gradient NMF (PG) [39], GRNMF and MCC NMF. To make a fair comparison, the initial values of and are the same for all algorithms at each run. There are many ways to construct the affinity matrix. Commonly-used ways of graph construction are 0-1 weighting, heat kernel weighting, and histogram intersection kernel weighting. As discussed in [32], a reasonable weighting method is important to the performance of methods involving graph regularization. We use 0-1 weighting for GRNMF and MCCGR NMF algorithms due to its simplicity. We believe other weighting methods may outperform this one. However, the comparison between weighting schemas is beyond the scope of this paper.

The comparison results of accuracy for can be found in Table I and II, and Figure 3. The NMI results are shown in Table III and IV, and Figure 4. As to the accuracy, the best value of each is bold. From the two tables we can see the supremacy of our proposed method. MCCGR outperforms the rest algorithms in most cases when . As to the cases that MCCGR is not the best, its performance is second to the best. GRNMF and MCC achieve the second best performance next to MCCGR in most cases. The performance of PG drops quickly as increases. The rest algorithms fall in between MCCGR and PG. If we look further at the results in Figure 3, when , an obvious observation is that the accuracy of all algorithms decreases as increases. However, MCCGR is very robust to the increment of the number of clusters. When it comes to the NMI measure, we can also find that the proposed MCCGR algorithm achieves the best performance among all.

There are several reasons we believe MCCGR is superior:

  • Unlike distance, MCCGR uses nonlinear kernel (correntropy) to measure the reconstructed errors. Similarity measurement based on linear kernel may not be suitable to capture the nonlinear manifold structure of data, like images, as discussed in [26].

  • Compared with other NMF algorithms that also map the features into nonlinear dataspace (e.g. the projected gradient kernel method in [51]), our MCCGR learns different kernels of different data features. This adaptive process somehow helps MCCGR correctly cluster most images into the right partition. Based on the experiment results of two datasets, the usage of correntropy as similarity measure shows its advantage in image clustering task.

  • The graph regularizer ensures that the algorithm preserves the local invariant information during factorization. Thus, the NMF on the manifold further improves the selective ability of our proposed algorithm.

Vi Conclusion

In this paper, we presented a graph regularized non-negative matrix factorization method by maximizing the correntropy between the original and the approximated matrices for unsupervised image clustering task. The MCC algorithm can properly model the reconstructed errors in low-level feature space, while the graph regularization ensures that the proposed algorithm preserves the local invariance information. We investigated the effects of different parameter selections on the accuracy, proved and demonstrated the convergence of the proposed algorithm, and tested the proposed method against other NMF algorithms on Caltech101 and Caltech256 datasets with multiple feature extraction settings. The average of the 50 repeated runs shows the benefits of using this graph-regularized nonlinear similarity measure for image representation. In future, we will investigate the possibility of our proposed method in medical instrument[52], mechanical instrument[53] and other related areas [54, 55, 56, 57, 58, 59, 60].

Acknowledgement

This work was partially supported by National Natural Science Foundation of China under Grant No.61273217, 61175011 and 61171193, the 111 project under Grant No.B08004.

References

  • [1]

    J. A. Suykens and J. Vandewalle, “Least squares support vector machine classifiers,”

    Neural processing letters, vol. 9, no. 3, pp. 293–300, 1999.
  • [2] O. Chapelle, P. Haffner, and V. N. Vapnik, “Support vector machines for histogram-based image classification,” Neural Networks, IEEE Transactions on, vol. 10, no. 5, pp. 1055–1064, 1999.
  • [3] Y. Zhou, L. Li, T. Zhao, and H. Zhang, “Region-based high-level semantics extraction with cedd,” in Network Infrastructure and Digital Content, 2010 2nd IEEE International Conference on.   IEEE, 2010, pp. 404–408.
  • [4] L. Li, Y. Zhou, and H. Zhang, “Adaptive learning of region-based plsa model for total scene annotation,” in International Conference on Information and Multimedia Technology, 2010.
  • [5] C. Fang, Y. Xu, and D. N. Rockmore, “Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias,” in

    International Conference on Computer Vision

    , 2013.
  • [6] C. Fang and L. Torresani, “Measuring image distances via embedding in a semantic manifold,” in European Conference on Computer Vision (ECCV), Oct. 2012, pp. 402–415.
  • [7] Z. Qin, C. R. Shelton, and L. Chai, “Social grouping for target handover in multi-view video,” in Multimedia and Expo (ICME), 2013 IEEE International Conference on.   IEEE, 2013, pp. 1–6.
  • [8] Z. Qin and C. R. Shelton, “Improving multi-target tracking via social grouping,” in

    Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on

    .   IEEE, 2012, pp. 1972–1978.
  • [9] Z. Yu, O. C. Au, R. Zou, W. Yu, and J. Tian, “An adaptive unsupervised approach toward pixel clustering and color image segmentation,” Pattern Recognition, vol. 43, no. 5, pp. 1889–1906, 2010.
  • [10] Z. Yu, O. C. Au, K. Tang, and C. Xu, “Nonparametric density estimation on a graph: Learning framework, fast approximation and application in image segmentation,” in Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on.   IEEE, 2011, pp. 2201–2208.
  • [11] T. Sun, S. Ding, and Z. Ren, “Novel image recognition based on subspace and sift.” Journal of Software (1796217X), vol. 8, no. 5, 2013.
  • [12] H. Song, X. Li, and P. Wang, “Image annotation refinement using dynamic weighted voting based on mutual information.” Journal of Software (1796217X), vol. 6, no. 11, 2011.
  • [13] L. Xu, Z. Zhan, S. Xu, and K. Ye, “Cross-layer detection of malicious websites,” in Proceedings of the third ACM conference on Data and application security and privacy.   ACM, 2013, pp. 141–152.
  • [14] H. Zhang, Z. Zhang, H. Dai, R. Yin, and X. Chen, “Distributed spectrum-aware clustering in cognitive radio sensor networks,” in Global Telecommunications Conference (GLOBECOM 2011), 2011 IEEE.   IEEE, 2011, pp. 1–6.
  • [15] Q. Sun, R. Ma, Q. Hao, and F. Hu, “Space encoding based human activity modeling and situation perception,” in Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), 2013 IEEE International Multi-Disciplinary Conference on.   IEEE, 2013, pp. 183–186.
  • [16] Q. Sun, F. Hu, and Q. Hao, “Mobile target scenario recognition via low-cost pyroelectric sensing system: Toward a context-enhanced accurate identification,” 2013.
  • [17] J. Shen, P.-C. Su, S.-c. S. Cheung, and J. Zhao, “Virtual mirror rendering with stationary rgb-d cameras and stored 3-d background,” Image Processing, IEEE Transactions on, vol. 22, no. 9, pp. 3433–3448, 2013.
  • [18] J. Shen and S.-C. S. Cheung, “Layer depth denoising and completion for structured-light rgb-d cameras,” in Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on.   IEEE, 2013, pp. 1187–1194.
  • [19] Y. Wang, W. Jiang, and G. Agrawal, “Scimate: A novel mapreduce-like framework for multiple scientific data formats,” in Cluster, Cloud and Grid Computing (CCGrid), 2012 12th IEEE/ACM International Symposium on.   IEEE, 2012, pp. 443–450.
  • [20] S. E. Palmer, “Hierarchical structure in perceptual representation,” Cognitive psychology, vol. 9, no. 4, pp. 441–474, 1977.
  • [21] N. K. Logothetis and D. L. Sheinberg, “Visual object recognition,” Annual review of neuroscience, vol. 19, no. 1, pp. 577–621, 1996.
  • [22] E. Wachsmuth, M. Oram, and D. Perrett, “Recognition of objects and their component parts: responses of single units in the temporal cortex of the macaque,” Cerebral Cortex, vol. 4, no. 5, pp. 509–522, 1994.
  • [23] W. Xu, X. Liu, and Y. Gong, “Document clustering based on non-negative matrix factorization,” in Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval.   ACM, 2003, pp. 267–273.
  • [24] F. Shahnaz, M. W. Berry, V. P. Pauca, and R. J. Plemmons, “Document clustering using nonnegative matrix factorization,” Information Processing & Management, vol. 42, no. 2, pp. 373–386, 2006.
  • [25] D. Seung and L. Lee, “Algorithms for non-negative matrix factorization,” Advances in neural information processing systems, vol. 13, pp. 556–562, 2001.
  • [26] R. Sandler and M. Lindenbaum, “Nonnegative matrix factorization with earth mover’s distance metric for image analysis,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 33, no. 8, pp. 1590–1602, 2011.
  • [27] W. Xu and Y. Gong, “Document clustering by concept factorization,” in Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval.   ACM, 2004, pp. 202–209.
  • [28] D. Zhang, Z.-H. Zhou, and S. Chen, “Non-negative matrix factorization on kernels,” in

    PRICAI 2006: Trends in Artificial Intelligence

    .   Springer, 2006, pp. 404–412.
  • [29] J. J.-Y. Wang, X. Wang, and X. Gao, “Non-negative matrix factorization by maximizing correntropy for cancer clustering,” BMC bioinformatics, vol. 14, no. 1, p. 107, 2013.
  • [30] R. He, W.-S. Zheng, and B.-G. Hu, “Maximum correntropy criterion for robust face recognition,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 33, no. 8, pp. 1561–1576, 2011.
  • [31] R. Chang, X. Mu, and L. Zhang, “Software defect prediction using non-negative matrix factorization.” Journal of Software (1796217X), vol. 6, no. 11, 2011.
  • [32] D. Cai, X. He, J. Han, and T. S. Huang, “Graph regularized nonnegative matrix factorization for data representation,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 33, no. 8, pp. 1548–1560, 2011.
  • [33] D. Cai, X. He, X. Wu, and J. Han, “Non-negative matrix factorization on manifold,” in Data Mining, 2008. ICDM’08. Eighth IEEE International Conference on.   IEEE, 2008, pp. 63–72.
  • [34] B. Shen and L. Si, “Non-negative matrix factorization clustering on multiple manifolds.” in AAAI, 2010.
  • [35] L. D. Baker and A. K. McCallum, “Distributional clustering of words for text classification,” in Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval.   ACM, 1998, pp. 96–103.
  • [36] X. Liu, Y. Gong, W. Xu, and S. Zhu, “Document clustering with cluster refinement and model selection capabilities,” in Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval.   ACM, 2002, pp. 191–198.
  • [37] S. C. Deerwester, S. T. Dumais, T. K. Landauer, G. W. Furnas, and R. A. Harshman, “Indexing by latent semantic analysis,” JASIS, vol. 41, no. 6, pp. 391–407, 1990.
  • [38] C. Ding, T. Li, W. Peng, and H. Park, “Orthogonal nonnegative matrix t-factorizations for clustering,” in Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining.   ACM, 2006, pp. 126–135.
  • [39] C.-J. Lin, “Projected gradient methods for nonnegative matrix factorization,” Neural computation, vol. 19, no. 10, pp. 2756–2779, 2007.
  • [40]

    J.-Y. Wang, I. Almasri, and X. Gao, “Adaptive graph regularized nonnegative matrix factorization via feature selection,” in

    Pattern Recognition (ICPR), 2012 21st International Conference on, 2012, pp. 963–966.
  • [41] Y.-H. Xiao, Z.-F. Zhu, Y. Zhao, and Y.-C. Wei, “Class-driven non-negative matrix factorization for image representation,” Journal of Computer Science and Technology, vol. 28, no. 5, pp. 751–761, 2013.
  • [42] H. Liu, Z. Wu, X. Li, D. Cai, and T. S. Huang, “Constrained nonnegative matrix factorization for image representation,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 34, no. 7, pp. 1299–1311, 2012.
  • [43] Q. Sun, P. Wu, Y. Wu, M. Guo, and J. Lu, “Unsupervised multi-level non-negative matrix factorization model: Binary data case.” Journal of Information Security, vol. 3, no. 4, 2012.
  • [44] M. Belkin and P. Niyogi, “Laplacian eigenmaps and spectral techniques for embedding and clustering.” in NIPS, vol. 14, 2001, pp. 585–591.
  • [45] X. Niyogi, “Locality preserving projections,” in Neural information processing systems, vol. 16, 2004, p. 153.
  • [46] L. Fei-Fei, R. Fergus, and P. Perona, “Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories,” Computer Vision and Image Understanding, vol. 106, no. 1, pp. 59–70, 2007.
  • [47] G. Griffin, A. Holub, and P. Perona, “Caltech-256 object category dataset,” 2007.
  • [48] P. Gehler and S. Nowozin, “On feature combination for multiclass object classification,” in Computer Vision, 2009 IEEE 12th International Conference on.   IEEE, 2009, pp. 221–228.
  • [49] D. G. Lowe, “Object recognition from local scale-invariant features,” in Computer vision, 1999. The proceedings of the seventh IEEE international conference on, vol. 2.   Ieee, 1999, pp. 1150–1157.
  • [50] H. W. Kuhn, “The hungarian method for the assignment problem,” Naval research logistics quarterly, vol. 2, no. 1-2, pp. 83–97, 1955.
  • [51] S. Zafeiriou and M. Petrou, “Nonlinear nonnegative component analysis,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on.   IEEE, 2009, pp. 2860–2865.
  • [52] A. Anderson, P. K. Douglas, W. T. Kerr, V. S. Haynes, A. L. Yuille, J. Xie, Y. N. Wu, J. A. Brown, and M. S. Cohen, “Non-negative matrix factorization of multimodal mri, fmri and phenotypic data reveals differential changes in default mode subnetworks in adhd,” NeuroImage, 2013.
  • [53] S. Cui, R. Manica, R. F. Tabor, and D. Y. Chan, “Interpreting atomic force microscopy measurements of hydrodynamic and surface forces with nonlinear parametric estimation,” Review of Scientific Instruments, vol. 83, no. 10, p. 103702, 2012.
  • [54] L. Li and M. D. Smucker, “Tolerance of effectiveness measures to relevance judging errors,” in Advances in Information Retrieval.   Springer, 2014, pp. 148–159.
  • [55] L. Li, “Evaluating information retrieval systems with multiple non-expert assessors,” 2013.
  • [56] J. Yang, Y. Wang, K. Hua, and W. Wang, “Fairness based dynamic channel allocation in wireless mesh networks,” in Computing, Networking and Communications (ICNC), 2014 International Conference on.   IEEE, 2014, pp. 556–560.
  • [57] J. Yang and Z. Fei, “Broadcasting with prediction and selective forwarding in vehicular networks,” International Journal of Distributed Sensor Networks, 2013.
  • [58] R. P. Singh, P. X. Gao, and D. J. Lizotte, “On hourly home peak load prediction,” in Smart Grid Communications (SmartGridComm), 2012 IEEE Third International Conference on.   IEEE, 2012, pp. 163–168.
  • [59] P. X. Gao and S. Keshav, “Spot: a smart personalized office thermal control system,” in Proceedings of the fourth international conference on Future energy systems.   ACM, 2013, pp. 237–246.
  • [60] P. X. Gao, “Facial age estimation using clustered multi-task support vector regression machine,” in Pattern Recognition (ICPR), 2012 21st International Conference on.   IEEE, 2012, pp. 541–544.

Le Li

received Master of Mathematics on Computer Science degree from University of Waterloo, Canada in 2013, and Bachelor of Science degree from Beijing University of Posts and Telecommunications, China in 2010. His current research includes machine learning, information retrieval and computer vision.

Jianjun Yang is an assistant professor in the University of North Georgia. He got his Ph.D. of Computer Science from University of Kentucky, USA in 2011. His research interests include Wireless Networks.

Kaili Zhao is a Phd student at Beijing University of Posts and Telecommunications (BUPT). She got her bachelor degree at Hefei University of Technology. She was a visiting student at Ohio State University for 6 months from September in 2013. Now she is doing research in computer vision at Carnegie Mellon University.

Yang Xu is currently an undergraduate student in Beijing University of Posts and Telecommunications. His research interests include image recognition and computer vision.

Honggang Zhang is an associate professor and director of web search center at Beijing University of Posts and Telecommunications, China. He received bachelor degree from Shandong University in 1996, master and PhD degree from BUPT in 1999 and 2002 respectively. He was visiting scholar at Carnegie Mellon University in 2008. His current research interests are in the area of pattern recognition, machine learning and computer vision.

Zhuoyi Fan is currently an undergraduate student in Huazhong University of Science and Technology, China. Her research interests include image processing and signal processing.