As the generalization of the graph model [1, 2], the hypergraph model is more flexible and more intuitive to depict the complex relation of data, since the edge of hypergraph, which is known as the hyperedge, can contain more than two vertices. Due to this desirable property, hypergraph learning is recently drawn intensive attention. Over past decades, extensive hypergraph learning approaches has been proposed and successfully applied to tackle a lot of fundamental tasks, such as clustering [1, 3], classification [4, 5, 6], segmentation , dimensionality reduction [8, 9] and multi-label learning [10, 11].
As same as graph learning, the hypergraph construction process plays a vital role in hypergraph learning and a good quality hypergraph should well reveal the real relation of samples. Hypergraph learning is a frequently used tool for unsupervised and semi-supervised learning. In these two cases, the previous hypergraph learning works often adopt the neighbourhood-based (distance-based) strategy to build the hypergraph[1, 12, 13]. More specifically, for each sample, a hyperedge is generated by connecting this centroid sample and its nearest neighbors. However, as such neighborhood-based often cannot well even correctly discover the real relations of samples and it is also very sensitive to noises, the quality of hypergraph is lowered which directly degrades the performance of hypergraph learning.
. We use it here for intuitively specifying the advantages of SR over the conventional neighbourhood-based method in the relevant sample selection procedure. The figure shows the top 10 most relevant face images selected by SR and K-Nearest Neighbour (KNN) based on a given query face image. This experiment is conducted in a subset of FERET database (72 subjects with 6 images in each subject). The first two rows of the figure are the selection results of SR while the last two rows are the selection results of KNN. The left subfigure reports the results on the original FERET database while the right one reports the results on the modified FERET database in which 30% of pixels of each image has been corrupted by noise. In the figure, the first face image of each image array is the query image and the rest ten images are the relevant face images selected by SR or KNN. The histograms above the image array demonstrates the confidence scores of these top ten relevant face images. If the subjects of the return face image and the query face image are identical, its corresponding histogram is positive otherwise it is negative. In the figure, SR gets five hits either on the original FERET database or on the noisy FERET database while KNN only gets three and two hits on these two datasets respectively. Clearly, this phenomenon demonstrates the advantages of SR in relevant sample selection.
In this paper, we generalize the idea of sparse graph to present a novel hypergraph construction framework, which can leverage the regression model to construct the high quality hypergraph. We name such framework Regression-based Hypergraph (RH) model. More specifically, in RH, each sample represents a vertex and constructs a regression system together with the rest samples for measuring correlations of samples. Then, based on the obtained correlations, each sample and its top most relevant samples are employed to define a hyperedge. Moreover, the mean of the correlations among samples in a hyperedge is considered as the weight of this hyperedge, since the correlation is an intuitive measure of the closeness between two samples. We also plug the regression-based hypergraph into two classical hypergraph learning frameworks, namely hypergraph spectral clustering and hypergraph transduction, to present Regression-based Hypergraph Spectral Clustering (RHSC) and Regression-based Hypergraph Transduction (RHT) models for addressing the clustering and classification issues. As two of the most influential regression models for visual learning, Sparse Representation (SR) and Collaborative Representation (CR) are adopted as two examples to instantiate two RH instances. Since SR and CR are actually the -norm and -norm regularized regression models, we name these two instances -Hypergraph (H) and -Hypergraph (H) respectively. Similarly, their hypergraph spectral clustering and hypergraph transduction algorithms are named as -Hypergraph Spectral Clustering (HSC) and -Hypergraph Transduction (HT) where or 2 if H or H is applied.
Regression-based Hypergraph (RH) model inherits the advantages of both hypergraph and regression models. Compared to the conventional hypergraph models, RH can incorporate some properties from the chosen regression models. For example, if the Sparse Representation (SR) or Collaborative Representation (CR) is selected for hypergraph construction, the constructed RH should be more discriminative and robust, since these two regression models are better to discover the relevances among samples over the conventional neighborhood-based hypergraph construction fashions. Compared to the regression approaches, RH constructs a hypergraph to sufficiently exploit the correlation of each pair of samples instead of just utilizing the correlations between the target sample and the other samples as the regression approaches do. Compared to the regression-based graph approaches, such as the sparse graph, RH is a hypergraph model which owns a better capability and flexibility to depict the complex high-order data relations.
We employ six popular visual databases to validate our works. The experimental results demonstrate the superiority of the RH model over the conventional hypergraph models. We conclude three main contributions of our works as follows:
We provide a general idea which utilizes the regression models to construct the high quality hypergraphs. To the best of our knowledge, this paper is the first to formally and systematically build a bridge between the hypergraph model and the regression model.
We present two novel hypergraph learning frameworks based on the RH model to tackle the clustering and classification tasks respectively.
We adopt two recently influential regression models, namely Sparse Representation (SR) and Collaborative Representation (SR), to instantiate two RH instances called -Hypergraph (H) and -Hypergraph (H) which are experimentally proved to be more discriminative and robust than the conventional hypergraphs.
Ii Previous Works
Ii-a Regression Models
Regression model is a common technique for data analysis and has been successfully applied to almost all the areas in computer vision, machine learning and image processing [27, 14, 22]. Sparse Representation (SR) may be the most influential regression approach in the recent decade. SR is mainly inspired by the idea of compressed sensing . In SR, a or -norm constraint is introduced to the common regression model for compulsively selecting only a few of relevant measurements and ignoring the irrelevant ones by assigning their corresponding regression coefficients to zero. This endows SR with a strong discriminating power and a good robustness. However, Zhang _meaning:NTF . et al _catcode:NTF a et al. et al. [22, 24]
argued that the collaboration of samples instead of the sparsity is the essential factor that leads to such good discriminating ability and robustness. They proposed a linear regression model named Collaborative Representation (CR) via employing a relatively mild-norm constraint to replace the
-norm constraint to achieve the collaboration property. Many works have shown that CR is more efficient and can get a similar or even better performance. Due to the desirable properties of SR and CR, they have achieved remarkable successes in many areas and promotes the presentations of many impressive regression approaches for addressing different computer vision, machine learning and image processing issues. For examples, Gao _meaning:NTF . et al _catcode:NTF a et al. et al. kernelized SR for face recognition and image classification. Yuan _meaning:NTF . et al _catcode:NTF a et al. et al. presented a multitask joint sparse representation model to combine the strength of multiple features and/or instances for visual classification 
. Huang _meaning:NTF . et al _catcode:NTF a et al. et al. presented a SR-based classifier named Class Specific Sparse Representation (CSSR) which incorporated the properties of both SR and CR via defining the homogenous samples as a group and making them competition for representing the test sample. Yang _meaning:NTF . et al _catcode:NTF a et al. et al. proposed Relaxed Collaborative Representation (RCR) to effectively exploit the similarity and distinctiveness of samples . Although these regression approaches have obtained promising performances in different fields, they all have an obvious drawback that they can only utilize the correlation between the testing sample and the training samples. On the contrary, the proposed Regression-based Hypergraph (RH) model can sufficiently exploit the correlations among all samples. Another merit of RH is that there exists extensive regression approaches which can bring more flexility to the hypergraph model.
Ii-B Sparse Graph
Since Sparse Representation (SR) is good at selecting the relevant samples for a test sample even in the noisy conditions, some researchers have attempted to use SR to construct high quality graphs for addressing different issues. In these works, such constructed graphs are often called -graph or sparse graph and have achieved very promising performances. More specifically, Qiao _meaning:NTF . et al _catcode:NTF a et al. et al. and Timofte _meaning:NTF . et al _catcode:NTF a et al. et al. successively use SR to construct a sparse graph for dimensionality reduction [19, 20]. Huang _meaning:NTF . et al _catcode:NTF a et al. et al. leverage SR to measure the correlations between each two samples and then construct a sparse graph for transduction . The Sparse Subspace Clustering (SSC) algorithms [21, 31, 32] learn a sparse graph for clustering via considering the data self representation problem as a SR issue. Similar to [19, 20], Cheng _meaning:NTF . et al _catcode:NTF a et al. et al. utilize SR to construct the -graph (sparse-graph) for spectral clustering, subspace learning and semi-supervised learning . Although the applications and the learning (or construction) procedures of these works are very different, the obtained sparse graphs are very similar which all demonstrate the better discriminative abilities and robustness over the conventional graph models. The main drawback of the sparse graph models is that they cannot intuitively describe the high-order complex data relations, because these sparse graph models are essentially graph model whose edges can only depict the simple pairwise data relation. Since Regression-based Hypergraph (RH) model is deemed as a generalization of sparse graph from the perspectives of both regression and hypergraph, it does not suffer from this issue.
Ii-C Hypergraph Models
. The main difference between graph and hypergraph is that the edge of hypergraph can own more than two vertices which endows hypergraph with a high flexility for depicting the high-order relation. Benefitted by this desirable property, hypergraph models have been successfully applied into dozens of computer vision, machine learning and pattern recognition areas. In the past, the researchers were more keen to develop different hypergraph frameworks which define different theories to depict the hypergraph structure. The representative approaches include Clique Expansion, Star Expansion , Zhou’s Normalized Laplacian , Clique Averaging , Bolla’s Laplacian  and so on. However, as was shown in 
, all of the previous approaches, despite their very different formulations, can be proved to be equivalent to each other under specific conditions. Currently, the researchers pay more attention on developing the algorithms for the hypergraph constructions under the aforementioned hypergraph frameworks. In hypergraph, hyperedge defines the relation of data. Therefore, the hyperedge generation is very crucial to the quality of the constructed hypergraph. Conventionally, most of hypergraph models adopt the neighbourhood-based fashion to generate the hyperedges. For examples, Huang _meaning:NTF . et al _catcode:NTF a et al. et al. proposed a hypergraph learning framework for image retrieval, in which each image and its-nearest neighbors form the hyperedge . Zhou _meaning:NTF . et al _catcode:NTF a et al. et al. also adopted such neighbourhood-based fashion to generate the hyperedges for unsupervised and semi-supervised hypergraph learning . The main problems of these approaches are that they often cannot well reveal the real relation of data and are sensitive to noise. Some researchers also employed the clustering techniques to generate the hyperedges and then construct the hypergraph. As the representative approach of such category, Gao _meaning:NTF . et al _catcode:NTF a et al. et al. proposed a hypergraph-based 3-D object retrieval approach via utilizing the -means to cluster the views of the 3-D objects and consider each cluster as a hyperedge . Since the hyperedges, which are formed by the clusters, cannot share intersection vertices, these hypergraphs cannot capture the correlations of data. Another popular method is to adaptively contruct the hypergraph via imposing some meaningful constraints. As an instance of this category, Yu _meaning:NTF . et al _catcode:NTF a et al. et al. introduced a -norm constraint to the hyperedge weight matrix to present a hypergraph transduction approach for image classification . This method generates the hyperedges via adaptively assigning the weights to the hyperedges. However, it cannot guarantee the inexistence of the isolated vertices. Similar to the work , Wang _meaning:NTF . et al _catcode:NTF a et al. et al. imposed a Laplacian cost constraint and a -norm constraint to the hyperedge weights for adaptively learning the hyperedge weights in a hypergraph model . In its hyperedge generation procedure, the traditional neighbourhood-based fashion is employed to define a candidate hyperedge vertex set and then SR is applied to prune noisy vertices in this set for forming the final hyperedge. Such idea is similar but also different to us. It still considers the neighbours of a sample as its relevant samples and SR here only plays a role as a noise remover. On the contrary, Regression-based Hypergraph (RH) model thoroughly utilizes the regression model (includes SR) to generate the hyperedged. And RH is more formal and systematic to introduce how to use regression models to construct the high quality hypergraph.
Iii-a Regression-based Hypergraph
In order to incorporate some desirable properties of the regression algorithms, we introduce a new hypergraph learning framework named Regression-based Hypergraph (RH), which leverages different regression models to construct the hypergraphs. Let a -dimensional matrix be the sample matrix, where is the dimension of sample and is the number of sample. The
-dimensional column vectoris a sample which is also the
-th column of sample matrix. We apply the general formulation of regression model to estimate the correlations between each sample and the rest samples,
where and are the regression error and the regularization term respectively. is the sample matrix which excludes the -th sample . The ()-dimensional column vector is the regression coefficient vector with respect to the sample . Each element of regression coefficient vector encodes the correlation between the target sample and the sample. According to the aforementioned correlation computation fashion, each pair of samples can get two correlations, _meaning:NTF . i.e _catcode:NTF a i.e. i.e., the samples and has two correlations and . Extensive literatures [18, 19, 39, 30, 40] show that such correlation between two samples is a high quality similarity measure of samples. Therefore, following the sample similarity computation fashion in [18, 19], we define the sample similarity as the mean of the correlation absolute values of each pair of samples to guarantee the nonnegativity and symmetry of the similarity. More specifically, the similarity between the sample and the sample can be mathematically denoted as follows
The regression model cannot compute the self-correlations of samples. In other words, the self-similarity computation of a sample is still not provided. In such case, we define the self-similarity of a sample as the sum of the similarities between this sample and the rest samples,
. According to the obtained similarities among all the samples, it is not hard to construct a similarity matrix (or affinity matrix)where is the -th element of . Then, the normalization of the sample similarities can be done as follows
where -dimensional matrix is diagonal matrix whose -th diagonal element is the sum of elements in the -th row of , .
After obtaining the similarities, we define a hypergraph for depicting the relations among samples where and are the collections of its vertices and hyperedges respectively. In this hypergraph, each sample is deemed as a vertex, _meaning:NTF . i.e _catcode:NTF a i.e. i.e., the sample is corresponding to the vertex . Same as the conventional hypergraph construction fashion, a sample and its top most similar samples are employed to define a -length hyperedge . Therefore, for a data collection constructed by samples, we can obtain hyperedges. The weight of the hyperedge is defined as the mean similarity of samples in this hyperedge,
where is the number of pairs of vertices in the hyperedge .
Iii-B Learning with Regression-based Hypergraph
Spectral clustering and hypergraph transduction are the most common unsupervised and supervised hypergraph learning techniques respectively. In this subsection, we apply our Regression-based Hypergraph (RH) model to these two techniques for validating the effectiveness of our model. We develop a novel spectral clustering and hypergraph transduction framework and name them Regression-based Hypergraph Spectral Clustering (RHSC) and Regression-based Hypergraph Transduction (RHT) respectively. We begin by introducing some common definitions of hypergraph learning . We denote the degrees of vertex and hyperedge as the sum of weights of hyperedges which are incident to the given vertex and the number of vertices in the hyperedge respectively. Mathematically, the degrees of vertex and hyperedge are respectively reformulated as and . The vertex-edge incident matrix is a common tool for depicting the structure of hypergraph. each of its rows and columns are corresponding to the vertex and the hyperedge of hypergraph respectively. More specifically, for a hypergraph consisted by vertices and hyperedges, its vertex-edge incident matrix is a -dimensional binary matrix. If the vertex is on the hyperedge , the ()-th element of is 1, otherwise, 0. Due to the hyperedge generation fashion of RH, the dimension of its vertex-edge incident matrix is .
From the perspective of graph learning, the spectral clustering is actually a graph (or hypergraph) partition issue [1, 41, 42]. Then, we can consider the regression hypergraph-based spectral clustering problem as a normalize hypergraph cut issue. According to Zhou’s work , such issue can be solved by following optimization model,
where the -dimensional matrix is the collection of the hypergraph cuts of the given regression hypergraph . The -dimensional column vector is a hypergraph cut which introduces a binary partition to the given hypergraph and its elements indicate the confidences of how the corresponding vertices belonging to a subgraph after partition. is a row of matrix which encodes the elements of hypergraph cuts corresponding to the vertex .
According to the normalized cut criterion , the optimal hypergraph cuts should maximize the compactness of partitioned subgraphs and minimize the compactness of the boundaries between the subgraphs simultaneously. The compactness of subgraphs and boundaries is measured by the normalized summation of the hyperedge weights of a vertex set. With several reductions, Equation 5 can be further translated into the following matrix expression,
where function returns the trace of matrix. , and are the diagonal matrix forms of , and respectively.
is the identity matrix andis the derived normalized hypergraph Laplacian matrix which encodes the structure of the regression hypergraph . The detail deductions of this equation can be referred to the works [1, 11].
The problem in Equation III-B
is a typical eigenvalue problem. It can be easily solved by eigenvalue decomposition technique. The topoptimal hypergraph cuts are exactly the top eigenvectors corresponding the top minimal nonzero eigenvalues. Finally, the learned hypergraph cut collection is deemed as the new representation of data for clustering.
Graph-based transduction is a semi-supervised learning technique which is often leveraged to address the labeling and classification issues. In the semi-supervised case, the labels of some of data are available. Therefore, an optimal hypergraph cut should not only minimize the loss of the geometric structure of data (the loss of data relation) but also minimize the labeling error. Conventionally, the labeling error is measured by the Euclidean distance between the labels and the hypergraph cuts, since the hypergraph cuts can be deemed as the collection of the label indicator of vertex. Thus, the original hypergraph partition model in Equation 5 can be further improved as the following regularized hypergraph partition model for considering the labeling error of data,
where is a positive parameter to reconcile these two losses and the -dimensional matrix is the collection of labels. is the label vector of the -th class. Let us denote is the label of vertex . Then, we have or -1 if the vertex belonging to -th class or other classes respectively, and 0 if the vertex is unlabeled. Note, here the collection of hypergraph cuts has the same size as where .
Iii-C Two RH Instances: -Hypergraph and -Hypergraph
Sparse Representation (SR) and Collaborative Representation are two of the recent most influential regression models in computer vision and machine learning. We employ them to instantiate two Regression-based Hypergraph (RH) instances. Since SR and CR are actually the -norm and -norm regularized regression models, we name these two RH instances -Hypergraph (H) and -Hypergraph (H) respectively. The detail information of H and H are presented in Table 1. We have also plug H and H into Regression-based Hypergraph Transduction (RHT) and Regression-based Hypergraph Spectral Clustering (RHSC) frameworks to produce four new semi-supervised or unsupervised hypergraph learning approaches respectively named -Hypergraph Transduction (HT), -Hypergraph Transduction (HT), -Hypergraph Spectral Clustering (HSC), and -Hypergraph Spectral Clustering (HSC). The relations of these approaches are shown in Table II. In the next section, we apply these four RH instance algorithms to demonstrate the superiorities of our models over the conventional hypergraph models and verify the assumption that RH should inherit some desirable properties from the chosen regression models.
|Name||Regression model||in Equation 1||in Equation 1|
In this section, we conduct some experiments to employ the aforementioned four RHSC and RHT intances, namely HSC, HSC, HT and HT, to tackle the image clustering and classification tasks respectively. Since these four algorithms are all generated from Sparse Representation (SR) or Collaborative Representation (CR) which enjoy the robustness to noise and occlusion, we also conduct some experiments to discuss if these four algorithms have inherited such desirable property.
Iv-a Datasets and Compared Methods
Six image datasets, named AR , ORL , COIL20 , ETH80 , Scene15  and Caltech256 , are leveraged for validating our works. AR face database consists of more than 4,000 color images of 126 subjects . Following paper [27, 8], a subset contains 2600 images with 100 subjects are constructed in our experiment. Each subject has 26 images. The first 14 images of each subject are not involved any occlusion while the rest 12 images are involved the occlusions. In these 12 images, the faces in the first six images are occluded by the sunglasses and the faces in the other six images are occluded by the scarfs. In the general image classification and clustering experiments, only the images without any occlusion are utilized. The whole dataset is leveraged to analysis the robustness of the proposed work to disguise. The size of face image on AR database is 6043 pixels. ORL database is a face image database, which contains 400 images from 40 subjects . Each subject has ten images acquired at different times. The size of face image on ORL database is 3232 pixels. The COIL-20 database has 20 objects and each object has 72 images which are obtained by the rotation of the object through 360 in 5 steps (1440 images in total) . The size of each image is 3232 pixels on COIL20 database. The ETH80 object database  contains 80 objects from 8 categories. Each object is represented by 41 views spaced evenly over the upper viewing hemisphere (3280 images in total). The original size of each image in this dataset is 128128 pixels. We resize them to 3232 pixels. Scene15 database  is a scene database, which has 15 classes with 100 samples per category. Following paper , a subset of Caltech256 database , which has 20 classes with 100 samples per category, is used in our experiments. We directly use grayscale as the feature on ORL, AR, COIL20 and ETH80 databases. PiCoDes  is adoptted to represent the images on Scene15 and Caltech256 databases, since they are more challenging. The dimension of PiCoDes feature is 2048. Figure 2 shows some samples of these six image databases.
Nonnegative Matrix Factorization (NMF) , Graph regularized Nonnegative Matrix Factorization (GNMF) , Normalized Cut (NCut) , Normalized Hypergraph Spectral Clustering (NHSC) , Large-scale Spectral Clustering (LSC) , Clique Expansion-based Hypergraph Spectral Clustering using Matrix Trace Weights (CEHSC+Trace) , Normalized Hypergraph Spectral Clustering using the mean of distances between the centroid and the vertices in a hyperedge (NHSC+Cent) , and -Graph Spectral Clustering (GSC)  are chosen as the compared approaches in the image clustering experiments. NCut can be deemed as a sort of regular graph spectral clustering algorithm. So there are three graph spectral clustering algorithms. They are Ncut, LSC and GSC. NHSC, CEHSC+Trace and NHSC+Cent are three hypergraph spectral clustering methods. The only difference between NHSC and NHSC+Cent is the weights of hyperedges. NHSC uses the mean of distances between each two samples in a hyperedge while NHSC+Cent uses the mean of distances between the hyperedge centroid and the samples in a hyperedge. NHSC+Cent and CEHSC+Trace are referenced from our recent work  which empirically studied the effect of hyperedge weighting scheme to hypergraph learning and select the most optimal weighting schemes for different hypergraph frameworks. NHSC+Cent and CEHSC+Trace are the best hyperedge weighting scheme and hypergraph spectral clustering combinations for addressing clustering issue reported in .
, LIB-Support Vector Machine (LIBSVM), Normalized Hypergraph Transduction (NHT) , Graph Transduction (GT) , Adaptive Hypergraph-based Classifier (AHC) , Normalized Hypergraph Transduction using Matrix Trace Weights (NHT+Trace) , Normalized Hypergraph Transduction using the Volume of Simplex Weights (NHT+Volume) , and Sparse Graph-based Classifier (SGC)  as the compared approaches for image classification. GT and SGC are the graph transduction algorithms while AHC, NHT, CEHT+Trace and NHT+Volume are the hypergraph transduction algorithms. CEHT+Trace and NHT+Volume are the best hyperedge weighting scheme and hypergraph transduction combinations for addressing classification issue reported in . In the experiments, all the compared methods are well tuned.
|Methods||Clustering Accuracy||Normalized Mutual Information (NMI)|
Iv-B Image Clustering
, Clustering Accuracy and Normalized Mutual Information (NMI) are leveraged as two evaluation metrics.
Table III reports the clustering performances of different clustering methods. The bold number indicates the best accuracy in a dataset under same metric. We can find that HSC and HSC outperform almost all compared approaches in all experiments and achieve remarkable improvements over the conventional hypergraph spectral clustering algorithms. For examples, the clustering accuracy gains of HSC over NHSC are 33.50%, 11.50%, 6.25%, 5.49%, 9.73% and 5.75% on AR, ORL, COIL20, ETH80, Scene15 and Caltech256 datasets respectively. Similarly, such gains of HSC are 36.72%, 11.75%, 5.55%, 10.33% and 8.55%. The experimental results also show that, as the generalization of sparse graph spectral clustering, HSC and HSC obtain the better performances over GSC which is known as a sparse graph spectral clustering algorithm. More specifically, The NMI gains of HSC over GSC on AR, ORL, COIL20, ETH80, Scene15 and Caltech256 datasets are 7.94%, 4.71%, 13.23%, 8.72%, 7.66% and 4.02% respectively. Similarly, the NMI improvements of HSC over GSC are 7.96%, 4.72%, 11.50%, 3.97%, 9.06% and 5.74% on AR, ORL, COIL20, ETH80, Scene15 and Caltech256 datasets. Another interesting phenomenon can be observed from the experimental results is that HSC performs much better than HSC comprehensively. We attribute this to the fact that Collaborative Representation (CR) is often more discriminative than Sparse Representation (SR) [22, 23].
|Methods||Classification Errors (Mean|
Iv-C Image Classification
We employ AR, COIL20, ETH80 and Scene15 databases for evaluating image classification performances of different classifiers. The two-fold cross-validation scheme is applied in the image classification experiments.
Table IV reports the classification errors of different classifiers on different databases. Similar to the experimental results of image clustering, the proposed approaches, HT and HT achieve very promising classification performance and consistently outperform the other hypergraph transduction approaches. Particularly, It is worthwhile to point out that HT ranks first on three databases among all four databases. More specifically, the classification accuracy gains of HT over the other hypergraph transduction approaches, namely NHT, AHC, NHT+Trace and NHT+Volume on AR database are 27.21%, 28.35%, 27.42% and 27.50% respectively. Such numbers of HT on ETH80 database are 10.15%, 9.69%, 8.66% and 8.50% respectively. From the observations, it is not hard to find that HT often performs much better than the SGC which can be deemed as the pairwise version of HT. We believe that such phenomenon well verifies the better representational power of -hypergraph over -graph.
Iv-D Robustness Analysis
It is well known that Sparse Representation (SR) and Collaborative Representation (CR) are robust to the disguise and noise. Since -Hypergraph (H) and -Hypergraph (H) are generated by SR and CR, here we conduct several experiments to see if H and H also enjoy such desirable properties.
Iv-D1 Robust to Noise
In order to evaluate the robustness of our works to noise, we construct five noisy versions of COIL dataset via randomly assigning zero or 255 to a certain proportion of pixels in the image. In these noisy COIL20 databases, we use the Noise Level to measure the noise degree of data where the Noise Level is defined as the ratio of the number of noisy dimensions to the number of total dimensions. The Noise Levels of these five noisy versions of COIL20 database are 0.1, 0.2, 0.3, 0.4 and 0.5 respectively (see the examples in Figure 3). We follow the same fashions as mentioned in the previous sections to conduct the image classification and clustering experiments in these noisy image databases.
We plot the experimental results of different approaches under different Noise Levels in Figure 4. Figures 4(a) and 4(b) report the clustering accuracy and NMI respectively. From these observations, we can see that the clustering performances of all approaches are decreased along with the increasing of Noise Levels. However, compared with the conventional hypergraph methods, such as NHSC and CEHSC, GSC, HSC and HSC perform much better and their clustering performances are apparently decreased much slower along with the increasing of Noise Level. Clearly, such phenomenon shows that HSC and HSC are more robust to noise in comparison with the conventional hypergraph spectral clustering models. Figure 4(c) shows the classification errors of different approaches under different Noise Levels. The observations of image classification experiments are quite different to the ones of the aforementioned image clustering experiments. The performances of all approaches are very similar, and only HT slightly outperforms the others. We mainly attribute this phenomenon to the fact that the training samples all contain noise and the supervision from category labels compulsively introduces the noise samples to the hypergraph learning models. Therefore, the contributions of the robust sample selection procedures from Collaborative Representation (CR) and Sparse Representation (SR) are weakened. Comprehensively speaking, H and H enjoy a certain amount of robustness to noise particularly in the unsupervised way.
Iv-D2 Robust to Disguise
We employ AR database to evaluate the image clustering and classification performances under the disguise case, since AR database have already provided the face images with natural disguise (sunglasses and scarfs) for each subject. In the image clustering experiments, all the face images are applied. In the image classification case, we use the face images without any occlusion for training while use the face images with occlusion for testing.
Table V lists the clustering performances of our works and three other conventional hypergraph approaches. In these experiments, HSC and HSC show the significant advantages. For examples, the clustering accuracy gains of HSC over NHSC, NHSC+Cent and CEHSC+Trace are 27.68%, 27.62% and 29.00% respectively. HSC performs even better. The clustering accuracy improvements of HSC over these conventional hypergraph approaches are 47.87%, 47.81% and 49.13% respectively. Table VI tabulates the classification errors of different approaches in the disguise case. From the observations, HT and HT obtain very promising performance while all the other traditional hypergraph transduction approaches are totally failed. The classification error of HT is 23 times lower than the ones of NHT, AHC, NHT+Trace and NHT+Volume. Clearly, the experimental results demonstrate that HT and HT are robust to disguise.
Iv-E Parameter Settings
In this section, we discuss the influences of different parameters to our works. The RHSC methods, such as HSC and HSC, mainly involve two parameters, and , where is used for controlling the regression regularization term in a regression model while is the hyperedge length. The RHT methods, such as HT and HT, have one more parameter, , which is used for balancing the labeling error of data and the loss of hypergraph partition. Although the hyperedge length is an important parameter which can deeply influence the quality of hypergraph, it has numerous possible values and it is impossible for us to evaluate each possible value. So, in our experiments, we define a hyperedge length candidate collection and then conduct experiments to empirically find the optimal value of hyperedge length. Tables VII and VIII respectively list the hyperedge length candidate collections of the proposed RH works and report their optimal hyperedge lengths. Figure 5 demonstrates the influences of to the clustering performances of HSC and HSC. We choose the mean of clustering accuracy and NMI as the comprehensive clustering performance evaluation metrics. From the observations in Figure 5, we can find that the performance of HSC is quite insensitive to and the optimal of HSC on ORL, AR, COIL20, ETH80, Scene15 and Caltech256 databases, are , , , and respectively. With regard to HSC, a higher value of is much better on Scene15, Caltech256, COIL20 and ETH80 databases while a medium is more suitable to the AR and ORL databases. Figure 6 shows the impacts of and to the classification performances of HT and HT. The phenomena as similar as the ones in Figure 5 are observed. HSC is not very sensitive to and the best for AR, COIL20, ETH80 and Scene15 databases are , , and respectively. A medium works well for HT on AR and COIL20 databases while HT with a higher shows the better performances on Scene15 and ETH80 databases. Moreover, from the observations in Figures 6(c) and 6(d), both HT and HT can achieve the best classification performances when is larger than 1.
In this paper, we presented a new solution for hypergraph construction in which the regression models are used for measuring the closeness among samples. We named this new hypergraph framework Regression-based Hypergraph (RH). Based on two conventional hypergraph learning models, namely Hypergraph Spectral Clustering (HSC) and Hypergraph Transduction (HT), We also developed the Regression-based Hypergraph Spectral Clustering (RHSC) and Regression-based Hypergraph Transduction (RHT) models for addressing the clustering and classification issues. As two influential regression approaches for visual learning, Sparse Representation (SR) and Collaborative Representation (CR) are employed to instantiate two instances of RH and their RHSC and RHT algorithms. Six popular image databases are leveraged for validating the effectiveness of our works. It can be concluded from observations of the experiments that RH inherits the desirable properties from both hypergraph and regression model.
There still exist many interesting future works based on our models, since RH is a general framework for hypergraph construction in which researchers can flexibly choose their own appropriate regression models to construct RHs for tackling different tasks. For examples, RH can be further applied to hypergraph-based subspace learning 56], multi-label learning [6, 57] or attribute learning . Moreover, to investigate the adaptive and efficient RH construction fashion is also a very meaningful direction for improving the utility and efficiency [4, 58].
-  D. Zhou, J. Huang, and B. Schölkopf, “Learning with hypergraphs: Clustering, classification, and embedding,” in Advances in neural information processing systems (NIPS), 2006, pp. 1601–1608.
-  J. Y. Zien, M. D. Schlag, and P. K. Chan, “Multilevel spectral hypergraph partitioning with arbitrary vertex sizes,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 18, no. 9, pp. 1389–1399, 1999.
-  S. Huang, A. Elgammal, and D. Yang, “On the effect of hyperedge weights on hypergraph learning,” arXiv preprint arXiv:1410.6736, 2014.
-  J. Yu, D. Tao, and M. Wang, “Adaptive hypergraph learning and its application in image classification,” IEEE Transactions on Image Processing, vol. 21, no. 7, pp. 3262–3272, 2012.
-  M. Wang, X. Wu et al., “Visual classification by l1-hypergraph modeling,” IEEE Transactions on Knowledge and Data Engineering, vol. 27, no. 9, pp. 2564–2574, 2015.
-  G. Chen, J. Zhang, F. Wang, C. Zhang, and Y. Gao, “Efficient multi-label classification with hypergraph regularization,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009, pp. 1658–1665.
-  P. Ochs and T. Brox, “Higher order motion models and spectral clustering,” in IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 614–621.
-  S. Huang, D. Yang, Y. Ge, D. Zhao, and X. Feng, “Discriminant hyper-laplacian projections with its applications to face recognition,” in IEEE conference on Multimedia and Expo Workshop on HIM (ICMEW), 2014, pp. 1–6.
-  S. Huang, D. Yang, J. Zhou, and X. Zhang, “Graph regularized linear discriminant analysis and its generalization,” Pattern Analysis and Applications, vol. 18, no. 3, pp. 639–650, 2015.
-  L. Sun, S. Ji, and J. Ye, “Hypergraph spectral learning for multi-label classification,” in ACM international conference on Knowledge discovery and data mining (SIGKDD), 2008, pp. 668–676.
-  S. Huang, M. Elhoseiny, A. Elgammal, and D. Yang, “Learning hypergraph-regularized attribute predictors,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 409–417.
-  Y. Huang, Q. Liu, S. Zhang, and D. N. Metaxas, “Image retrieval via probabilistic hypergraph ranking,” in IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 3376–3383.
-  Y. Gao, M. Wang, D. Tao, R. Ji, and Q. Dai, “3-D object retrieval and recognition with hypergraph analysis,” IEEE Transactions on Image Processing, vol. 21, no. 9, pp. 4290–4303, 2012.
-  J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust face recognition via sparse representation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210–227, 2009.
-  S. Gao, I. W.-H. Tsang, and L.-T. Chia, “Kernel sparse representation for image classification and face recognition,” in European Conference on Computer Vision (ECCV), 2010, pp. 1–14.
-  C.-Y. Lu, H. Min, J. Gui, L. Zhu, and Y.-K. Lei, “Face recognition via weighted sparse representation,” Journal of Visual Communication and Image Representation, vol. 24, no. 2, pp. 111–116, 2013.
-  S. Huang, D. Yang, J. Zhou, and L. Huangfu, “Sparse graph-based transduction for image classification,” Journal of Electronic Imaging, vol. 24, no. 2, p. 023007, 2015.
-  B. Cheng, J. Yang, S. Yan, Y. Fu, and T. S. Huang, “Learning with L1-graph for image analysis,” IEEE Transactions on Image Processing, vol. 19, no. 4, pp. 858–866, 2010.
-  R. Timofte and L. Van Gool, “Sparse representation based projections,” in British machine vision conference (BMVC), 2011, pp. 61–1.
-  L. Qiao, S. Chen, and X. Tan, “Sparsity preserving projections with applications to face recognition,” Pattern Recognition, vol. 43, no. 1, pp. 331–341, 2010.
-  E. Elhamifar and R. Vidal, “Sparse subspace clustering,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009, pp. 2790–2797.
-  L. Zhang, M. Yang, and X. Feng, “Sparse representation or collaborative representation: Which helps face recognition?” in International Conference on Computer Vision (ICCV), 2011, pp. 471–478.
-  M. Yang, L. Zhang, D. Zhang, and S. Wang, “Relaxed collaborative representation for pattern classification,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 2224–2231.
-  R. Timofte and L. Van Gool, “Weighted collaborative representation and classification of images,” in International Conference on Pattern Recognition (ICPR), 2012, pp. 1606–1610.
-  S. Huang, Y. Yang, D. Yang, L. Huangfu, and X. Zhang, “Class specific sparse representation for classification,” Signal Processing, vol. 116, pp. 38–42, 2015.
-  P. J. Phillips, H. Wechsler, J. Huang, and P. J. Rauss, “The feret database and evaluation procedure for face-recognition algorithms,” Image and Vision Computing, vol. 16, no. 5, pp. 295–306, 1998.
-  I. Naseem, R. Togneri, and M. Bennamoun, “Linear regression for face recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 11, pp. 2106–2112, 2010.
-  D. L. Donoho, “Compressed sensing,” IEEE Transactions on Information Theory, vol. 52, no. 4, pp. 1289–1306, 2006.
-  X.-T. Yuan, X. Liu, and S. Yan, “Visual classification with multitask joint sparse representation,” IEEE Transactions on Image Processing, vol. 21, no. 10, pp. 4349–4360, 2012.
-  S. Huang, D. Yang, J. Zhou, L. Huangfu, and X. Zhang, “Sparse graph-based transduction for image classification,” Journal of Electronic Imaging, vol. 24, no. 2, p. 023007, 2014.
-  X. Peng, L. Zhang, and Z. Yi, “Scalable sparse subspace clustering,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 430–437.
-  G. Liu, Z. Lin, and Y. Yu, “Robust subspace segmentation by low-rank representation,” in International Conference on Machine Learning (ICML), 2010, pp. 663–670.
-  B. Cheng, J. Yang, S. Yan, Y. Fu, and T. S. Huang, “Learning with L1-graph for image analysis,” IEEE Transactions on Image Processing, vol. 19, no. 4, pp. 858–866, 2010.
-  L. Pu and B. Faltings, “Hypergraph learning with hyperedge expansion,” in Machine Learning and Knowledge Discovery in Databases, 2012, pp. 410–425.
-  A. Panagopoulos, C. Wang, D. Samaras, and N. Paragios, “Simultaneous cast shadows, illumination and geometry inference using hypergraphs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 2, pp. 437–449, 2013.
-  S. Agarwal, J. Lim, L. Zelnik-Manor, P. Perona, D. Kriegman, and S. Belongie, “Beyond pairwise clustering,” in IEEE conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, 2005, pp. 838–845.
-  M. Bolla, “Spectra, euclidean representations and clusterings of hypergraphs,” Discrete Mathematics, vol. 117, 1993.
-  S. Agarwal, K. Branson, and S. Belongie, “Higher order learning with graphs,” in International Conference on Machine Learning (ICML), 2006, pp. 17–24.
-  S. Yan and H. Wang, “Semi-supervised learning by sparse representation.” in SIAM International Conference on Data Mining (SDM). SIAM, 2009, pp. 792–801.
-  L. Zhuang, H. Gao, Z. Lin, Y. Ma, X. Zhang, and N. Yu, “Non-negative low rank and sparse graph for semi-supervised learning,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2012, pp. 2328–2335.
A. Y. Ng, M. I. Jordan, Y. Weiss et al.
, “On spectral clustering: Analysis and an algorithm,”Advances in neural information processing systems (NIPS), vol. 2, pp. 849–856, 2002.
-  J. Shi and J. Malik, “Normalized cuts and image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888–905, 2000.
-  A. Martínez and R. Benavente, “The ar face database,” Bellatera, Jun 1998.
-  F. S. Samaria and A. C. Harter, “Parameterisation of a stochastic model for human face identification,” in Proceedings of the Second IEEE Workshop on Applications of Computer Vision, 1994, pp. 138–142.
-  S. A. Nene, S. K. Nayar, H. Murase et al., “Columbia object image library (COIL-20),” Technical Report CUCS-005-96, Tech. Rep., 1996.
-  B. Leibe and B. Schiele, “Analyzing appearance and contour based methods for object categorization,” in IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2003, pp. II–409.
-  S. Lazebnik, C. Schmid, and J. Ponce, “Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories,” in IEEE conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, 2006, pp. 2169–2178.
-  G. Griffin, A. Holub, and P. Perona, “Caltech-256 object category dataset,” 2007.
-  A. Bergamo, L. Torresani, and A. W. Fitzgibbon, “Picodes: Learning a compact code for novel-category recognition,” in Advances in Neural Information Processing Systems (NIPS), 2011, pp. 2088–2096.
-  D. D. Lee and H. S. Seung, “Learning the parts of objects by nonnegative matrix factorization,” Nature, vol. 401, pp. 788–791, 1999.
-  D. Cai, X. He, J. Han, and T. S. Huang, “Graph regularized nonnegative matrix factorization for data representation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 8, pp. 1548–1560, 2011.
X. Chen and D. Cai, “Large scale spectral clustering with landmark-based
AAAI Conference on Artificial Intelligence (AAAI), 2011.
-  C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,” ACM Transactions on Intelligent Systems and Technology, vol. 2, pp. 27:1–27:27, 2011.
-  O. Duchenne, J.-Y. Audibert, R. Keriven, J. Ponce, and F. Ségonne, “Segmentation by transduction,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2008, pp. 1–8.
-  D. Cai, X. He, and J. Han, “Document clustering using locality preserving indexing,” IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 12, pp. 1624–1637, 2005.
-  Z. Zhang, P. Ren, and E. R. Hancock, “Unsupervised feature selection via hypergraph embedding.” in Birtish Machine Vision Conference (BMVC), 2012, pp. 1–11.
-  L. Sun, S. Ji, and J. Ye, “Hypergraph spectral learning for multi-label classification,” in ACM international conference on Knowledge discovery and data mining (SIGKDD), 2008, pp. 668–676.
-  W. Liu, J. He, and S.-F. Chang, “Large graph construction for scalable semi-supervised learning,” in International conference on machine learning (ICML), 2010, pp. 679–686.