1 Introduction
High resolution (HR) images are generally preferred to low resolution (LR) ones in many applications of computer vision, such as remote sensing, medical imaging and video surveillance. However, the resolution is always limited by the constraint of optical imaging systems and hardware devices. As a software technique to break this limitation, superresolution (SR) has been developed to reconstruct HR images from the observed LR ones using specific algorithms. SR methods can be divided into two categories: reconstruction based and learning based methods.
Reconstruction based methods recover HR images with help of prior knowledge and statistics of natural images, such as gradient profile prior Sun
GMM , wavelet based model wavelet and total variation (TV) TV . Global constraint Irani has also been widely used as a typical backprojection technique for SR.Learning based methods predict the missing HR details by learning the model of relationships between pairs of LR and HR examples. The representative methods include neighbor embedding (NE) algorithm Chang , sparse coding (SC) based method Yang2010 and positionbased method Ma . As for the above mentioned learning based methods, external examples from training images are required.
However, internal examples instead of external ones can also be utilized. In nature images, sufficient examples which are highly correlated to the input patches can be found in the input image, its repeatedly downsampled and subsequently upscaled versions. In the past few years, selfsimilarity has been successfully utilized for SRGlasner ; ZhangK2013 ; Yu2014 ; Bevilacqua2014 . Glasner et al. Glasner first designed an appealing selfsimilarity learning framework. With the help of selfexamples, the input image is repeatedly magnified to the desired size. By this coarsetofine strategy, the difficulty of each step is alleviated, which benefits the performance of the whole SR system. Due to these advantages, selfsimilarity learning has been followed by many researchers in recent years. Bevilacqua et. al Bevilacqua2014 proposed a new double pyramid SR model with simple multivariate regression to learn the direct mappings between LR and HR patches. Zhang et. al ZhangK2013 presented a neighbor embedding based selfsimilarity learning SR scheme with spatially nonlocal regularization. Yu et. al Yu2014 combined the selfsimilarity learning with sparse representation to perform SR. However, due to the degradation (i.e. blurring and downsampling) of the observed image which is also the source of selfexamples. The above mentioned conventional learning methods are prone to produce unfaithful representation coefficients, which are not suitable for accurate SR reconstruction. To solve the problem, a nonlocally constrained learning methods have been introduced recently Dong2013 . Dong et. al Dong2013
proposed a nonlocally centralized sparse representation for image restoration using PCA dictionary trained from selfexamples. Specifically, they defined the deviation of the learned sparse codes from the expected true ones as sparse coding noise (SCN). As suggested by their work, SR performance can be improved by suppressing SCN through calculating nonlocal means of the sparse codes for similar neighbors of the LR input patches as an estimation of the optimal codes. However, the estimation is still a weighted linear combination of codes for the similar patches. In our previous work
prework , we found that nonlinear lowrank constraint can be used to suppress SCN in selfsimilarity learning scheme for SR. Furthermore, in this paper, we propose a novel kernel based lowrank sparse coding (KLRSC) method via selfsimilarity learning for single image SR. Selfexamples are extracted from the input image itself, its degraded versions and upscaled ones. The input image is gradually superresolved. In each magnification, similar column components of a nonlocal data matrix which consists of a vectorized input patch and its nonlocal neighbors can be observed. This property of similarity leads to the nature of lowrank. Furthermore, we also use the kernel method kernel1997 to capture nonlinear structures of data, the nonlocal data are mapped into a highdimensional feature space by kernel method. In our work, we find that the lowrank property is preserved when the nonlocal patches are mapped into the kernel space. Due to this observation, we assume that the sparse codes for nonlocal matrices should be approximately lowrank. Thus, we perform kernel based lowrank sparse coding to gain accurate coefficients for selfsimilarity learning based SR. Experimental results demonstrate the advantage of our proposed method in both visual quality and reconstruction error. Our contributions are two folds:
The lowrank property is proved to be preserved when the nonlocal data are mapped into high dimensional space by kernel method.

A novel kernel based lowrank sparse coding based scheme for single image SR is proposed, which exploits both lowrank property and nonlinear structural information of nonlocal data in a highdimensional space.
The remainder of this paper is organized as follows. In Section 2, we describe the proposed method in detail. The experimental results are given in Section 3. We conclude this paper in Section 4.
Our preliminary work has appeared in prework .
2 Proposed kernel based lowrank sparse model for single image SR
2.1 Overview
In this section, we start the discussion of the kernel based lowrank sparse model for single image SR. We adopt the double pyramid selfsimilarity learning framework which is the same as that of Bevilacqua2014 . By coarsetofine strategy, the observed image is zoomed in by several times to reach the expected size. In each magnification, we perform KLRSC to learn the representation coefficients for SR reconstruction. Both the selfexamples, the structural information and the underlying nonlinear structure of nonlocal similar examples are exploited in the coding stage. Then, an interim image can be recovered by the learned coefficients and selfexamples for the next magnification. When the image reaches the desired size, the iterations of magnification will terminate.
2.2 Selfsimilarity learning and gradual magnification
In the stage of selfsimilarity learning, the pairs of examples are extracted from two pyramids of images. The flowchart of the double pyramids model is shown in Fig. 1. We denote the pyramid which composes of the sequences of the input image and its several downsampled versions as for . Given the input image , the downsampling is repeated for times with a factor of at each time. The th layer is represented as:
(1) 
where is a convolution operator and denotes the downsampling operator by a factor of .
is a Gaussian blur kernel with a standard variance
which can be computed as Blur :(2) 
The upscaled pyramid
by bicubic interpolation is established as:
(3) 
where is the th layer of the upscaled pyramid with respect to the layer and is an upscaling operator by a factor of s. In order to obtain the pairs of selfexamples, layer and the corresponding layer are divided into overlapping patches. For each patch from , we use four highpass filters to extract its gradient feature of the first and secondorder gradients in both vertical and horizontal directions:
(4) 
The four highpass filtered features are concatenated into vector as a descriptor of the patch. As for the corresponding patch from , we extract its intensity feature by subtracting its mean value. We collect these two kinds of features from all layers and normalize them to unit norm to construct the dictionary. Let represent the dictionary for reconstruction and denote the one for learning.
In multistep magnification, we gradually superresolve the input image . Given the total upscale factor , we repeat the magnification for times, where returns the nearest integer larger than . In the th magnification for , we produce the interim layer by:
(5) 
Then we partition the th layer is into overlapped patches and convert them into a set of normalized gradient features . We recover the corresponding patches by kernel based lowrank sparse representation and reconstruct the layer by weighted average operation on the overlapped region, which we describe in the following subsection.
2.3 Kernel based lowrank sparse representation for SR reconstruction
In this subsection, we present how to recover the superresolved layers from the interpolated ones using our proposed KLRSC. We first describe the algorithm of KLRSC for SR reconstruction. Then we present the postprocessing procedure by the incorporation of iterative back projection (IBP) Irani and pixelwise autoregressive (AR) model regularization Dong2011 to improve the quality of reconstructed layer.
Fig. 2 gives the illustration of KLRSC for SR reconstruction. Nonlocalsimilarity is an effective prior for image reconstruction ZhangK2013 ; Yu2014 ; Dong2013 ; Lu2014 ; NLSC ; selfsimilarity2016 , which means that small patches tend to appear repeatedly at different locations of a natural image. For each jth feature vector extracted from the ith layer , we select its most similar nonlocal neighbors in the same layer and stack them as columns where refers to the indices of the nonlocal data. We also find its nearest atoms in the dictionary to create an subset for learning and accumulate the corresponding atoms in the dictionary to form the subset for reconstruction. denotes the indices of the selected atoms.
2.3.1 Implementation of KLRSC for SR
In standard sparse coding, the sparse coding for the feature vector can be formulated as:
(6) 
Different from the conventional approach, Recently, Zhang et. al Zhang2013 proposed a lowrank sparse coding method for image classification. They encoded densely sampled SIFT features in spatially local domain. The codes for spatially local features were assumed to be lowrank. we introduce a lowrank constraint to regularize the representations for similar feature vectors. We attach the feature vector to the nonlocal feature vectors to combine a grouped matrix with the nature of lowrank property. The corresponding sparse coefficient matrix for representing the data upon the subset is also expected to be lowrank. Lowrank optimization relies on the proof that the convex envelope of rank is the nuclear norm under broad conditions lowrankproof . Based on this theorem, lowrank optimization has been successfully used in many applications Candes2011 ; Liu2015 ; TILT ; Tang2014 ; Ren2015 ; ICIPfusion We use nuclear norm constraint lowrankproof to formulate the lowrank optimization. The augmented optimization problem can be written as:
(7) 
where represents the corresponding weights that each atom in the subset contributes in the reconstruction of the augmented data . The nuclear norm
is calculated by the sum of the matrix singular values, which is an approximation of rank.
and are the parameters balancing different regularization terms.We also use the kernel method kernel1997 to capture the nonlinear structures of features, which can reduce the feature quantization error and improve the coding performance. As suggested by KSR ; Shi2015 , we transform the augmented data and the LR subset into high dimensional space by the nonlinear mapping: to capture the relationship between them. The augmented features are transformed to and the corresponding LR subset is mapped to . Given two column features and , we define a kernel function . In our work, we use Gaussian kernel function . Thus the kernel matrix can be represented as where the element .
In Fig. 3, we draw the nuclear norm distributions of the nonlocal matrices consisting of nonlinearly mapped nonlocal feature vectors. The feature vectors extracted from a test image and their nonlocal neighbors are concatenated to form matrices of nonlocal features. Note that the nuclear norm of nonlinear mapped nonlocal matrix is calculated as where
denotes the diagonal matrix with the eigenvalues of
on the diagonal and is the matrix trace operator. It shows that the matrices of nonlocal features tend to have relatively lower nuclear norms than their maximum (), which indicates the lowrank property of nonlocal matrices.Thus, with this preservation of lowrank property, the optimization problem of (7) in kernel space can be rewritten as:
(8) 
However, since the optimization problems of nuclear norm and norm (8) are difficult to solve simultaneously, we introduce two more relaxation variables and impose fidelity constraints between the pairs of relaxation variables:
(9) 
We use inexact augmented Lagrange multiplier (IALM) method Lin2009 to solve problem (9), which has also been used to efficiently solve other lowrank problems (i.e. RPCA for low rank matrix recovery Candes2011 ). We add two more variables to relax the fidelity constraints. The augmented Lagrange function for (9) is:
(10) 
where is the operator to get matrix trace. and are scalar constants. and are Lagrange multiplier matrices. and are the parameters balancing the difference between pairs of objective variables and other regularization terms.
2.3.2 Optimization of KLRSC
There are three objective variables in (10) which we alternatively update, followed by the adjustment of multipliers. Softthreshold operations on matrix elements and singular values are used to solve the problem of norm and nuclear norm optimizations. The update steps of and the multipliers are given below.
Update
Firstly, we update and meanwhile fix other variables. The optimization function with respect to derived from (10) can be formulated as:
(11) 
The norm optimization problem of (11) can be solved by softthresholding:
(12) 
where signmax is a shrinkage operator on values of matrix .
Update
Then we update and fix others by solving the following optimization problem:
(13) 
The nuclear norm optimization problem of (13) can be solved by singular value softthresholding:
(14) 
where is a shrinkage operator on singular values of matrix and
is the singular value decomposition of
.Update
The optimization function with respect to is given by:
(15) 
Solving the optimization problem (15), we update by:
(16)  
where
is an identity matrix and the matrix
is represented as:(17)  
Update multipliers
(18)  
where is a scalar constant.
When the changes of objective variables during updates are below a defined threshold , the optimization reaches convergence. We summarize this optimization in Algorithm 1.
2.3.3 Effectiveness of KLRSC
To explain the effectiveness of KLRSC, we perform an experiment to investigate the statistical property of sparse coding noises (SCN) for different coding methods. We use
image as a test image. Its LR counterpart is generated through blurring (Gaussian kernel with standard deviation
), downsampling and upscaling (with a factor of ). We collect pairs of LR and HR features from the LR and HR images. DCT dictionary is used in our experiment. We denote method of sparse coding with lowrank constraint as ’LRSC’ which appeared in our preliminary work prework . We firstly calculate the sparse coefficients for them using KLRSC, LRSC and SC, respectively. We calculate SCN by following the definition in Dong2013 . In our experiment, We evaluate SCN by norm. In Fig 4, we draw the norm distributions of SCN for KLRSC, LRSC and conventional SC. The distribution for KLRSC, LRSC and SC is drawn in red, blue and black lines, respectively. It is shown that KLRSC get lower SCN than the other two methods do, which means that the proposed KLRSC approach effectively suppress SCN utilizing the lowrank property of nonlocalsimilarity and improve the coding performance.2.3.4 Postprocessing procedure
When the optimization converges, the solution becomes both sparse and lowrank. Then we distil the first column of as the sparse weight for the reconstruction of the HR patch because lowrank constraint does not change the identities of columns. The HR patch can be sparsely represented upon as:
(19) 
where is the norm of the corresponding LR feature and denotes the average intensity of the corresponding LR patch. Having obtained all HR patches , we merge them into the layer by averaging the intensity of the overlapping pixels between the adjacent patches.
To enhance quality of the reconstructed interim layer, we apply IBP algorithm Irani and pixelwise autoregressive (AR) model Dong2011 to both enforce the global reconstruction constraint between the interim layer and the input and refine the relation between neighboring pixels.
The jth pixel of the reconstructed layer is expected to be predicted as a linear combination of its neighboring pixels in a square window: , where is the central pixel and is the vector consisting of its neighbors. To learn the combination weights , we collect the N nearest neighbors of the centered patch from other already reconstructed HR layers. These patches are assumed to share the same neighboring relationship. The combination weights can be obtained by the following optimization problem:
(20) 
can be derived by:
(21) 
where , and is the identity matrix. Thus, we regularize the estimated layer by minimize the AR prediction error and the global reconstruction error by:
(22) 
where describes pixelwise relationships in , denotes the initial HR estimation, is the LR observation, and are the downsampling and blurring operator of the th layer, respectively. The layer is updated by:
(23) 
where is the step size for gradient descent.
According to the selfsimilarity learning framework, we repeat the aforementioned lowrank sparse representation based SR for times followed by a fine adjustment to get the final SR result.
2.4 Summary
The complete SR process is summarized in Algorithm 2.
3 Experimental results
In our experiments, we use nine test images from the software package for Dong2011 . These images (see Fig. 5) cover various contents including humans, animals, plants and manmade objects. The size of image parthenon is and the size of other images is . We compare our method with SCSR Yang2010 , ASDS Dong2011 , LRNESR Chen2014 , DMSR Bevilacqua2014 , NCSR Dong2013 and Aplus Aplus2015 . Since the human visual systems are more sensitive to luminance changes in color images, we only perform our proposed method on the luminance component. The SR performances are evaluated in the luminance channel by the peak signaltonoise ratio (PSNR) and the structural similarity (SSIM) SSIM objectively.
3.1 Experimental setting
The color test images are blurred with Gaussian kernel with standard variation and then downsampled by bicubic interpolation to generate the LR input images. All the layers of images are split into patches with overlap of five pixels. The number of layers of LR and HR pyramids to train the selfexample dictionary is . The upscaling factor for each time of magnification is . The standard variance of Gaussian Blur kernel for the generation of the th layer is computed by (2). The number of the neighbors for lowrank sparse representation is 60. The number of similar nonlocal neighbors is 20. We set , , . We obtain initialization of codes by standard sparse coding Yang2010 and let at the beginning of optimization. We set and . For the stage of IBP and AR regularization, the window size is . The maximum iteration times is set to . We set the step size . The parameters and are set to and .
For fairness of the comparisons, according to the experimental setting, we retrain the LRHR dictionary for Yang2010 and Chen2014 and change all downsampling and upscaling for Dong2013 and Dong2011 to bicubic and retrain its AR models and nonlocal adaptive regularization models before implementation.
3.2 Experimental results
The PSNRs and SSIMs of different methods for comparisons are shown in Table 1 for the scaling factor and Table 2 for the scaling factor , respectively. Our proposed method gets better quantitative SR performances on most of the test images than other methods. The average gains of our proposed method for the scaling factor over the second best method are 0.293dB in PSNR and 0.0078 in SSIM. In the case of , the average gains are 0.420dB in PSNR and 0.0117 in SSIM.
Image  Bi  SC  ASDS  LRNE  DM  NCSR  Aplus  Pro 

cubic  SRYang2010  Dong2011  SRChen2014  SRBevilacqua2014  Dong2013  Aplus2015  posed  
Lena  29.600  30.489  31.232  30.618  30.691  31.357  31.610  31.884 
0.8306  0.8556  0.8693  0.8523  0.8555  0.8747  0.8708  0.8791  
Girl  32.724  33.529  33.753  33.422  33.599  33.982  33.367  34.319 
0.8162  0.8386  0.8417  0.8321  0.8410  0.8489  0.8205  0.8540  
Butt  23.103  24.337  25.196  24.815  24.940  25.391  26.808  26.735 
erfly  0.7926  0.8375  0.8680  0.8553  0.8615  0.8754  0.8980  0.9003 
Parrot  27.406  28.488  29.309  28.339  28.728  29.355  29.460  29.503 
0.8692  0.8910  0.9025  0.8876  0.8932  0.9063  0.9049  0.9087  
Flower  26.935  27.949  28.489  27.885  28.177  28.581  28.802  29.230 
0.7594  0.8025  0.8215  0.7978  0.8113  0.8285  0.8382  0.8437  
Pepper  27.269  28.275  29.065  28.605  28.527  29.140  29.514  30.059 
0.8393  0.8628  0.8827  0.8697  0.8704  0.8862  0.8764  0.8967  
Bike  22.502  23.385  23.928  23.407  23.555  24.010  24.264  24.429 
0.6640  0.7235  0.7494  0.7177  0.7313  0.7563  0.7782  0.7721  
Hat  28.958  29.824  30.363  30.030  30.107  30.530  30.915  31.177 
0.8233  0.8454  0.8568  0.8508  0.8536  0.8641  0.8670  0.8739  
Parth  25.593  26.081  26.543  26.137  26.228  26.608  26.823  26.869 
enon  0.6683  0.6978  0.7118  0.6923  0.6992  0.7180  0.7315  0.7266 
Avg.  27.121  28.040  28.653  28.140  28.284  28.773  29.063  29.356 
0.7848  0.8172  0.8337  0.8173  0.8241  0.8398  0.8428  0.8506 
Image  Bi  SC  ASDS  LRNE  DM  NCSR  Aplus  Pro 

cubic  SRYang2010  Dong2011  SRChen2014  SRBevilacqua2014  Dong2013  Aplus2015  posed  
Lena  27.820  28.529  29.254  28.591  29.039  29.908  29.610  30.083 
0.7605  0.7849  0.8047  0.7838  0.8000  0.8234  0.8204  0.8252  
Girl  31.263  31.814  32.211  31.826  32.211  32.109  31.971  32.635 
0.7601  0.7768  0.7873  0.7743  0.7885  0.7751  0.7699  0.7978  
Butt  21.438  22.656  23.666  22.737  23.781  24.208  23.996  24.827 
erfly  0.7121  0.7555  0.8145  0.7721  0.8198  0.8295  0.8316  0.8511 
Parrot  25.512  26.416  26.966  26.276  26.796  27.428  27.228  27.183 
0.8145  0.8380  0.8521  0.8346  0.8486  0.8654  0.8638  0.8639  
Flower  25.259  26.113  26.452  25.952  26.510  26.850  26.706  27.022 
0.6689  0.7157  0.7306  0.7071  0.7366  0.7552  0.7498  0.7597  
Pepper  25.527  26.358  27.153  26.480  27.011  27.668  27.367  27.887 
0.7660  0.7866  0.8174  0.7925  0.8135  0.8296  0.8279  0.8340  
Bike  21.021  21.747  22.248  21.738  22.172  22.646  22.423  22.716 
0.5585  0.6170  0.6488  0.6108  0.6462  0.6774  0.6673  0.6800  
Hat  27.487  28.187  28.699  28.344  26.510  29.347  29.022  29.485 
0.7771  0.7933  0.8108  0.8009  0.7366  0.8277  0.8226  0.8319  
Parth  24.875  25.388  25.682  25.366  25.628  25.549  25.516  25.783 
enon  0.6296  0.6574  0.6683  0.6518  0.6659  0.6620  0.6593  0.6748 
Avg.  25.578  26.356  26.926  26.368  26.629  27.301  27.093  27.513 
0.7163  0.7472  0.7705  0.7476  0.7617  0.7828  0.7792  0.7909 
Fig. 68 show the visual SR results in the case of on the images girl butterfly and hat by different methods, respectively. Fig. 911 show the visual SR results of the same test images with the factor of . The SCSR method Yang2010 generates blurry along edges (i.e., the boundary of the girl’s nose) because the single overcomplete dictionary learned from the external training images is not prone to produce sharp edges. LRNESR Chen2014 tends to lose highfrequency details while smooth regions and clean edges are produced. As one of the stateoftheart methods for image SR, Aplus Aplus2015 obtains the second best quantitative SR performances (see Table 1 and 2), however, too sharp boundaries and ringing artifacts can also be observed. Our proposed method generates obvious boundaries and suppresses artifacts. We can see clear edges of girl s nose and natural patterns in the wing of butterfly. As seen from the visual experimental results, our proposed method gets better results than other methods perceptually.
3.3 Evaluation of the different contributions
To further validate the effectiveness of proposed method, we test the SR performances with the scaling factor using sparse coding with different constraints and regularization. The results are shown in Fig. 12. For the convenience of description, we denote sparse coding as ’SC’, sparse coding with lowrank constraint as ’LRSC’ and autoregressive as ’AR’. Our proposed KLRSC incorporated with AR model gains the highest PSNRs and SSIMs for all the reconstructions of images. Table 3 shows the average SR performance using SC, LRSC, KLRSC and KLRSC+AR. The proposed KLRSC+AR method has an average improvement of 0.664dB in PSNR and 0.0148 in SSIM over the method using standard sparse representation, where the average PSNR/SSIM contributions of the lowrank constraint, the kernel method and the AR regularization are 0.120dB/0.0028, 0.284dB/0.0041 and 0.260dB/0.0079, respectively. It indicates that the incorporation of lowrank constraint, kernel method and AR regularization indeed boosts the SR results.
SC  LRSC  KLRSC  proposed  
KLRSC+AR  
PSNR  SSIM  PSNR  SSIM  PSNR  SSIM  PSNR  SSIM  
Avg.  28.692  0.8358  28.812  0.8386  29.096  0.8427  29.356  0.8506 
4 Conclusion
In this paper, we propose a novel single image SR method by incorporating selfsimilarity learning framework with kernel based lowrank sparse coding. The kernel method is used which captures the nonlinear structures of the input data. A novel kernel based lowrank sparse coding based scheme for single image SR is proposed, which exploits both the structural information of nonlocalsimilarity in kernel space. Furthermore, we exploit the selfsimilarity redundancy among patches across different scales in a single natural image to train a selfexample dictionary. The gradual magnification framework compatible to the selfexample dictionary is adopted. Experimental results demonstrate that our proposed method improves SR performances both quantitatively and perceptually.
References
References
 (1) J. Sun, J. Sun, Z. Xu, H.Y. Shum, Gradient profile prior and its applications in image superresolution and enhancement, IEEE Transactions on Image Processing 20 (2011) 1529–1542. doi:10.1109/TIP.2010.2095871.
 (2) Y. Ogawa, Y. Ariki, T. Takiguchi, Superresolution by gmm based conversion using selfreduction image 22 (10) (2012) 1285–1288. doi:10.1109/ICASSP.2012.6288124.
 (3) C. V. Jiji, M. V. Joshi, S. Chaudhuri, Singleframe image superresolution using learned wavelet coefficients, International Journal of Imaging Systems and Technology 14 (14) (2004) 105–112. doi:10.1002/ima.20013.
 (4) A. Marquina, S. J. Osher, Image superresolution by tvregularization and bregman iteration, J. Sci. Comput. 37 (2008) 367–382. doi:10.1007/s1091500892148.
 (5) M. Irani, S. Peleg, Motion analysis for image enhancement: resolution, occlusion, and transparency, J. Vis. Commun. Image Represent. 4 (1993) 324–335. doi:10.1006/jvci.1993.1030.

(6)
H. Chang, D.Y. Yeung, Y. Xiong, Superresolution through a neighbor embedding, in: IEEE Conference on Computer Vision and Pattern Recognition, 2004, pp. 275–282.
doi:10.1109/CVPR.2004.243.  (7) J. Yang, J. Wright, T. Huang, Y. Ma, Image superresolution via sparse representation, IEEE Transactions on Image Processing 19 (2010) 2861–2873. doi:10.1109/TIP.2010.2050625.
 (8) X. Ma, J. Zhang, C. Qi, Hallucinating face by positionpatch, Pattern Recognition 43 (6) (2010) 2224–2236. doi:10.1016/j.patcog.2009.12.019.
 (9) D. Glasner, S. Bagon, M. Irani, Superresolution from a single image, in: IEEE International Conference on Computer Vision, 2009, pp. 349–356. doi:10.1109/ICCV.2009.5459271.

(10)
K. Zhang, X. Gao, D. Tao, X. Li, Single image superresolution with multiscale similarity learning, IEEE Transactions on Neural Networks and Learning Systems 24 (10) (2013) 1648–1659.
doi:10.1109/TNNLS.2013.2262001.  (11) J. Yu, X. Gao, D. Tao, X. Li, K. Zhang, A unified learning framework for single image superresolution, IEEE Transactions on Neural Networks and Learning Systems 25 (2014) 780–792. doi:10.1109/TIP.2010.2050625.
 (12) M. Bevilacqua, A. Roumy, C. Guillemot, M.L. Morel, Singleimage superresolution via linear mapping of interpolated selfexamples, IEEE Transactions on Image Processing 23 (2014) 5334–5347. doi:10.1109/TIP.2014.2364116.
 (13) W. Dong, L. Zhang, G. Shi, X. Wu, Nonlocally centralized sparse representation for image restoration, IEEE Transactions on Image Processing 20 (2011) 1838–1857. doi:10.1109/TIP.2012.2235847.
 (14) J. Shi, C. Qi, Lowrank sparse representation for single image superresolution via selfsimilarity learning, in: IEEE International Conference on Image Processing, 2016, pp. 1424–1428. doi:10.1109/ICIP.2016.7532593.

(15)
B. Schölkopf, A. Smola, K.R. Müller, Kernel principal component analysis, in: International Conference on Artificial Neural Networks, 1997, pp. 583–588.
doi:10.1007/BFb0020217.  (16) C.Y. Yang, J.B. Huang, M.H. Yang, Exploiting selfsimilarities for single frame superresolution, in: Asian Conference on Computer Vision, 2010, pp. 497–510. doi:10.1007/9783642193187_39.
 (17) W. Dong, L. Zhang, G. Shi, X. Wu, Image deblurring and superresolution by adaptive sparse domain selection and adaptive regularization, IEEE Transactions on Image Processing 20 (2011) 1838–1857. doi:10.1109/TIP.2011.2108306.
 (18) J. Lu, H. Zhang, Y. Sun, Video super resolution based on nonlocal regularization and reliable motion estimation, Signal Processing: Image Communication 29 (4) (2014) 514 – 529. doi:http://dx.doi.org/10.1016/j.image.2014.01.002.
 (19) J. Mairal, F. Bach, J. Ponce, G. Sapiro, A. Zisserman, Nonlocal sparse models for image restoration, in: IEEE International Conference on Computer Vision, 2009, pp. 2272–2279. doi:10.1109/ICCV.2009.5459452.
 (20) H. Chen, X. He, Q. Teng, C. Ren, Single image super resolution using local smoothness and nonlocal selfsimilarity priors, Signal Processing: Image Communication 43 (2016) 68 – 81. doi:http://dx.doi.org/10.1016/j.image.2016.01.007.
 (21) T. Zhang, B. Ghanem, S. Liu, C. Xu, N. Ahuja, Lowrank sparse coding for image classification, in: IEEE International Conference on Computer Vision, 2013, pp. 281–288. doi:10.1109/ICCV.2013.42.
 (22) B. Recht, M. Fazel, P. A. Parrilo, Guaranteed minimumrank solutions of linear matrix equations via nuclear norm minimization, SIAM Review 52 (3) (2010) 471–501. doi:10.1137/070697835.
 (23) E. J. Candes, X. D. Li, Y. Ma, J. Wright, Robust principal component analysis?, Journal of the ACM 58 (2011) 11:1–11:37. doi:10.1145/1970392.1970395.
 (24) X. Liu, G. Zhao, J. Yao, C. Qi, Background subtraction based on lowrank and structured sparse decomposition, IEEE Transactions on Image Processing 24 (2015) 2502–2514. doi:10.1109/TIP.2015.2419084.
 (25) Z. Zhang, A. Ganesh, X. Liang, Y. Ma, Tilt: Transform invariant lowrank textures, International Journal of Computer Vision 99 (2012) 1–24. doi:10.1007/s112630120515x.
 (26) Y. Tang, Y. Yuan, Learning from errors in superresolution, IEEE Transactions on Cybernetics 44 (2014) 2143–2154. doi:10.1109/TCYB.2014.2301732.
 (27) K. Ren, F. Xu, Superresolution images fusion via compressed sensing and lowrank matrix decomposition, Infrared Physics and Technology 68 (2015) 61–68. doi:10.1016/j.infrared.2014.11.006.
 (28) S. Wang, S. Lin, X. Liang, B. Yue, L. Jiao, External and internal learning for singleimage superresolution, in: IEEE International Conference on Image Processing, 2015, pp. 128–132. doi:10.1109/ICIP.2015.7350773.

(29)
S. Gao, I. W.H. Tsang, L.T. Chia, Kernel sparse representation for image classification and face recognition, in: European Conference on Computer Vision, Heraklion, 2010, pp. 1–14.
doi:10.1007/9783642155611_1.  (30) J. Shi, C. Qi, Kernelbased face hallucination via dual regularization priors, IEEE Signal Processing Letters 22 (8) (2015) 1189–1193. doi:10.1109/LSP.2015.2390972.
 (31) Z. Lin, A. Ganesh, J. Wright, L.Wu, M. Chen, Y. Ma, Fast convex optimization algorithms for exact recovery of a corrupted lowrank matrix, in: Technical Report UILUENG092214, 2009.
 (32) X. Chen, C. Qi, Lowrank neighbor embedding for single image superresolution, IEEE Signal Processing Letters 21 (2014) 79–82. doi:10.1109/LSP.2013.2286417.
 (33) R. Timofte, V. De Smet, V. Luc, A+: Adjusted anchored neighborhood regression for fast superresolution, in: Asian Conference on Computer Vision, 2015, pp. 111–126. doi:10.1007/9783319168173_8.
 (34) Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing 13 (4) (2004) 600–612. doi:10.1109/TIP.2003.819861.