I Introduction
Image superresolution provides a low cost, softwarebased technique to improve the spatial resolution of an image beyond the limitations of the imaging hardware devices. The areas of application include medical imaging [1] and satellite imaging [2] and highdefinition television (HDTV). In such cases, it is standard to assume that the observed lowresolution image(s) is a blurred and downsampled version of the highresolution image. The goal is then to recover the highresolution image using its lowresolution observation(s). Meanwhile, with the growing capabilities of highresolution displays, effective image superresolution algorithms is essential for us to make the best use of such devices.
The most typical superresolution methods require multiple lowresolution images, with subpixel accuracy alignment. In this approach superresolution can be considered as an inverse problem, where it is essential to assume a prior on the solution to regularize the illposed nature of the problem and avoid infinitely many solutions [3, 4].
In some applications, the number of lowresolution images is limited and it is desired to reconstruct the high resolution image from a single lowresolution image. One approach to overcome this limitation is to use interpolation methods
[5, 6, 7], where unknown pixels of the highresolution image are estimated using nearby known pixels from the lowresolution image based on some assumptions on the relation between pixels. Simple interpolation methods such as bilinear and bicubic interpolations result in blued images with ringing artifacts near edges. While more advanced methods
[6, 7] try to avoid this problem by exploiting natural image priors, they are often limited in accounting for the complexity of images which have regions with fine textures or smooth shadings. As a result, watercolorlike artifacts are often observed in some regions.A more successful class of methods for single input image superresolution are learning based methods. Here a cooccurrence prior between the highresolution and lowresolution images is used to reconstruct the highresolution image [8, 9, 10, 11]. The learning procedure has been done via different schemes such as Markov Random Field (MRF), Primal Sketch and Locally Linear Embedding (LLE) [8, 9, 10]. These algorithms require enormous databases to handle the learning stage and thus are computationally expensive. One of the recently developed successful methods [11] uses sparsity as the cooccurrence prior between the highresolution and lowresolution images. This approach reduces the size of the training database and consequently the computational load. In this method it is assumed that both images share the same sparse representation over a pair of jointly trained dictionaries. Using this assumption we can find the jointly sparse representation via the lowresolution image and then use it to recover the highresolution image. Finding the jointly sparse representation properly, is important and influences the quality of the reconstruction result. In the current paper our intention is to improve the performance of this algorithm to find the jointly sparse representation more accurately and thus to improve the quality of the reconstructed highresolution image.
Sparse representation of a vector over a known dictionary is an illposed, combinatorial optimization problem
[12]. Several relaxation approaches have been used to convexify this problem. The most common approaches are Matching Pursuit (MP)[13], Basis Pursuit (BP) [14], and Focal Underdetermined System Solution (FOCUSS) [15]. In this paper, we adopt a recently proposed Smoothed norm (SL)) algorithm [16], a faster solver for sparse representation, to lessen the computational load and to improve the quality of the reconstructed highresolution image.The rest of the paper is organized as follows. In section II, we briefly present the approach that is used to decompose a signal on an overcomplete dictionary. Section III is devoted to the superresolution algorithm and its alternation. Section VI summarizes experimental results. The paper is finally concluded in section V.
Ii Sparse Representation via Smoothed Norm
Finding the sparsest representation of a source signal over an overcomplete dictionary is formulated under the topic of compressed sensing (CS) theory [12, 17]. Let be the source signal, and the dictionary (). Our goal is to find the sparsest solution of the following linear system:
(1) 
where . The recovery of from based on (1) is impossible to implement in a unique and stable way, unless it is known that is sparse enough to have a relatively low value of [12]. This assumption is only valid if the dictionary is chosen or learned properly. Several algorithms have been provided to design such dictionaries [18, 19, 20].
Equivalently (1) can be formulated as the following optimization problem:
(2) 
Due to the highly nonconvex nature of norm, this problem is illposed and intractable. In conventional CS theory [12, 17] it is proven that if the dictionary satisfies the restricted isometry property (RIP) [12, 17] with respect to a certain class of sparse signals to which is assumed to belong, then can be recovered as a solution to [21, 22]
(3) 
which is a convex minimization problem. It is straightforward to reformulate this equivalent problem in terms of linear programming. This approach results in a tractable problem but is still timeconsuming for large scale systems. It is also important to note that the equivalence between
norm and norm is only valid asymptotically and does not always hold [23]. In a different approach, the norm is approximated directly by a smooth convex function [16]. This approach has proved to be faster with possibility of resulting in sparser representation [16].Consider the smooth function, . As approaches zero, we have the following equivalence:
(4) 
This equivalence does not help us in practice. However, one can assume that if is set to be nonzero and sufficiently small then we can approximate the norm of a vector by
(5) 
This enables us to approximate the norm with a smooth, differentiable function. This is the key fact that enables us to replace the norm minimization with a convex problem, so that we can take advantage of common techniques, such as steepest descent, to tackle the optimization problem. The value of controls the tradeoff between the closeness to the norm and the smoothness of the approximation. Now if we define , then the minimization of can be done by maximizing by choosing a proper value for . Due to the nonconvex nature of the norm, will archive a lot of local extreme points for small values of . Consequently, finding the global maxima will become difficult. On the other hand, if the value of is chosen to be sufficiently large, there will be no local maxima [16] and asymptotically the solution for is equivalent to the norm solution. Considering these facts, the authors of [16] provided Algorithm 1 to solve the optimization problem .
Here, the final estimation of each step is used for the initialization of the next steepest ascent. By a proper selection of the sequence of , we may avoid being trapped in the local maxima. Compared to conventional CS solvers, this algorithms proves to be faster with the possibility of recovering a sparser solution [16].
Iii Image SuperResolution Based on Sparse Representation
Sparse representation has been applied to multiple inverse problems in image processing such as denoising [24], restoration [25] and superresolution [11, 4]. Generally sparsity is used as a prior on source signal to avoid illposed nature of inverse problems. In such applications, there exist a stage, which involves expansion of a source signal over an overcomplete dictionary, sparsely. The output quality of these algorithms depends on the accuracy in finding the sparse representation. In this section, SL0 algorithm is employed in a superresolution algorithm to improve quality of the output highresolution image.
Assume that the lowresolution image is produced from a high resolution image by
(6) 
where represents a blurring matrix, and is a downsampling matrix. The recovered highresolution output image must be consistent with the lowresolution input image. This problem is highly illposed and infinitely many solutions satisfy (6) and are consistent with lowresolution image. To provide a unique solution, local sparsity model maybe applied as a prior. We assume that there exist two dictionaries, and , for which each patch of low, , and high, , resolution images can be represented sparsely simultaneously and jointly as follows:
(7) 
These coupled dictionaries are trained simultaneously and jointly over a set of low/high resolution images such that both low/high resolution images result in the same sparse representation coefficients. Having the dictionaries trained, for each patch of our low resolution image we need to calculate the sparse representation. The authors of [11] used norm minimization method for this propose. Here Algorithm 1 is adopted. Having the sparse representation, calculated using the lowresolution patch, we reconstruct the high resolution image patches using the high resolution dictionary, . The patches are chosen to overlap so as to reduce the artifacts in patch boundaries. Next we regularize and merge the patches to produce an entire image using the reconstruction constraint (6). These procedure can be formulated as the following optimization problems:
(8) 
(9) 
where is a matrix that extracts the () block from the image, is the dictionary with , is the regularization parameter, is the image obtained by averaging the blocks obtained using sparse representation, and is the sparse vector of coefficients corresponding to the () block of the image. Here, (8) refers to the sparse coding of local image patches with bounded prior, hence building a local model from sparse representations. On the other hand, (9) demands the proximity between the lowresolution image, , and the output image , thus enforcing the global reconstruction constraint. In [11], norm minimization is used to solve (8) , whereas we use SL0 algorithm to solve this stage. The solution to (9) can be done iteratively using a gradient descent algorithm as follows:
(10) 
where is the estimation of the highresolution image after the th iteration, and is the step size.
The proposed image superresolution algorithm is summarized in Algorithm 2.
Compared to the original algorithm in [11], reduction in the computational complexity and the possibility of improving the output quality are expected.
Iv Experimental Results
In this section we compare the proposed superresolution algorithm with bicubic interpolation and the method given in [11]. The image super resolution methods are tested on various images. To be consistent with [11], patches of pixels were used on the low resolution image and the scaling factor was set to . Each patch is converted to a vector with length . The trained dictionaries, provided by authors of [11], with the sizes of and for the low and the high resolution dictionaries were used, respectively. To remove artifacts on the patch boundaries we set a overlap of one pixel in the patches.
Fig. 1 and Fig. 2 (subplots (a) and (e)) depict the original Pepper and Barbara images and their corresponding lowresolution versions. In the same figures subplots (bd) depict reconstructed highresolution images using the proposed methods. Subplots (fh) depict the corresponding SSIM maps [26]. A close look on the reconstructed images, enlarged image regions (subplots (il)), and the corresponding SSIM maps shows that while bicubic method works pretty well in smooth regions, significant blurring occurs on edges. The method of [11] is able to recover the edges better but does not work as well in smooth regions. Compared to [11] our approach is able to recover the edges and meanwhile it works better in smooth regions. One possible approach for further improving the image quality might be using a combination of bicubic and our approach. The quantitative results for different images reconstructed from different algorithms are shown in Table 1.
Image  Barbara  Lena  Baboon  House  Watch  Pepper  Parthenon  Splash  Aeroplane  Tree  Girl  Bird  Average 

PSNR comparison (in dB)  
Bicubic  27.08  32.70  26.34  33.97  26.94  30.84  28.07  33.58  28.50  29.41  35.12  29.49  30.17 
Yang et al.  27.15  33.33  26.41  34.00  27.32  30.41  28.41  33.87  29.02  29.60  34.52  29.63  30.31 
Proposed  27.13  33.45  26.52  33.97  27.41  30.55  28.68  34.13  29.24  29.67  35.25  29.82  30.49 
SSIM comparison  
Bicubic  0.744  0.876  0.700  0.870  0.790  0.869  0.763  0.871  0.857  0.852  0.925  0.877  0.832 
Yang et al.  0.747  0.884  0.702  0.873  0.806  0.850  0.765  0.918  0.868  0.857  0.901  0.882  0.837 
Proposed  0.746  0.894  0.727  0.886  0.808  0.857  0.770  0.922  0.892  0.861  0.925  0.886  0.847 
Image  Barbara  Lena  Baboon  House  Watch  Pepper  Parthenon  Splash  Aeroplane  Tree  Girl  Bird  Average 

Yang et al.  13.79  12.07  12.07  12.07  15.81  13.26  26.90  11.14  13.23  13.95  15.08  12.54  14.33 
Proposed  1.48  1.39  1.38  1.39  1.49  1.53  2.87  1.36  1.87  1.44  1.43  1.39  1.67 
Time saving  89.3 %  86.2 %  86.2 %  86.2 %  90.6 %  88.5 %  89.3 %  88.1 %  85.9 %  89.7 %  90.5 %  88.9 %  88.3 % 
All the highresolution output images have been compared with their original counterparts in terms of PSNR as well as of the structural similarity index (SSIM) of [26], which is believed to be a better indicator of perceptual image quality [27]. It can be observed that the proposed method outperforms the other methods in terms of both SSIM and PSNR in most cases and on average it outperforms both methods.
We have also included execution times for Yang et. al and our approach in Table 2. Comparison of execution times also confirms that our approach is about an order faster than the method in [11]. This result is expected, SL0 is estimated to be about one to twoorder faster than norm based minimization methods.
V Conclusion
In this paper, we attempt to take advantage of the SL0 sparse coding solver in order to improve one of thestateoftheart single input image superresolution algorithms based on sparse signal representation. Compared with the method in [11], our approach significantly reduces computational complexity, and yet improves the output perceptual quality. Our simulations demonstrate the potential of the SL0 algorithm in improving the current image processing algorithms that use sparse coding in one of their stages. In the future, the algorithm maybe further improved by advanced design of the dictionary.
Acknowledgement
This work was supported in part by the Natural Sciences and Engineering Research Council of Canada and in part by Ontario Early Researcher Award program, which are gratefully acknowledged. The authors would also like to acknowledge Jianchao Yang for providing superresolution codes.
References
 [1] H. Greenspan, “Superresolution in medical imaging,” Computer Journal, vol. 52, no. 1, pp. 43–63, 2009.

[2]
F. Zhu, J. Li, B. Zhu, D. Li, and X. Yang,
“Superresolution image reconstruction based on rbf neural network,”
Opt. Precis. Eng., vol. 18, no. 6, pp. 1444–1451, 2010.  [3] R. C. Hardie, K. J. Barnard, and E. A. Armstrong, “Joint map registration and highresolution image estimation using a sequence of undersampled images,” IEEE Transactions on Image Processing, vol. 6, pp. 1621–1633, 1997.
 [4] S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multiframe superresolution,” IEEE Transactions on Image Processing, vol. 13, pp. 1327–1344, 2004.
 [5] H. S. Hou and H. C. Andrews, “Cubic spline for image interpolation and digital filtering,” IEEE Transactionson Signal Processing, vol. 26, pp. 508–517, 1987.

[6]
S.Dai, M. Han, W. Xu, Y. Wu, and Y. Gong,
“Soft edge smoothness prior for alpha channel superresolution,”
in
in IEEE Conference on Computer Vision and Pattern Classification (CVPR)
, 2007, pp. 1–8.  [7] J.Sun, Z. Xu, and H. Shum, “Image superresolution using gradient profile prior,” in in IEEE Conference on Computer Vision and Pattern Classification (CVPR), 2008, pp. 1–8.
 [8] J.Sun, Z. Xu, and H. Shum, “Learning lowlevel vision,” in IJCV, 2000.
 [9] C. Liu, H. Y. Shum, and W. T. Freeman, “Face hallucination: theory and practice,” IEEE Transactionson Signal Processing, vol. 75, pp. 115–134, 2007.
 [10] Q. Wang, X. Tang, and H. Shum, “Patch based blind image super resolution,” in Proc. ICCV, 2005.
 [11] J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image superresolution via sparse representation,” IEEE Transactions on Image Processing, vol. 19, pp. 2861–2873, 2010.
 [12] E. J. Candés, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Info. Theory, vol. 52, no. 2, pp. 489–509, 2006.
 [13] S. Mallat and Z. Zhang, “Matching pursuits with timefrequency dictionaries,” IEEE Transactions on Signal Processing, vol. 41, pp. 3397–3415, 1993.
 [14] C. E. Shannon, “Communication in the presence of noise,” Proceedings of the IRE, vol. 37, no. 1, pp. 10–21, 1949.
 [15] S. Mallat and Z. Zhang, “Sparse signal reconstruction from limited data using focuss:a reweighted minimum norm algorithm,” IEEE Transactions on Signal Processing, vol. 45, pp. 600–616, 1997.
 [16] G. Hosein Mohimani, Massoud BabaieZadeh, and Christian Jutten, “A fast approach for overcomplete sparse decomposition based on smoothed l0 norm,” IEEE Transactions on Signal Processing, vol. 57, pp. 289–301, 2009.
 [17] Y. Tsaig and D. L. Donoho, “Compressed sensing,” IEEE Trans. Inform. Theory, vol. 52, pp. 1289–1306, 2006.
 [18] M. Aharon, M. Elad, and A. Bruckstein, “Ksvd: An algorithm for designing overcomplete dictionaries for sparse representation,” EEE Transactions on In Signal Processing, vol. 54, pp. 4311–4322, 2006.

[19]
Z. Szabo, B. Poczos, and A. Lorincz,
“Online groupstructured dictionary learning,”
in
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)
, 2011.  [20] M. Aharon, M. Elad, and A. Bruckstein, “Dictionary learning algorithms for sparse representation,” Neural computation, vol. 15, pp. 349–396, 2003.
 [21] D. L. Donoho and Y. Tsaig, “Fast solution of norm minimization problems when the solution may be sparse,” 2006.
 [22] W. Yin, S. Osher, D. Goldfarb, and J. Darbon, “Bregman iterative algorithms for minimization with applications to compressed sensing,” SIAM J. Imaging Sciences, vol. 1, no. 1, pp. 143–168, 2008.
 [23] E. J. Candés, M. B. Wakin, and M. B. Boyd, “Enhancing sparsity by reweighted l1 minimization,” Journal of Fourier Analysis and Applications, pp. 877–905, October 2008.
 [24] M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Transactions on Image Processing, vol. 17, pp. 3736–3745, 2006.
 [25] IJ. Mairal, G. Sapiro, and M. Elad, “Learning multiscale sparse representations for image and video restoration,” Multiscale Modeling and Simulation, vol. 7, pp. 214–241, 2008.
 [26] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600–612, 2004.
 [27] Z. Wang and A. C. Bovik, “Mean squared error: love it or leave it?  a new look at signal fidelity measures,” IEEE Signal Processing Magazine, vol. 26, no. 1, pp. 98–117, 2009.
Comments
There are no comments yet.