I Introduction
Image stitching has been well studied and widely applied in computer vision, where the most crucial step is to determine a 2D warp for each image and transform it into a common plane. Among these warps, similarity warp can preserve the angles between lines and maintain a uniform scale factor (ratio). Homography is the most flexible and popular global warp which preserves all straight lines, however, it introduces projective distortion at the same time. Basically, the projective distortion usually consists of distortion in vertical and horizontal. Many efforts have been devoted to mitigating these projective distortion. AutoStitch
[1] uses cylindrical warp as preprocessing step to address this problem, however, it inevitably curves straight lines and suffers from horizontal distortion. SPHP [2] takes advantage of similarity to maintain the original perspective of input image but suffers from linebending. QH [3] uses the proposed quasihomography warp to mitigate projective and perspective distortion simultaneously, but there still exists horizontal stretches and vertical distortion. In this letter, an effective algorithm based on the property of cylindrical warp is proposed to mitigate the projective distortion produced by homography. The strategy of image resizing for seamcutting significantly decreases the processing time in different resolutions. Fig. 1 shows the flowchart of our proposed algorithm.Ii Proposed halfcylindrical warp
Cylindrical warp [4] is often applied to image stitching to reduce the projective distortion produced by homography and get a multiview panorama when the camera is assumed to be rotating around its vertical axis. A sketch of cylindrical warp is illustrated in Fig. 2. Generally, let be the focal length of the cylinder, then a plane 3D point is corresponding to the 3D cylindrical points , and the 2D point after cylindrical warp is computed by
(1) 
Suppose to be the center of the cylinder, then the warping function can be rewritten as
(2)  
(3) 
Note that if is a fixed constant, then both and are linear functions of ,
(4) 
Cylindrical warp maps the vertical line to (itself), which preserves the value of points and ratio of lengths on this line. In this manner, we can divide the image plane into two regions by the line , and separately manipulate homography warp and a composition of homography and cylindrical warps on each region.
Given an image with height and width , we let the upperleft point to be the origin, and axis towards right, axis towards down. The warping function of halfcylindrical warp is defined as
(5) 
where denotes the homography warp, and are two regions divided by a line to apply different warps. The composition of homography and cylindrical warps efficiently mitigates the projective distortion in vertical, however, there is still some horizontal stretches or squeezes which makes the image unnatural, see Fig. 3.
Iii Pixel selection for ratiopreserving
The composition of homgraphy and cylindrical warps can only reduce the vertical distortion efficiently. A novel pixel selection strategy is proposed to preserve image width when employing warping function in the nonoverlapping region. Without ambiguity, we use to denote original points, to denote points after homography, to denote points after composition of homography and cylindrical warps. Our strategy is proceeded by:

Using feature points to estimate a similarity warp
(6) and compute the scale of pixel selection by , where and are the translation in horizontal and vertical respectively.

For any integer , selecting the points (where ) on horizontal lines among the image domain as sample points, where and is defined by
(7) Recording these sampling points which are in the nonoverlapping region after homography and cylindrical warps by .

Constraining the horizontal distance of these neighboring sample points to be one pixel in the final canvas, then applying the inversion map of cylindrical and homography warps on points and manipulating bilinear interpolation when the transformed coordinate is not an integer.
As we can see from the above procedures, the pixel selection strategy preserves the ratio as similarity in the nonoverlapping region after . Moreover, the inversion of warps and interpolation make the results without blurring. Therefore, by combining warp and pixel selection strategy we finally get a more natural stitched result with less vertical and horizontal distortion.
Iv Parameter estimation
It is worth noting that there are some crucial parameters in Eq. (2,3), which should be carefully dealt with before applying halfcylindrical warp . In the following, we only consider horizontal stitching, vertical stitching can be obtained by a similar manner.
Firstly, we need to determine the abscissa of center. Since our final stitched result is divided by a line for applying different warps by its side, we set to be or (depending on the position of the target image). This line also divides the image into overlapping region and nonoverlapping region, and the coordinate of homography and cylindrical warp on this line equals each other, thus we ensure the continuity of the stitched image.
Then, the ordinate of center should make the pair of corresponding points (e.g., and ) between vertical boundaries with minimal offset, as shown in Fig. 4. We subsequently sample and track the points on the left and right boundaries of original image. Suppose and to be the ordinate of warped points under homography on the left and the right boundary respectively, then is estimated by
(8) 
In fact, there exists a unique horizontal line that remains horizontal, i.e., the offset equals , more details can be found in [3]. Our estimation can be regarded as the discretization process of that method.
Finally, an appropriate focal length should be carefully estimated, since with too large or too small focal length can greatly affect the quality of the final stitched results. The primary purpose of our cylindrical warp is trying to make the image preserve the average height as original as possible after warping. For point after homography, the height in position after the composition of homgoraphy and cylindrical warps, denoted by , can be derived from Eq. (3)
(9) 
where is the height after homography warp in (see the green line in Fig. 4). Then, we estimate focal length by solving the following nonlinear error minimisation task
(10) 
where is the height of desired average reference height after cylindrical warp. We set
(11) 
to preserve the average height after homography. The selection of determines the focal length then further changes the shape of final results.
V Experiments
In our experiments, we employ SIFT [6] to extract and match features, RANSAC [7] to estimate a global homography, and seamcutting to blend the overlapping region.
For naturalness, we compare our warp with homography, SPHP [2] and QH [3]. Fig. 5 illustrates the naturalness comparison of stitched results, it is clear that our halfcylindrical ratiopreserving warp looks more natural than other warps in aspects of distortion in horizontal and vertical.
For efficiency, we resize the input images to of original size to speed up the timeconsuming seamcutting stage, then map the seam to original image for blending. Generally, our warp needs very little time, and the average total processing time for different resolutions of proposed method is significantly reduced as shown in Table I.
resolution (pixel)  warp time (s)  total time (s)  

original  after resizing  
15002000  0.09  59.71  1.43 
19202560  0.16  127.13  2.18 
24483264  0.26  195.47  3.65 
Vi Conclusions
In this letter, a novel halfcylindrical ratiopreserving warp utilizing the property of cylindrical warp and a horizontal pixel selection strategy has been proposed to mitigate the projective distortion produced by homography. The parameter estimation stage further guarantees the robustness and naturalness of our warp. Experiments demonstrated that our warp can effectively mitigate projective distortion both in horizontal and vertical, and possesses high time efficiency.
References
 [1] Brown, M. and Lowe, D. G.: ‘Automatic panoramic image stitching using invariant features’, Int. J. Comput. Vis., 2007, 74, (1), pp. 5973
 [2] Chang, C.H., Sato, Y. and Chuang, Y.Y.: ‘Shapepreserving halfprojective warps for image stitching’, Proc. of IEEE CVPR, 2014, pp. 32543261
 [3] Li, N., Xu, Y. F. and Wang, C.: ‘Quasihomography warps in image stitching’, IEEE Trans. Multimedia, 2017, online
 [4] Szeliski, R., and Shum, H.Y.: ‘Creating full view panoramic image mosaics and environment maps’, Proc. of ACM SIGGRAPH, 1997, pp. 251258
 [5] Szeliski, R.: ‘Computer Vision: Algorithms and Applications’, 2010
 [6] Lowe, D. G.: ’Distinctive image features from scaleinvariant keypoints’, Int. J. Comput. Vis., 2004, 60, (2), pp. 91110
 [7] Fischeler, M. A., and Bolles, R. C.: ’Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography’, Commun. ACM, 1981, 24, (6), pp. 381395
Comments
There are no comments yet.