Current Issue Cover

樊逸清,李海晟,楚东东(华东师范大学计算机科学技术系, 上海 200062;中国银行软件中心上海分中心, 上海 201201)

摘 要
目的 图像配准是影响拼接质量的关键因素。已有的视差图象拼接方法基本上都利用全局变换配准图像,这类模型容易造成匹配特征点对间的错误配准,引起不自然的拼接痕迹。针对这一问题,提出了使用线约束运动最小二乘法的配准算法,减少图像的配准误差,提高拼接质量。方法 首先,计算目标图像和参考图像的SIFT特征点,应用RANSAC方法建立特征点的匹配关系,由此计算目标到参考图像的最佳单应变换。然后,使用线约束运动最小二乘法分别配准两组图像:目标图像和参考图像、经单应变换后的目标图像和参考图像。前者使用逐点仿射变换进行配准,而后者配准使用了单应变换加上逐点仿射变换。最后,在重叠区域,利用最大流最小割算法寻找最优拼接缝,沿着拼接缝评估两组配准的质量,选取最优的那组进行融合拼接。结果 自拍图库和公开数据集上的大量测试结果表明本文算法的配准精度超过95%,透视扭曲比例小于17%。与近期拼接方法相比,而本文的配准算法精度提高3%,拼接结果中透视扭曲现象减少73%。结论 运动最小二乘法可以准确地配准特征点,但可能会扭曲图像中的结构对象。而线约束项则尽量保持结构,阻止扭曲。因此,线约束运动最小二乘法兼顾了图像结构的完整性和匹配特征点的对准精度,基于此配准模型的拼接方法能够有效减少重影和鬼影等人工痕迹,拼接结果真实自然。
Parallax image stitching using line-constraint moving least squares

Fan Yiqing,Li Haisheng,Chu Dongdong(Department;of Computer science and technology, East China Normal University, Shanghai, 200062;Software center Shanghai branch, Bank of China ,Shanghai 201201)

Objective Image alignment is a key factor to determine the stitching performance. As we know, image deformation is a critical step of alignment model for parallax image stitching, and directly determines aligning quality. Since it is a hard challenge to accurately align all points in the overlapping region of parallax images, what we need is an alignment strategy which can produce visually pleasurable stitching results. The recently state-of-the-art stitching methods almost combine homography with content-preserving warping. Either they first use homography to pre-align two images, and then apply content-preserving warp to refine alignment, or they globally optimize mesh deformation by solving an energy function that is a weighting linear combination of homography and content-preserving warp. As both approaches commonly use homography in the aligning phase, they easily produce perspective distortion. Meanwhile, they still maybe misalign object’s edges when handling the image with several dominant structural objects. To address these problems, this paper presents a novel combination stitching method, which utilizes homography, deformation using moving least squares (MLS for short hereinafter) and line constraint. The deformation method based on MLS has interpolation property, so that it is able to align matching feature points as accurate as possible. However, this deformation method maybe distort some structural regions, so line constraint item is added to deformation model for preserving structure.Method For clear depiction, here we take the two-image stitching as an example. The input two images are called target and reference image separately, and denoted by T and R individually. Firstly, Feature detection and matching estimation using SIFT and RANSAC, and furthermore using distance similarity to check matching accuracy of feature points. Meanwhile, the homography denoted by H with the best geometric fit is selected. Then, Apply H to the target image T, and denote the transformed image by TH. Afterwards, the two group images (T, R) and (TH, R) are respectively aligned using line constraint MLS. To eliminate perspective distortion in the deformation image, the affine transformation is used in moving least squares. However, a simple affine transformation is not enough to handle parallax, so an additional pair of images (TH, R) is processed as candidate stitching result for the pair of images (T, R). The test experiments show that many examples obtain more natural stitching result only utilizing affine transformation rather than the composite transformation of homography and affine transformation, i.e. the alignment between T and R is better than that of TH and R. Now, taking the deformation from target image T to reference image R for instance, the line constraint MLS is outlined as follows. First the four corner points of T are deformed to the coordinate system of R using matching feature points as control points based on MLS, and then we deform the rest points on the four border lines (top, bottom, left and right boundaries) of T using line constraint MLS, and here line constraint is constructed based on the requirement preserving the relative position of each point of a border line, and forms deformation objective function as a constraint item. Similarly, we handle the internal points of T using vertical and horizontal grid lines as constraint conditions, and the vertical and horizontal grid lines are the constraint lines of their intersection point. Finally, evaluate the quality of each alignment and choose the best one to blend them. In the overlapping regions, use the max-flow min-cut algorithm to respectively find best stitching seam-cut of two alignments, and assess the alignment quality along the seam-cut. The aligning quality assessment mainly considers the color and structure differences between overlapping regions of two images, and the structure is roughly represented by gradient. Then, use feathering approach to blend the two images of the best alignment.Result To test our stitching algorithm, 23 pairs of pictures are taken, which cover natural and man-made scenes commonly seen. In addition, we made several experiments on publicly published data provided by recently related works. The experimental results demonstrate that the alignment accuracy of our method is greater than 95%, and the ratio of perspective distortion is lower than 17%. Compared with recently state-of-the-art methods, the alignment accuracy increases by 3%, and the ratio of perspective distortion decreases by 73%. Therefore, our method has better performance in handling image stitching with large parallax, and stitching result is authentic and natural.Conclusion This paper presents a hybrid transformation for aligning two images, which combines line constraint with moving least squares. Meanwhile, an alignment quality evaluation rule is introduced by computing weighted differences of the points along the stitching seam-cut and the remaining points in the overlapping region. As the proposed method makes a balance between image aligning accuracy and structure preserving, it addresses the misalignment issues easily caused by current stitching approaches for parallax image, and effectively reduces stitching artifacts, such as ghosting and distortion etc.