Current Issue Cover
结合最佳缝合线和多分辨率融合的图像拼接

谷雨,周阳,任刚,冯秋晨,鲁国智(杭州电子科技大学;北京宇航系统工程研究所;杭州电子科技大学 通信息传输与融合技术国防重点学科实验室)

摘 要
目的:针对图像拼接过程中,缝合线通过运动物体或配准不准确区域等情况导致融合图像出现鬼影、重影的问题,提出了一种基于差异图像加权的改进最佳缝合线算法,采用基于多分辨率和加权平均的分区图像融合算法解决了拼接线问题。方法:该算法首先将两幅图像的重叠区域划分为缝合线区域和过渡区域;在缝合线区域内,使用差异图像加权的最佳缝合线搜索准则构建准则值图像,基于动态规划思想来搜索得到最佳缝合线;基于缝合线生成掩码图像,并对重叠区域图像进行扩展,采用多分辨率融合算法实现了非严格重叠区域的融合;在过渡区域采用加权平均算法来消除拼接线。结果:采用含有大量运动物体的图像序列对算法进行测试,实验结果表明,基于差分图像加权的最佳缝合线有效避开了大部分运动物体,当缝合线难以绕开运动物体时,能够尽量少地穿过运动物体;通过多分辨率和加权平均融合算法消除了拼缝等问题。结论:提出的最佳缝合线算法能够有效地避免缝合线通过运动物体、配准不准确的区域,将多分辨率图像融合算法应用于非严格重叠图像融合,能够合成高质量的全景图像。
关键词
Image stitching by combining optimal seam and multi-resolution fusion

guyu,zhouyang,rengang,fengqiuchen,luguozhi()

Abstract
Objective: Image stitching technique can synthesize a panoramic image from multiple successive images, and can be applied in many military and civil applications. Ghost problem exists in the overlapping areas between two images being stitched when moving objects exist, or registration error occurs during image stitching. The issues of stitching line, and color inconsistency, etc, occur when camera exposure time and illumination changes during imaging. These factors may affect the panoramic image if the images are simply fused, so the improved image fusion technique, which is one of key technologies in image stitching, can be used to solve the above problems. Since Optimal seam algorithm is one of effective methods, an improved optimal seam algorithm based on differential image weighting is proposed to solve ghost problem where seams pass through moving object or inaccurate registration areas in classical optimal seam algorithm. Partition fusion algorithm based on multi-resolution fusion and weighted average fusion is presented to solve the stitching line problem caused by the change of exposure time and illumination. Method: Firstly, the images being stitched are mapped to cylindrical surface after registration, where Harris corner is used to find correspondences between images. Secondly, the overlapping areas between the image being stitched and the fused image are calculated, and then partitioned into three areas, including an optimal seam search area and two transition regions. The optimal seam search area is set to occupies three fifth, and both transition regions occupy one fifth separately in the paper. Thirdly, Differential image weighted optimal seam algorithm is proposed to search optimal seam in seam search regions. Not only considering the difference of color and structure, the metric of computing optimal seam for each pixel is also weighted by image difference. The weighting coefficient is set to infinity if this difference is above a threshold, so the moving object region if image difference is big can be bypassed when searching seam line. After the metric image is computed, dynamic programming algorithm is used to search the optimal seam in this metric image. Finally, the mask image is generated based on the obtained optimal seam. Because there are a lot of invalid areas due to the mapping to cylindrical surface, the method of extending image is adopted to fill these areas using the nearby pixels before combining two images. Multi-resolution image fusion algorithm is performed in the whole overlapping region after image extension, and then weighted average fusion algorithm is adopted to eliminate stitching line in transition regions. Result: In this paper, several image sequences captured by mobile phone at the crossing are used to test the proposed algorithm, and the proposed algorithm is compared with the algorithm of dynamic image mosaic via SIFT and dynamic programming and the stitching algorithm implemented in OpenCV. There are a large number of moving objects(e.g. cars, pedestrians, etc.) in the images, and experimental results demonstrate that, the probability of optimal seam passing through moving objects and inaccurate registration areas is obviously reduced due to the improved optimal seam search algorithm, and it is as low as possible to pass through moving objects when the optimal seam is difficult to circumvent them. The stitching line is effectively eliminated by combining the algorithms of multi-resolution fusion and weighted average fusion, and the quality of the panoramic image using our algorithm is better than that using multi-resolution fusion algorithm only, especially in the scenario the difference of illumination exists. The stitching result is the same or better than using OpenCV, although there are some distortions in panoramic image due to misregistration. The comparison of computational cost between the proposed algorithm and the dynamic image mosaic algorithm is also presented. Conclusion: The proposed optimal seam algorithm can avoid the problem, where seam line passes through moving object region or registration error exists. The multi-resolution image fusion algorithm is applied to non-strict overlapping image fusion through image extension after region partition, and high-quality panoramic image is synthesized. However, our algorithm has the following shortcomings: the code must be optimized to meet real-time requirements, and global registration optimization can be used to reduce distortion problem in the panoramic image during the process of image stitching.
Keywords
QQ在线


关注微信