Current Issue Cover


摘 要
目的 基于网格变形的图像配准方式,针对待拼接图片重叠区域的视差具有一定的容忍性,并且能够适应更复杂的图像拼接场景。在NISwGSP算法基础上提出了一种具有直线结构保护的图像拼接算法(MISwLP),该算法通过提取图片中的直线结构并施加约束,可以得到视觉效果自然、畸变较小的图像拼接结果。方法 首先对图片进行网格划分,建立网格优化模型,针对网格顶点坐标集定义能量函数,在保证图片重叠区域高度对齐的同时,对网格进行相似性连续约束,并辅以直线结构约束,最后使用共轭梯度最小二乘法求解得到最优网格顶点集,指导网格变形。结果 针对不同场景下的图片进行拼接实验,同时和几种比较流行的图像拼接软件和算法进行比较。结果表明,同经典拼接算法,比如Autostitch相比,基于网格优化的图像拼接算法能够适应更加复杂的多平面场景,在减小投影失真和对齐误差方面表现更好;同现在比较好的几种网格拼接算法,比如SPHP、APAP、NISwGSP等的比较,MISwLP算法不仅能够很好地对齐图像和避免投影失真,并且能够保持图像重叠区域到非重叠区域的一致性,即保护原图中的直线结构。结论 提出了一种基于网格优化的直线约束方法,对于具有显著几何结构的图像拼接场景,能够较好的保护拼接后图像中原有的直线结构,具有较好的应用价值。
Mesh-based Image Stitching Algorithm with Linear Structure Protection


Objective The image registration method based on mesh deformation has the ability to handle some parallax in the overlapping area of input images and can adapt to more complex scenario which the scenery is not in the same plane. A new Mesh-based Image Stitching with Linear structure Protection (MISwLP) is proposed, by applying constraints to the lines extracting from the images to protect them from being distorted by the mesh deformation process, thus obtaining a more natural panoramas with less distortion. Method MISwLP is based on mesh deformation. Images are meshed with a set of vertices, and the image deformation is guided by the indexed vertices. The algorithm can be implemented with four steps. The first step is called APAP pre-registration. APAP algorithm is applied to align the images. And the feature matching pairs obtained by the APAP algorithm can be used to get all the vertices matching pairs in the overlapped area of the image matching pairs, which is called matching points. The matching points are distributed more evenly and can be used to get a better alignment ability for the mesh optimization model. The second step is called global similarity estimation. The relative 2D rotation angle and the relative scale between two images are estimated in this step. Then a similarity transform between two images can be constructed. In the third step, a mesh optimization model is established for the input vertices of images. The mesh optimization process is implemented in two stages. In the first stage, the energy function includes three terms, namely, alignment term, local similarity term, global similarity term and the original vertices are taken as input for this function. It is solved by least-squared conjugate gradient method. The main object of the first stage is to align the images. Then the outputs of the first stage are used as the input vertices of the second stage. In the second stage, a new term called line protection term is added for further optimization. The lines are extracted by LSD algorithm with a threshold or user guided interface and sampled across the grid. The line protection term constraints the sample points on a straight line. The optimization solution is computed with sparse matrix effectively. At this time, the distorted lines in the first stage of this step were straightened. In the fourth step, a texture mapping method is applied by affine transforming the input grids into the output grids. All the images are blended with a linear blending method. Result The performance of MISwLP is verified under some images taken from different scenery by handheld devices like mobile phones and digital cameras, and also some open datasets. Those scenes include urban sceneries and nature sceneries. Compared to those image-stitching algorithms using merely one global homograhpy, such as AutoStitch, MISwLP can handle more complicated image stitching tasks which the scenery consists of two planes above. And MISwLP gives a more natural stitching result with less projective distortion. In additions, MISwLP outperforms some state of art methods, such as SPHP, APAP and NISwGSP. Those algorithms use similarity transform to protect the non-overlapping area from projective distortion. Consequently, inconsistency is introduced between overlapped area and non-overlapped area. And it can be perceptive by human eyes that some geometry structures of the transitional area is destroyed. MISwLP handles this problem with a lines protection term and gives a better result with little geometry distortion. The proposed method works especially well for urban sceneries that contain many linear structures. For those sceneries which contain no obvious geometry, a user guided auxiliary method is provided for selecting lines to protect. MISwLP is base on NISwGSP algorithm, but the experiments show that the time complexity is all most the same. Conclusion The performance of the proposed method (MISwLP) is superior to the state of art image stitching methods. MISwLP protect the linear structure in the image stitching process, giving a more pleasure stitching result with no geometry distortion and projective distortion, which has a good application value.