Current Issue Cover


摘 要
目的 针对传统基于样本块的图像修复算法中仅利用图像的梯度信息和颜色信息来修复破损区域时,容易产生错误填充块的问题,本文在Criminisi算法的基础上,利用结构张量特性,提出了一种改进的基于结构张量的彩色图像修复算法。方法 算法首先利用结构张量的特征值定义新的数据项,以确保图像的结构信息能够更加准确的传播;然后利用该数据项构成新的优先权函数,使得图像的填充顺序更加精准;最后利用结构张量的平均相干性来自适应选择样本块大小,以克服结构不连续和错误延伸的缺点;同时在匹配准则中,利用结构张量特征值来增加约束条件,以减少错误匹配率。结果 实验结果表明,改进算法的修复效果较理想,在主观视觉上有明显的提升,其修复结果的峰值信噪比(PSNR)和结构相似度(SSIM)都有所提高;与传统Criminisi算法相比,其峰值信噪比(PSNR)提高了1~3dB。结论 本文算法利用结构张量的特性实现了对不同结构特征的彩色破损图像的修复,对复杂的线性结构和纹理区域都有较理想的修复,有效地保持了图像边缘结构的平滑性,而且对大物体的移除和文字去除也有较好的修复效果。
An improved Criminisi algorithm based on structure tensor

He Yuting,Tang Xianghong(School of Communication Engineering,Hangzhou Dianzi University)

Objective With the rapid development of multimedia information technology, images have become the main carrier of information transmission in people’s lives. People communicate information mainly through various means, such as voice, images, text and video, and so on. As a result, digital image inpainting technology has gradually attracted more and more people’s attention, and its application fields are very extensive. Digital image inpainting refers to the process of repairing or rebuilding the missing information of the damaged image by using a specific image inpainting algorithm, so that the observer can’t easily detect that the image has been repaired or damaged. At present, image inpainting technology has been used in many research areas, such as the restoration of old photos, the removal of image text, the preservation of cultural relics and so on. Aiming at the problem that the traditional exemplar-based image inpainting algorithm only uses the gradient information and the color information of the image to repair the damaged area, it is easy to generate the incorrect filling patch. In addition, due to the definition of the priority function is not reasonable, it causes the wrong filling order in the process of inpainting, and then affects the overall restoration effect. To solve the above problems, an improved color image inpainting algorithm based on structure tensor was proposed in this paper. Method The structural tensor is often used to analyze the local geometry of an image, which not only contains the intensity information of the local region, but also contains the main directions of the neighborhood gradient of the particular pixel and the degree of coherence of these directions. Its two eigenvalues can distinguish the edge area, the texture area and the flat area of the image. Firstly, the proposed algorithm uses the structure tensor to define the data items to ensure that the structure information of the image can be transmitted more accurately, then using the data items to form a new priority function to make the filling order more precise. Secondly, because the image has different structural features in different regions, different sizes of sample patch can be used to search for the best matching patch. So in this paper, the size of the sample patch is adaptively selected according to the average coherence of the structure tensor. In other words, when the average coherence of the patch to be repaired is large, it means that this patch is in the edge region of the image and a smaller sample patch should be used; when the average coherence is small, it is in the flat region of the image and a larger sample patch should be used. In this way, when repairing complex damaged images, the continuity of the edge structure can be maintained; when repairing the flat area of the image can be well repaired. Finally, in the traditional inpainting algorithm, it only use the color information of the image to find the best matching patch, which makes the matching patch not optimal. So in this paper, the eigenvalues of the structure tensor are added to the matching criteria to reduce the false matching rate. Result The experimental results show that the improved algorithm is more effective in subjective vision than the other related algorithms. It can achieve good results for different types of damaged images and effectively maintain the smoothness of the edge structure of the image. Compared with the traditional Criminisi algorithm, the power signal-to-noise ratio (PSNR) of the final result has improved by approximately 1dB~ 3dB, and the structure similarity (SSIM) has improved. In addition, from the point of view of running time, the proposed algorithm is higher than other algorithms, because in the inpainting process, proposed algorithm uses the adaptive sample patch size to search for the best matching patch, and when analyzing the local structural features of the image, it needs to calculate the coherence factor of the pixels. Therefore, these steps will increase the running time and reduce the efficiency of image inpainting. Conclusion When the traditional algorithm repairs the strong damaged area of the edge, it is difficult to balance the structural integrity and the good visual effect of the image. In this paper, we use the structure tensor of color image to analyze the structure and texture area of the image, and discuss a color image inpainting algorithm based on structure tensor. The proposed algorithm firstly uses the eigenvalue of structure tensor instead of the isophote line in the traditional algorithm to improve the data items, which can spread the structure information of the image more accurately. Then we use the average coherence factor of structure tensor to analyze the texture features and structural features of the image, in order to achieve the repair of different image structural features. Finally, by adding the constraint of the structural tensor to the traditional matching criteria, the matching rate for finding the best matching patch has improved. The proposed algorithm can obtain better visual effect for damaged images with different structural features, it can maintain the structural integrity of the image better, and in the complex texture area does not appear the wrong filled patch. Moreover, the large object removal and text removal by proposed algorithm also have better restoration effect. Compared with the related Criminisi algorithms, the proposed algorithm has better repair effect on complex linear structures and texture regions, and effectively improves the overall quality of image restoration.