何雨亭,唐向宏,张越,杨瑞(杭州电子科技大学通信工程学院, 杭州 310018;杭州电子科技大学通信工程学院, 杭州 310018;杭州电子科技大学信息工程学院, 杭州 310018)
目的 针对传统基于样本块的图像修复算法中仅利用图像的梯度信息和颜色信息来修复破损区域时，容易产生错误填充块的问题，本文在Criminisi算法的基础上，利用结构张量特性，提出了一种改进的基于结构张量的彩色图像修复算法。方法 首先利用结构张量的特征值定义新的数据项，以确保图像的结构信息能够更加准确地传播；然后利用该数据项构成新的优先权函数，使得图像的填充顺序更加精准；最后利用结构张量的平均相干性来自适应选择样本块大小，以克服结构不连续和错误延伸的缺点；同时在匹配准则中，利用结构张量特征值来增加约束条件，以减少错误匹配率。结果 实验结果表明，改进算法的修复效果较理想，在主观视觉上有明显的提升，其修复结果的峰值信噪比（PSNR）和结构相似度（SSIM）都有所提高；与传统Criminisi算法相比，其峰值信噪比提高了1~3 dB。结论 本文算法利用结构张量的特性实现了对不同结构特征的彩色破损图像的修复，对复杂的线性结构和纹理区域都有较理想的修复，有效地保持了图像边缘结构的平滑性，而且对大物体的移除和文字去除也有较好的修复效果。
Improved Criminisi algorithm based on structure tensor
He Yuting,Tang Xianghong,Zhang Yue,Yang Rui(School of Communication Engineering, Hangzhou Dianzi University, Hangzhou 310018, China;School of Communication Engineering, Hangzhou Dianzi University, Hangzhou 310018, China;School of Information Engineering, Hangzhou Dianzi University, Hangzhou 310018, China)
Objective The rapid development of multimedia information technology has led to images that have become the main carrier of information in people's lives.People communicate information through various means,such as voice,images,text,and video.Consequently,digital image inpainting technology has gradually attracted increasing attention,and its application fields are extensive.Digital image inpainting refers to the process of repairing or rebuilding missing information in damaged images by using a specific image inpainting algorithm,such that the observer cannot easily detect that the image has been repaired or damaged.Image inpainting technology has been used in many research areas,such as the restoration of old photos,the removal of image text,and the preservation of cultural relics.The traditional exemplar-based image inpainting algorithm only uses gradient information and the color information of the image to repair damaged areas,which can easily generate incorrect filling patches.In addition,the definition of the priority function is unreasonable,and thus causes the wrong filling order during inpainting,which affects the overall restoration effect.To solve these problems,an improved color image inpainting algorithm that is based on structure tensor is presented in this paper.Method Structure tensor is often used to analyze the local geometry of an image,which not only contains the intensity information of the local region,but also the main directions of the neighborhood gradient of a pixel and the degree of coherence of these directions.Its two eigenvalues can distinguish the edge,texture,and flat areas of an image.First,the proposed algorithm uses the structure tensor to define the data items to ensure that the structure information of the image can be transmitted accurately,and then uses the data items to form a new priority function for a more precise filling order.Second,different sizes of sample patches can be used to search for the best matching patch because an image has different structural features in different regions.Therefore,the size of the sample patch is adaptively selected according to the average coherence of the structure tensor.In other words,when the average coherence of the patch to be repaired is large,this patch is at the edge of the image and a small sample patch should be used;when the average coherence is small,it is in the flat region of the image and a large sample patch should be used.In this manner,when repairing complex damaged images,the continuity of the edge structure can be maintained;the flat area of the image can be effectively repaired.Finally,the traditional inpainting algorithm only uses the color information of the image to find the best matching patch,which renders the matching patch sub optimal.In this study,the eigenvalues of the structure tensor are added to the matching criteria to reduce the false matching rate.Result Experimental results show that the improved algorithm is more effective in subjective vision than the other related algorithms.Moreover,the improved algorithm can achieve good results for different types of damaged images and effectively maintain the smoothness of the edge structure of the image.Compared with the traditional Criminisi algorithm,the power signal-to-noise ratio of the result has improved by approximately 1~3 dB and enhanced structure similarity.In addition,the proposed algorithm has a higher running time than other algorithms because in the inpainting process,the proposed algorithm uses the adaptive sample patch size to search for the best matching patch,and when analyzing the local structural features of the image,it needs to calculate the coherence factor of the pixels.These steps consequently increase the running time and reduce the efficiency of image inpainting.Conclusion When the traditional algorithm repairs the strong damaged area of the edge,the structural integrity and the good visual effect of the image are difficult to balance.In this study,we use the structure tensor of the color image to analyze the structure and texture area of the image.This paper discusses a color image inpainting algorithm based on the structure tensor.The proposed algorithm first uses the eigenvalue of the structure tensor instead of the isophote line in the traditional algorithm to improve the data items,which can spread the structure information of the image more accurately.Then,the average coherence factor of the structure tensor is used to analyze the texture and structural features of the image to repair the different image structural features.Finally,the matching rate used to select the best matching patch is improved with the addition of the constraint of the structural tensor to the traditional matching criteria.The proposed algorithm can obtain a better visual effect for damaged images with different structural features.The proposed algorithm can also effectively maintain the structural integrity of the image,and the complex texture area does not exhibit a wrong filled patch.Moreover,the large object and text removals by the proposed algorithm also have a good restoration effect.Compared with related Criminisi algorithms,the proposed algorithm has a better repair effect on complex linear structure and texture regions and effectively improves the overall quality of image restoration.