王志旋,钟若飞,谢东海(首都师范大学北京成像技术高精尖创新中心, 北京 100048;首都师范大学资源环境与旅游学院, 北京 100048;首都师范大学三维数据获取与应用重点实验室, 北京 100048)
目的 针对城市实施的路灯杆编码项目，即获取路灯坐标并依次编号，传统的测量方法耗费大量人力物力，且作业周期长，虽然激光测量精度高，但成本也高。为此本文提出一种将基于深度学习的目标检测与全景量测结合自动获取路灯坐标的方法。方法 通过Faster R-CNN（faster region convolutional neural network）训练检测模型，对全景图像中的路灯底座进行检测，同时输出检测框坐标，并与HOG（histogram of oriented gradient）特征结合SVM（support vector machine）的检测结果进行效果对比。再将检测框的对角线交点作为路灯脚点，采用核线匹配的方式找到两幅图像中一个或多个路灯相对应的同名像点并进行前方交会得到路灯的实际空间坐标，为路灯编码做好前期工作。结果 采用上述两种方法分别对100幅全景影像的路灯进行检测，结果显示Faster R-CNN具有明显优势，采用Faster R-CNN与全景量测结合自动获取路灯坐标。由于路灯底部到两成像中心的距离大小及3点构成的交会角大小对坐标量测精度具有较大影响，本文分别对距离约为7 m、11 m、18 m的点在交会角为0°180°的范围进行量测结果对比。经验证，交会角在30°150°时，距离越近，对量测精度的影响越小。基于上述规律，在自动量测的120个路灯坐标中筛选出交会角大于30°且小于150°，距离小于20 m的102个点进行精度验证，其空间坐标量测结果的误差最大不超过0.6 m，中误差小于0.3 m，满足路灯编码项目中路灯坐标精度在1 m以内的要求。结论 本文提出了一种自动获取路灯坐标的方法，将基于深度学习的目标检测应用于全景量测中，避免手动选取全景图像的同名像点进行双像量测，节省大量人力物力，具有一定的实践意义。本文方法适用于城市车流量较少的路段或时段，以免车辆遮挡造成过多干扰，对于路灯遮挡严重的街道全景，本文方法存在一定的局限性。
Automatically measuring the coordinates of streetlights in vehicle-borne spherical images
Wang Zhixuan,Zhong Ruofei,Xie Donghai(Beijing Advanced Innovation Center for Imaging Technology, Capital Normal University, Beijing 100048, China;College of Resources Environment and Tourism, Capital Normal University, Beijing 100048, China;Key Lab of 3D Information Acquisition and Application, Capital Normal University, Beijing 100048, China)
Objective With the development of urban management, a growing number of cities are implementing coding projects for streetlight poles. In such projects, the coordinates of streetlamps are obtained and serial numbers are assigned to them. The coordinates can be obtained in many ways, such as RTK and laser measurements. A quick and easy approach to obtain the data is required because tens of thousands of streetlamps are present in a city. In consideration of the cost, mobile panorama measurement is preferred. However, most current panorama measurements are conducted by means of human-computer interaction, in which homologous image points are selected to perform forward intersection to obtain the coordinates. This approach consumes substantial energy and time. Therefore, in this paper, we propose an automatic method to obtain the coordinates of streetlamps by combining object detection with panoramic measurement. Method The method combines deep learning and panoramic measurement to automatically obtain the coordinates of streetlight poles. No feature points are obvious on the poles because of their rod-shaped features, and the top of the streetlamp is different because of the different design. The distortion of panoramic images strongly influences the detection of the top of a streetlamp. Thus, the bottom of the poles is used as the detection target in this paper. The pole bottoms are detected by faster R-CNN. Meanwhile, the coordinate file that contains the upper-left and lower-right corners of the detection frames are output and compared with the detection results obtained by the combination of histogram of oriented gradient (HOG) and support vector machine (SVM). Then, the diagonal intersection of the detection box is regarded as the foot of the streetlight pole, and an epipolar line is used to find homologous image points in two panoramic images because multiple streetlight poles can be present in a panoramic image. Based on the matching results, the space coordinates of the streetlight poles are obtained by forward intersection of the panoramas, thereby confirming the potential of this preliminary work for the coding projects. Result The aforementioned two methods were used to detect the streetlights of 100 panoramic images, which include 162 streetlights. A total of 1826 detection results were obtained based on HOG features, of which the correct bottom of streetlamps is 142. A total of 149 detection results are based on the faster R-CNN, of which 137 are correct. We can conclude that the faster R-CNN has obvious advantages. Thus, in this study, we use the faster R-CNN combined with panoramic measurement to automatically obtain the streetlight coordinates. The distance from the bottom of the streetlamp to the two imaging centers and the intersection angle formed by the three points significantly affect the accuracy of coordinate measurement. To filter out the coordinates that are less affected by the aforementioned two factors, we compare measurement results, which are the distances of approximately 7, 11, and 18m; the intersection angles are 0° to 180°. We have verified that when the intersection angles are from 30° to 150°, the influence on the measurement accuracy is smaller because the distance is closer. Based on the preceding rules, 120 coordinates of streetlamps are selected to determine the statistical distribution of the intersection angle and distance. Points with a distance of less than 20 m and intersection angle greater than 30° and less than 150° are selected for the coordinate error analysis, and 102 points meet the requirements for accuracy verification. The deviation of space coordinate measurement is less than 0.3 m and the maximum is not more than 0.6 m, thereby satisfying the requirement that the accuracy of the coordinates is within 1 m. Conclusion This paper presents a method of automatically obtaining the coordinates of streetlamps. The method of target detection based on deep learning is applied to the panorama measurement. The method avoids the manual selection of the homologous image points for measurement, which saves considerable labor and material resources. We conclude that this method exhibits practical significance because it is suitable for road sections or periods with low traffic volume in the city, thereby preventing excessive obstruction caused by vehicles. However, for panoramas with seriously obstructed streetlights, this method has certain limitations.