Abstract
Mobile LiDAR is an emerging advanced technology for capturing three-dimensional road information at a large scale effectively and precisely. Pole-like road facilities are crucial street infrastructures as they provide valuable information for road mapping and road inventory. Thus, the automated localization and classification of road facilities are necessary. This paper proposes a voxel-based method to detect and classify pole-like objects in an expressway environment based on the spatially independent and vertical height continuity analysis. First, the ground points are eliminated, and the nonground points are merged into clusters. Second, the pole-like objects are extracted using horizontal cross section analysis and minimum vertical height criteria. Finally, a set of knowledge-based rules, which comprise height features and geometric shape, is constructed to classify the detected road poles into different types of road facilities. Two test sites of point clouds in an expressway environment, which are located in Bangkok, Thailand, are used to assess the proposed method. The proposed method extracts the pole-like road facilities from two datasets with a detection rate of 95.1% and 93.5% and an overall quality of 89.7% and 98.0% in the classification stage, respectively. This shows that the algorithm could be a promising alternative for the localization and classification of pole-like road facilities with acceptable accuracy.
1. Introduction
Expressway facilities, such as lighting poles, traffic signs, speed limit posts, overhead signs, emergency telephone posts, or telecommunication stations, are crucial components of the transportation infrastructure and play a vital role in expressway asset management. Almost all the facilities mentioned above have, entirely or partially, the shape of a pole [1]. For instance, expressway lights (all pole-like objects (PLOs)) provide lighting for vehicles and pedestrians at nighttime or in dark weather. Similarly, a traffic sign (partially shaped PLOs) is an essential facility on a road to guide users, to drive safely, and provides all the necessary information for their travel. A shortage of lighting poles, speed limit posts, or the inaccuracy of a traffic sign can lead to severe traffic casualties [2]. Therefore, collecting and updating all expressway facility information are essential for an asset management agency. Traditionally, a visual on-site survey was manually done by inspectors, to collect all expressway facility information. Nevertheless, due to the large scale and complexity of an expressway, such a visual manual method (e.g., optical images combined with real-time kinematic surveying) had drawbacks. These drawbacks are labor expenses, time consumption, tediousness, insufficient collected facility data, and slowly updating expressway assets.
With the development of laser scanning technologies, there are several emerging advanced methods, such as aerial laser scanning (ALS), terrestrial laser scanning (TLS), and mobile laser scanning (MLS), that can be used for large-scale city mapping in a three-dimensional (3D) format. By precise capturing and producing highly dense 3D point clouds (8,000 points/m2) of road facilities along the expressway at road speeds, MLS is more advantageous, compared with ALS or TLS. MLS has been broadly used in the recent decade to collect data for infrastructure management in an urban environment [3]. There are wide applications of MLS data in urban road engineering, which include urban power line extraction [4], tree extraction [5], traffic sign extraction [6], and 3D geometric road reconstruction [7]. Using point clouds collected from MLS, all road facilities can be obtained. Each facility object can be extracted individually for maintenance and asset management. However, manual road facility extraction is time consuming due to the bulk of point clouds (e.g., 1.7 billion points per km of road) [8]. Thus, it is essential to propose a method that can detect and classify PLOs in an automated and effective way by inheriting the advantages of MLS.
To date, several efforts have been devoted in developing an automatic algorithm to detect PLOs from point clouds. To minimize the complicated data and reduce the processing time, the extraction of the road surface is a crucial preparatory task before extracting PLOs in many existing methods. Yu et al. [2] first removed the ground points. The 3D matching framework was then used to extract PLOs. Yan et al. [9] eliminated the ground points, and the PLOs were extracted using a set of decision rules. However, the clustering method using DBSCAN is a crucial step, which affected the accuracy of this method. Yadav et al. [10] also first removed ground points. After that, voxelization, devoxelization, vertical fitting, and compactness factor evaluation were performed to identify the PLOs. A method to extract road light poles was proposed by Zheng et al. [11]. The piecewise elevation approach was built to remove the ground points. Then, the light poles were detected using a Gaussian mixture model-based method. Yan et al. [12] first removed the ground points. The shape and topology information (minimum pole height and cross section size) of the poles were then used to recognize the PLOs. Shi et al. [13] first removed the ground points and outliers. Next, the RANdom SAmple Consensus (RANSAC) cylinder model or the PCA linear feature was used to detect PLOs. Some other researchers analyzed a local shape feature of the point cloud to detect PLOs. Yokoyama et al. [14] used PCA and Laplacian smoothing to classify each point of each segment into planar, spherical, and linear. However, unsatisfactory detection results were obtained due to the low accuracy of segmentation. Cabo et al. [1] developed a voxel-based method for automatic detection of PLOs. This method cannot detect a pole that is close to other objects or partially hidden behind other objects. Rodríguez-Cuenca et al. [15] applied the Reed and Xiaoli anomaly detection algorithm to detect the candidate PLOs. Teo and Chiu [16] performed the coarse-to-fine framework method to extract the PLOs. However, low point density greatly influenced the resulting accuracy of this method. Yang et al. [17] generated a multiscale supervoxel to automatically extract urban objects. Some additional information of the point cloud is needed, such as colors and intensities. Kang et al. [18] proposed a voxel-based method to recognize PLOs. A cylinder model was used to detect linear voxels belonging to PLOs. The detection result was influenced by the accuracy of the initial extraction of linear objects. Li et al. [19] introduced a framework to detect the road furniture. The pole trunks and the attachments were decomposed into different components, based on their spatial relations. Li et al. [20] detected the road poles based on the independent pole features and continuous height in the vertical direction of the pole part. Machine learning was also employed to detect PLOs by other researchers. Ishikawa et al. [21] developed a method to recognize urban objects (including PLOs) on a road using the support vector machine (SVM) method. Due to the complex and tedious computing feature quantities for SVM, the method may not be a robust application. Fukano and Masuda [22] applied a supervised machine learning method to detect the PLOs. Additional information, such as GPS time and scan line, is needed for this method. The combined data scanned from the opposite direction or using a different scanner limited the detection of PLOs. Yu et al. [23] first constructed the visual phrase dictionary by using feature regions created from supervoxel segmentation. Then, the hierarchical deep model and bag-of-visual-phrases were applied to detect the traffic signs.
For PLO classification, Yokoyama et al. [14] used the shape features (pole height, number of attachments, and type of attachments) and context features (relative distance to nearby poles in the vicinity) to classify the PLOs. Nevertheless, the classification result is dependent on the distribution of the nearby poles. A set of semantic rules, including pole height and projected horizontal area of the entire extracted pole, was used by Yan et al. [9] to classify PLOs into five categories. Yu et al. [23] classified traffic signs using a machine learning method. A hierarchical classifier was constructed using a supervised Gaussian–Bernoulli deep Boltzmann machine. The catalog of known traffic signs was used to train the detected traffic signs into different classes. A machine learning method was also used by Yan et al. [12] to classify the PLOs. To classify the detected PLOs into four classes (utility poles, lamp posts, trees, and others), Kang et al. [18] used a set of semantic rules for each type of class (i.e., height and dispersion of attachments in 2D space). Shi et al. [13] classified the PLOs using 3D shape matching. The height and root mean square error of the 3D shape were calculated, and then the template and unclassified PLOs were compared. However, in the case of missing data, the height of the poles was shorter and misclassified as other types.
In general, the existing methods were able to detect and classify the pole-like road facilities in many complex scenes, especially poles in the overlapping regions. However, the roads in an expressway environment had significant height variations, compared with the flat roads in residential areas in some current studies. Thus, the reviewed studies may not be suitable [9, 10, 14]. Considering the drawbacks of past studies, this paper proposes the voxel-based method to detect PLOs based on spatially independent and vertical height continuity analysis. A set of semantic rules was constructed to classify the detected PLOs into different types of expressway facilities.
2. Materials and Methods
The method consists of three main steps (Figure 1): (1) nonground extraction: the ground points were removed from the original point cloud, and the nonground points were extracted and grouped into distinct clusters; (2) pole-like road object detection: the entire road poles were extracted from each cluster; and (3) pole-like road object classification: the detected road poles were classified into several types of expressway facilities, based on their height and geometric shape.
2.1. Test Sites
MLS point clouds of two test sites in an expressway environment were used to assess the efficiency of the proposed method. Both survey sites were congested expressways, located in Bangkok, Thailand (Figure 2(a)). The GT-4 MLS system, including two laser scanners RIEGL VQ-450, one Global Positioning System (GPS), one 360° camera, and an inertial measurement unit (IMU), was used to collect the point clouds (Figure 2(b)). Site 1 covered about 2.1 km of a cable stay on the Rama IX bridge with 147.2 million points, resulting in an average point density of approximately 1,800 pts/m2. Site 2 is a part (5.1 km) of the Bang Na elevated expressway. It included two ramps where a vehicle can exit or enter the road, with a total of seven lanes. A total of 441.5 million points were obtained at the test site, with an average point density of nearly 1,500 pts/m2. To collect all expressway objects, the speed of the MLS car was limited to 50 km/h, and each direction was scanned by two passes (inbound and outbound). An overview of two point clouds of two test sites is shown in Figure 3.
(a)
(b)
(a)
(b)
2.2. Nonground Extraction
MLS was capable of collecting all objects that were located away from the trajectory within an extensive measurement range (e.g., up to 800 m for a RIEGL VQ-450 scanner [24]); thus, the original point clouds captured by MLS were large. Some of these objects, such as building facades, factories, trees, and bushes, were objects (of noninterest) that were located away from the roadside. Moreover, the ground points generally were well represented in point clouds, while our regions of interest were the PLOs, which were in the nonground areas. Therefore, it was first necessary to eliminate the point clouds of regions of noninterest where the poles were not available, to mitigate the computation cost of our algorithm. Three substeps were introduced as follows: (1) preprocessing; (2) ground removal; (3) Euclidean clustering.
2.2.1. Preprocessing
The trajectory data were used to remove regions of noninterest, which were defined as the points far away from the boundary of the trajectory with a predefined distance, d. The boundary line of trajectory data (the side where the MLS car was driving on a road shoulder, near the roadside) was first obtained. Next, all point clouds were projected onto the XY plane. All points with a distance to the nearest boundary trajectory point of less than d were preserved while removing others. Finally, the points that satisfied the above conditions and points located within the boundary trajectory were maintained and considered as regions of interest for further processing. The result of preprocessing is illustrated in Figure 4(a).
(a)
(b)
(c)
(d)
2.2.2. Ground Removal
The nonground objects (including PLOs) are connected through ground points; thus, eliminating the ground points is needed, to group the remaining points into different clusters. The ground is assumed to be a nearly smooth surface in a local region; thus, the elevation difference between points and their neighborhoods is small at a certain distance. First, a new set of 2D points was created by projecting the 3D data points onto the XY plane. Then, a 2D bounding box was generated to cover the new 2D points. The box was defined by two points A(Xmin, Ymin) and B(Xmin + ∆, Ymin + ∆). Here, ∆ = max[|Xmax − Xmin|, |Ymax − Ymin|] and Xmax, Xmin, Ymax, and Ymin were the maximum and minimum of the X and Y coordinate values from the 2D data points. The 2D bounding box covering the point clouds of the new 2D dataset was then divided into a set of square grids with the grid size . The analysis was performed in each grid to extract the ground points.
The points satisfying the first term of equation (1) are referred to as ground points. In addition, it is observed that the PLOs may be connected to other nonground objects along the road by some short objects (e.g., guardrails, shrubs, and parapets). Therefore, some parts of the abovementioned short objects that exist along the expressway were removed to discontinue the connection between the nonground objects and also to minimize the computation time for the following Euclidean clustering. If the height of any grid is less than the minimum height threshold of the poles, then all points in this grid were also removed, as shown in the second term of equation (1):where is the elevation of the ith point in the jth grid and and are the maximum and minimum elevation of the jth grid. The value of is selected as the maximum elevation of all points in the jth grid. is computed as the smallest value of minimum elevation from the jth grid and its eight neighbors. This ensured that in the case of missing ground data below the attachment of PLOs, the minimum elevation of the current grid was evaluated appropriately (e.g., the lighting pole was erected behind and far away from the guardrail at a certain distance, and the ground data between them may be missing during the MLS data collection; thus, the height of the grid containing the bulb of the lighting pole and the missing ground data may be computed incorrectly if the minimum elevation of only this grid was used). and denote the height difference threshold and the minimum height of objects to be considered as PLOs. Figure 4(b) illustrates the results of the ground and short object part removal.
2.2.3. Euclidean Clustering
After the ground and points of noninterest were eliminated, the remaining points that consisted of PLOs were discrete and unorganized. Thus, the grouping of all points that belong to the same objects is needed for further PLO detection. Euclidean clustering was employed to group points into a set of distinct clusters. The points were organized using KD tree construction. If the distance between two neighboring points was less than the threshold distance (dE), they were merged into the same cluster. Otherwise, they belonged to different clusters. Figure 4(c) depicts an example of Euclidean clustering.
The original point clouds were separated into different clusters which could be used for PLO detection. However, some clusters contained small objects with short height (e.g., bushes, shrubs, vegetation, fragments, and outliers) or objects with immense height (e.g., pylons or cables in a cable-stayed bridge) that were not objects of interest. It was observed that the PLOs were erected on the ground or the parapet along the road. Thus, all clusters that were short, had large height, and were located above the ground at a certain distance threshold were removed, to reduce the computational complexity of PLO detection in the following step:where and represent the maximum and minimum elevation of the ith cluster; is the elevation of the ground near the ith cluster, and this value is estimated from the elevation of the nearest trajectory point to the ith cluster; and denote the maximum height of the objects to be considered as PLOs and the maximum vertical distance between the lowest point in the cluster and the ground, respectively.
The minimum number of points in each cluster can also be used to delete some outliers or small fragments (e.g., 100 points were empirically used in this study). Finally, the potential clusters that consist of single PLOs, trees, cars, vegetation, or PLOs mixed with other objects are (likely) objects of interest that were preserved (Figure 4(d)).
2.3. Pole-Like Road Object Detection
In this step, each potential cluster from the preceding step was checked whether it contained a PLO section or not. The whole PLO was then extracted from each cluster. Due to the specific traffic functionality of each type of PLO in the expressway, the spacing between two adjacent PLOs was sufficiently large at a certain distance. An entire or partial shape of a pole was a typical shape feature of all PLOs. Based on these features of PLOs, three assumptions about PLOs are given as follows: (i) the diameter of poles varied within a range; (ii) the pole was isolated from the other PLOs in the vicinity; (iii) the vertical height of a pole should meet the minimum height value. As shown in Figure 5, an approach for PLO detection is presented as follows: (1) voxelization; (2) selecting voxels that contain pole section points; (3) bottom-up region growing; (4) upward growing for attachment extraction.
2.3.1. Voxelization
To examine the existence of pole sections, the point cloud in each cluster was partitioned into a set of horizontal slices. Voxelization was applied, to subdivide the points of each cluster into a set of cubes (also called voxels) (similar to Vo et al. [25]), which were then structured into a group of slices (Figure 6(a)). In this study, the voxel size was selected as the termination condition .
(a)
(b)
2.3.2. Selecting Voxels That Contain Pole Section Points
(1) Selecting Voxels Based on Horizontal Cross Section Analysis. A circular shape with a certain diameter of the pole cross section is a specific characteristic of PLOs, which can be used to distinguish the other road objects. Furthermore, a cross section of poles typically occupies a smaller number of voxels in each horizontal slice compared with others (e.g., bushes, shrubs, parapets, guardrails, tree crowns, and cars). Based on these features, the voxels containing the points of a pole section in each horizontal slice could be identified. In each slice, the procedure to detect the voxels containing a pole section is performed individually. First, the connected-component labeling algorithm was implemented to cluster all adjacent voxels to be voxel groups. To determine the voxel groups that contain points of a pole section, the following criteria for these voxel groups were established:(i)Considering the maximum diameter of a pole cross section in test sites, the voxel groups that occupy the pole section points should have an area that is not greater than the area of the pole section with the maximum diameter. The number of voxels in each voxel group should be less than or equal to . It is observed that the value of the number is inversely proportional to the voxel size. For the case that the voxel size is quite large, the number may be small (e.g., 1 or 2). However, there may exist a voxel group that consists of four neighboring voxels with a large size, covering all pole section points. Consequently, the voxel group is considered as a candidate group of voxels containing pole section points if the number of voxels in this voxel group is smaller than or equal to N, as expressed in the following equation: where Rmax is the maximum radius of poles in a real scenario and is the voxel size. Figure 6(b) shows an illustration of voxel groups that meet the conditions of equation (3).(ii)After a constraint of the maximum number of voxels in each voxel group was applied, a set of candidate voxel groups that may contain pole section points was generated. However, there may be voxel groups that contain nonpole section points, where the number of voxels is less than a threshold, but the shape is a noncircular section (e.g., a scattered shape or the shape of part of a large object). Differentiating the voxel groups containing pole section points and nonpole section points can be implemented by using the circular section of a pole, which can be described by the circle radius. The points in each candidate voxel group were projected onto a horizontal plane, and the RANSAC circle fitting model [26] was employed to fit the circle through the points. The estimated radius of the circle fitting was used to identify whether the points of the pole trunk were available in this voxel group or not. If the estimated radius varied between a range of predefined PLO radii , this voxel group is a part of a pole section. The lower constraint value was set to delete some PLOs with a small diameter (e.g., tree branches, guardrails, wires, and small steel bars).
(2) Selecting Voxels Based on Vertical Height. After examining voxel groups in all horizontal slices, a set of voxels consisting of pole points was obtained using the above two criteria. Theoretically, only voxels containing pole points were detected. Nonetheless, due to the variation of point density, RANSAC circle fitting was capable of computing an estimated radius of point sets that belong to non-PLOs (e.g., parts of tree crowns, tree leaves, or vehicles) within the radius constraints (Figure 7(a)). In fact, a set of voxels constituting a pole trunk should meet the minimum vertical height to be considered as a partial part of a pole trunk or a whole pole trunk. Thus, this procedure can be utilized to discard some discrete and discontinuous voxels that meet the horizontal cross section analysis.
(a)
(b)
Only voxels that comply with the limited number of voxels and radius constraints (voxels which satisfied horizontal cross section analysis) were considered in the analysis. First, all voxels that possessed the same face, edge, or vertex were joined together to generate a group of voxel clusters. The vertical height threshold (h) was established to ensure that only voxel clusters with their vertical height above h were selected as a part of a pole trunk. Figure 7(b) illustrates the voxel clusters that belong to a pole trunk. As a result, only voxels containing pole trunk points are identified, which will be utilized for further processing.
2.3.3. Bottom-Up Region Growing
A set of voxel clusters containing a pole trunk was obtained from the preceding step. As shown in Figure 8(a), some voxel clusters occupied different parts of the same pole trunk. Based on the assumption that the pole was isolated from the other PLOs in the vicinity, a bottom-up region growing within a 2D horizontal distance constraint was employed to merge these voxels. It should be noted that only voxels containing pole trunk points from Section 2.3.2 were considered for a bottom-up region growing process. The process started from the voxel with the lowest elevation and grew upward to its neighboring voxels. However, only neighbors whose 2D horizontal distance is less than the threshold (d2D) were selected for clustering. This threshold is configured based on the minimum distance between two neighboring pole trunks in the test sites. Similarly, the growing iteratively continued to all neighbor voxels until no voxels were detected, and all voxels belonging to the identical pole trunk were grouped into the current cluster. Then, the same procedure was applied to the remaining voxels, and a new cluster, containing voxels of an individual pole trunk, was created.
(a)
(b)
After the bottom-up region growing, the pole trunk of each PLO was extracted individually. However, sometimes an incomplete and discontinuous pole trunk was obtained due to the connection of pole trunk points and nonpole points (e.g., the poles grew through the tree canopy or attachments were connected to the poles), as shown in Figure 8(b). Based on the assumption that the cross section of the pole trunk varied gradually along its axis, the cylindrical growing model was implemented to interpolate or estimate the missing pole trunk points. The process grew upward from the slice containing pole trunk points with minimum Z elevation (Sliceinitial) and downward from the slice containing pole trunk points with maximum Z elevation. Due to the similarity of upward and downward growing, only the downward process is presented in Algorithm 1. All points in Sliceinitial (or Slicek) and Sliceinitial−1 (or Slicem) were projected onto the XY plane, and RANSAC circle fitting was adopted, to determine the circle center and the radius of each dataset (center: Ck and Cm and radius: Rk and Rm). The points in the next slice (down-slice for downward growing) where the distance to the pole trunk axis, which defined by the vector goes through the circle center of the two upper slices , was less than the maximum radius were considered as the points belonging to pole trunk (see Figure 9(a)). The maximum radius was selected based on the larger radius of the pole trunk from two upper slices (R = max{Rk; Rm}). However, considering the increase of an actual pole’s diameter along the pole trunk height, a tolerance (∆) was added to the maximum radius, to cover all pole trunk points. A tolerance distance (∆) of 0.01 m was used in this study, which was chosen based on several preliminary experiments. If the number of points n was larger than the minimum number of points (N) for RANSAC circle fitting, the center C and the radius R of the pole trunk points in the current slice were calculated and updated for the next growing. Similarly, the growing iteratively continued to all next slices until no pole trunk points were detected. As a result, all complete pole trunks of PLOs were detected (Figure 9(b)).
|
(a)
(b)
2.3.4. Attachment Extraction
The purpose of this research is to extract the whole PLO, including the pole trunk and its attachments. The vertical region growing algorithm is adopted to allocate the attached components to their corresponding pole trunk. The procedure is similar to Euclidean clustering. However, the differences are as follows: (1) the process starts from the initial point with maximum Z elevation belonging to the pole trunk; (2) only neighboring points where the Z elevation is larger than the initial point are considered for grouping to the attachment. The spherical Euclidean distance of dE was used to merge all neighboring points.
2.4. Pole-Like Road Object Classification
In the classification step, nine types of PLOs were labeled, based on their different traffic functionality and geometric shapes. Figure 10 depicts the nine types of road facilities: high-mast lighting, lighting pole-2-sided, lighting pole-1-sided, CCTV camera, telecommunication tower, speed limit pole, lighting pole-1-sided (special type), overhead sign, and some types of small signboards (e.g., S-curve, intersection, metric distance, and warning sign). The height and bounding box of the projected area of PLOs onto the XY plane representing the PLOs’ geometric shapes were used as the two criteria for classification.
Height is one of the specific features of PLOs [18]. Due to the explicit function of each type of PLO, the heights for different kinds of PLOs are dissimilar. Based on examining the height of some known classifiers, a set of height variations for each PLO category was generated within this study (Table 1). From the extracted PLOs, only objects where the height varied between the ranges of the given height criteria were labeled as the same classifier. For instance, the high-mast lighting had the highest height, compared with other PLOs (>13 m); thus, all detected PLOs with a height larger than 13 m were classified as high-mast lighting.
However, there was an insignificant height difference for some types of PLOs (e.g., lighting pole-1-sided vs. CCTV camera/overhead sign or telecommunication tower vs. speed limit pole). The shape of a PLO can be roughly characterized by a bounding box, which can be utilized to separate the different PLOs with the same height [22]. First, the minimal bounding rectangle (MBR) enclosing all 2D projected points of PLOs was constructed [27]. As shown in Figure 11(a), the dimensions of the MBR of different PLOs were dissimilar (green rectangles). Therefore, the length (L) of MBRs for an overhead sign, lighting pole-1-sided, and CCTV camera was computed and was then used to distinguish PLOs. In this study, the PLOs were classified as an overhead sign if L ≥ LO, defined as lighting pole-1-sided if LL < L < LO, and labeled as a CCTV camera, otherwise. However, due to the similarity of MBR dimensions, using only the length of the rectangle cannot classify some PLOs (i.e., telecommunication tower vs. speed limit pole), as seen in Figure 11(b). The actual boundary shape of a speed limit pole, which is represented by a concave hull occupying all 2D boundary points in the horizontal plane is a rectangular shape. However, a telecommunication tower is considerably smaller than its bounding rectangle. Therefore, the boundary shape (BSH) of the object is considered, to differentiate PLOs (red polygon in Figure 11). If the ratio of the area of the BSH to the MBR was greater than rS, the object was a speed limit pole. Otherwise, the object was labeled as a telecommunication tower.
(a)
(b)
3. Results and Discussion
3.1. Parameter Settings
The input parameters for the two test sites are tabulated in Table 2. In the nonground extraction step, the value of dE should be less than the maximum spatial distance between two neighboring objects, to separate them into individual objects. However, this value should be larger than the point spacing and the distance between two neighboring scan lines. At a region near the laser scanner, the point spacing can be estimated based on the point density . The distance of 7 cm between two neighboring scan lines is approximate, according to the scan rate (200 scans/sec) and average speed of the MLS car during a measurement (50 km/h). Considering the point density reduction of PLOs that stand at the far range from the laser scanner, dE was selected as 0.2 m. In the PLO detection step, the voxel size is a critical parameter that significantly affects the efficiency of the detection approach. Due to the point density variation, the voxel size should be adequately large to ensure that there are sufficient points in each horizontal slice, during horizontal cross section analysis. However, according to isolation analysis, the voxel size should not be too large, to mitigate the connection of the surrounding non-PLO voxels to the pole trunk. As shown in Figure 12(a), due to the short vertically continuous height of an overhead sign pole (the center-to-center distance of the horizontal truss) in this study, should be chosen to guarantee the independent and vertical continuity analysis of a pole. Thus, was configured as 0.15 m based on the point spacing and the vertical height of the target pole trunk at the test sites. A voxel cluster that has a vertical height threshold (h) greater than 0.6 m was considered a part of a pole trunk. The lower bound of the PLO radius Rmin was set to avoid the detection of some small objects with a linear and cylindrical shape (e.g., small steel bars, wires, guardrails, and tree branches) while the upper bound of the PLO radius Rmax was configured based on the maximum pole radius. The 2D horizontal distance d2D should be less than the minimum distance between two neighboring pole trunks (Figure 12(b)).
(a)
(b)
3.2. Detection Results
Figure 13 depicts the results of the PLO detection from the two test sites. To assess the performance of the proposed method, the detected PLOs from the proposed approach were compared with those obtained from the ground truth data, which were manually extracted by a visual investigation. In this study, three evaluation indices, namely, recall, precision, and F1 score, were used for quantitative evaluation. Recall denotes the completeness of the PLO detection. Precision represents the correctness of PLO recognition. The F1 score is a combination of recall and precision and indicates the overall quality of the method. The calculation of the three indices is expressed by equations (4–6):where TP denotes the number of detected PLOs that match the ground truth, FN indicates the number of undetected PLOs, and FP represents the number of incorrectly detected PLOs.
(a)
(b)
There were 126 and 323 PLOs at Sites 1 and 2 that were manually extracted, respectively. As shown in Table 3, 120 PLOs were extracted successfully at Site 1, where 117 poles were correctly detected. There were 9 undetected PLOs, and 3 objects were misclassified as poles. As a result, the recall, precision, and F1 score were, respectively, 92.9%, 97.5%, and 95.1%, for Site 1. For Site 2, 319 PLOs were detected, where 300 objects were correctly recognized as PLOs by the proposed method, 23 PLOs were not detected, and 19 PLOs were falsely detected. The recall, precision, and F1 score of Site 2 were 92.9%, 94.0%, and 93.5%, respectively.
Almost all road facilities were extracted from both sites. However, there are some PLO detection errors. At Site 1, three bare poles were mistakenly recognized as PLOs because they had a circular section and met the vertical continuity pole trunk conditions (Figure 14(a)). At Site 2, almost all incorrectly detected PLOs were obtained near the tollgate. Four building frames with supporting columns that have a pole-like shape were mistakenly detected (red ellipses in Figure 14(b)). Due to the salient function of the tollgate, many objects with structures that were pole-like were located inside the tollgate and were falsely extracted. As shown in Figure 14(c), the tollgate contains pillars with a circular shape that were mistakenly recognized (purple color). Two parts of the collection booths were incorrectly extracted because of the linear and cylindrical appearance of their wall corners (red ellipses in Figure 14(c)). A drainage pipe close to the tollgate wall was also extracted due to its pole-like features (black ellipse in Figure 14(c)). Eleven nontarget objects with a cylindrical shape that were situated inside the tollgate were also extracted (5 small traffic lights, 1 bollard (short, vertical post), 3 stop-barrier arms, 1 welcome overhead sign, and 1 small camera pole). These mistaken detection errors can be avoided by eliminating the wrongly extracted tollgate. It was observed that the tollgate had an extreme 2D projected area. If the area of the MBR of extracted PLOs was larger than a predefined value, these objects were not considered as PLOs. Then, all detected objects which are positioned inside the tollgate were removed by considering their relative location with the BSH of a tollgate. If they are placed inside the tollgate’s BSH, they are removed.
(a)
(b)
(c)
As seen in Figure 15(a), eight speed limit signs were undetected at Site 1 because their pole trunk height did not satisfy the minimum vertical height (<0.6 m) and their radii were small (<Rmin = 0.03 m). One telephone pole was also not detected due to the existence of noise around the telephone pole, which damaged the isolation assumption (Figure 15(b)). A total of 23 poles were not detected at Site 2. One traffic light was occluded by a reflective board attached to its trunk, which resulted in a lack of vertical continuity, and its density was too low to be recognized (Figure 15(c)). One incomplete overhead sign with supporting columns that were mostly inclined was not detected (Figure 15(d)). One lighting pole was not detected because it was not situated on the ground and was removed in the nonground extraction (Figure 15(e)). Three small speed limit signs failed to be extracted due to a small radius (Figure 15(f)). Six kilometer signs were not detected due to the square section of their trunks. Poles with noncircular sections were PLOs of noninterest in this study and were not recognized by the proposed algorithm. However, they were a type of road facility and were considered as missing detection (Figure 15(g)). Figure 15(h) shows a set of small traffic signs, which are typical road facilities on an expressway. These PLOs were not extracted because they have a short trunk height and a small radius.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
As shown in Figures 16(a) and 16(b), the proposed method could correctly extract the poles in the occluded scenes where a lighting pole was mixed in with trees or a traffic sign was tangled with shrubs. Although the telecommunication pole was attached by electricity cables and signal wires to its trunk which may lead to an imperfect circular shape of the pole section, the algorithm successfully detected the pole due to the RANSAC circle fitting, which is robust to outliers and noise (Figure 16(c)). The complex overhead signs with four supporting pillars which were connected by diagonal and horizontal truss were also correctly recognized (Figure 16(d)). However, there are some incomplete extraction results for the attachments of the poles. For instance, only the upper part of the traffic board was extracted as an attachment of the traffic sign (green ellipse in Figure 16(b)). This is because the vertical region growing algorithm for attachment extraction was applied for only points where the Z elevation is higher than the point with maximum Z elevation belonging to the pole trunk. Similarly, all structural members (horizontal and diagonal trusses) constituting the supporting columns of the overhead sign were not extracted (Figure 16(d)). To tackle this challenge, the attachment extraction method was modified and applied to only the overhead sign. It is observed that the overhead sign contains four columns with a similar height. Also, the structural members are located above the bottom of the column at a certain distance. Based on these features of overhead sign, if the potential cluster contains four different groups of pole trunks (after Section 2.3.3) with a similar height, this potential cluster is considered to contain an overhead sign. Then, the modified vertical region growing is implemented for this cluster. The process starts from the initial point belonging to the column where the Z elevation is higher than the point with minimum Z elevation belonging to the column at a certain distance (e.g., 1.5 m in our real scenes). And, the connected-component analysis is applied to connect all points where the Z elevation is higher than the initial point.
(a)
(b)
(c)
(d)
3.3. Classification Results
The confusion matrix was computed to assess and analyze the performance of the classification method, which is represented by recall, precision, and overall quality. The first two indices were defined the same as those in the detection section, while the last index was determined as the ratio of the correct classification to the total of correctly detected PLOs. The road facilities at Site 1 are different from those at Site 2 because they belong to a different expressway network. Thus, the PLOs were divided into five types at Site 1 and nine types at Site 2, based on their different geometric shapes and height features. The criteria for PLO classification are shown in Table 4 for Site 1, and in Table 1 for Site 2.
The PLO classification results of Site 1 and Site 2 are shown in Figures 17 and 18, respectively. At Site 1, 105 PLOs were correctly labeled from 117 correctly extracted PLOs, resulting in 89.7% for overall quality (Table 5). At Site 2, 294 out of 300 PLOs were correctly classified, and the overall quality was 98.0% (Table 6). It is observed that the classification result for Site 2 is higher than Site 1. That is because the heights for different types of facilities at Site 2 are significantly identified, which led to most PLOs being correctly categorized. However, there are some classification errors. As seen in Figure 17(a), the seven correctly detected telephone poles were misclassified as small signboards at Site 1 because their height and geometric shape are similar to a signboard. Therefore, the proposed classification is unable to classify this type of road facility. Three lighting poles were also misclassified. This is because the data at the bottom part were missing, which resulted in a height reduction. A bare pole was miscategorized as a small signboard because its height and L of MBR satisfy the criteria for a signboard (cyan rectangle, in Figure 17(b)). As depicted in Figure 18(a), the lighting pole-1-sided (special type) was mislabeled as a small traffic signboard due to the missing data of the bottom part, which led to a height reduction of the poles. Figure 18(b) illustrates the incorrect classification of the lighting pole-2-sided. Since the lighting pole-2-sided is similar to the lighting pole-1-sided in height, the lighting pole-2-sided is misclassified as a lighting pole-1-sided.
(a)
(b)
(a)
(b)
3.4. Comparison with Past Studies
It is challenging to compare the results of this proposed method with previous methods since the experimental datasets of previous research are quite different in the road environment, road complexity, point density, and accuracy of laser scanners. Nevertheless, a quantitative comparison was conducted to produce a comprehensive understanding of the proposed method and existing methods, based on the presented datasets in the past studies. The average recall and precision were 92.3% and 83.8%, respectively, for the four test sites in the method proposed by Cabo et al. [1]. Teo et al. [16] presented the coarse-to-fine method to detect PLOs with a recall and precision of about 95%. An average detection rate of 95% was also achieved by Rodríguez-Cuenca et al. [15]. Yan et al. [9] extracted and classified the expressway facilities into five types with an overall accuracy of 91%. The detection rate yielded 88.9% for extracting the lighting poles and traffic signposts by the approach of Guan et al. [28]. Yan et al. [12] proposed a method to extract the PLOs in a motorway environment, and the detection rate from the two test sites was 93.7% and 95.9%, respectively. The PLO classification method was also presented in their study, and an overall accuracy of 96.5% and 97.9% for two datasets was achieved. The average recall and precision of the method by Shi et al. [13] were about 91.6% and 97.3%, respectively. They also successfully classified the PLOs into street lamps, utilities, and traffic signs with an overall accuracy of 91.8%. The method of Li et al. [19] detected the road furniture at a recall of 95% and a precision of 90% from two datasets. The average recall and precision were achieved at 95.5% and 83.6%, respectively, by Li et al. [20].
The proposed PLO detection method in this study achieved an average recall of 92.9% and a precision of 95.8% for two test sites. An overall quality of 89.7% and 98.0% was achieved for the classification of PLOs. The two test sites in our study demonstrated a typical expressway, which consisted of many overlapped objects, as the light poles were standing very close to guardrails or were partially occluded by shrubs. The results showed that our method could detect and classify with a relatively high detection rate, compared with the existing methods. Our method also successfully extracted and classified some common road facilities with 100% accuracy, such as overhead signs, which were not detected and labeled by Yan et al. [9].
4. Conclusions
This paper presented a method for the automatic detection and classification of the road facilities in an expressway environment from MLS point cloud data. First, the ground points were removed, and the nonground points were grouped into distinct clusters. Next, PLOs were detected based on spatially independent and vertical height continuity analysis. Finally, the detected PLOs were classified into different types of expressway facilities based on their height features and geometric shapes. Two test sites were used to evaluate the accuracy of the proposed method. The average F1 score was 94.3% in the detection stage, and an average overall quality of 93.9% in the classification stage was achieved. This shows that our method is effective, robust, and reliable, to detect and label road facilities with high accuracy. Our method uses only XYZ coordinates of the point cloud. This is suitable for other datasets. However, the method could not detect PLOs with a large inclined pole, some small signboards with a short trunk height, or poles with a square section. The method also failed to classify telephone poles where the height and geometric shape are similar to a signboard. Additional geometric features or training data can be used to overcome these limitations, which will be considered in our future research.
Data Availability
The data used to support the findings of this study are available from https://www.infradoctor.site/login and from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The authors would like to thank Metropolitan Expressway Co. Ltd., Elysium Co. Ltd., Aero Asahi Corporation, and Nikon-Trimble Co. Ltd. for providing equipment and the Expressway Authority of Thailand for their assistance during the data collection.