Abstract

Concealed cars extraction from point clouds data acquired by airborne laser scanning has gained its popularity in recent years. However, due to the occlusion effect, the number of laser points for concealed cars under trees is not enough. Thus, the concealed cars extraction is difficult and unreliable. In this paper, 3D point cloud segmentation and classification approach based on full-waveform LiDAR was presented. This approach first employed the autocorrelation coefficient and the echo ratio to determine concealed cars areas. Then the points in the concealed cars areas were segmented with regard to elevation distribution of concealed cars. Based on the previous steps, a strategy integrating backscattered waveform features and the view histogram descriptor was developed to train sample data of concealed cars and generate the feature pattern. Finally concealed cars were classified by pattern matching. The approach was validated by full-waveform LiDAR data and experimental results demonstrated that the presented approach can extract concealed cars with accuracy more than 78.6% in the experiment areas.

1. Introduction

Automatic object extraction becomes a more and more active research topic over the last decades [1]. Some researchers attempt to automatically identify and extract objects from traditional remote images. However successful scenarios are limited. Since the radiometric properties of objects are complicated and dynamic, algorithms that work well with one set of 3D objects are not suitable for other data sets [2]. Moreover, information of targets under vegetation coverage cannot be included in imagery. Thus, extraction of concealed targets cannot be realized based only on images. LiDAR (Light Detection and Ranging) data has unique properties for extraction of 3D objects. Because laser pulse can penetrate vegetation to obtain targets information, LiDAR has irreplaceable superiorities compared to traditional remote images in extraction of concealed targets under vegetation coverage [3]. The most common concealed targets under vegetation coverage are concealed cars. Concealed cars extraction from airborne LiDAR data can be potentially applied to many fields, such as military surveillance, homeland security, global warming, disaster rescue, emergency road service, and criminal searching [4].

Some investigations have been conducted on cars extraction using 3D LiDAR point clouds. Yao et al. [5] presented a method to extract individual car from common LiDAR data in urban areas. Firstly, ground level separation was used to exclude the irrelevant objects and provide the “Region of Interest.” Then the marker-controlled watershed transformation assisted by morphological reconstruction was performed on the gridded and filled raster of ground level points to delineate the single cars. The evaluation of experimental results showed the high potential of airborne LiDAR in outlining single cars in urban areas, which allowed accurate 3D point retrieval of single car. To reliably extract car targets from LiDAR data, Sun et al. [6] proposed a pixel and object oriented method for car extraction based on elevation and intensity data after filtering. The experimental results demonstrated the feasibility of the method. Borcs and Benedek [7] proposed a novel two-level MPP (Marked Point Process) model for the extraction of cars and traffic segments in airborne laser point cloud data. The efficiency of the approach was evaluated with LiDAR data. However, these methods were applied to extract cars on bare ground. Though all bare ground point extraction algorithms performed well on LiDAR point clouds from smooth rural landscapes, they produced errors in rough terrain with vegetation canopy. Chang et al. [8] built a system to detect cars underneath canopy in forested terrain from LiDAR point clouds. They developed an automatic canopy removal algorithm and a novel bare-earth extraction algorithm to reveal the LiDAR points underneath forest canopy and filter ground points, respectively. All obscure cars underneath tree canopy were revealed as demonstrated. It showed that the state-of-the-art airborne LiDAR system can provide valuable data which can effectively support the occluded car extraction in forest terrain. However, only the elevation and intensity features were used to extract the concealed cars in the method.

Since the traditional LiDAR records only several returned echoes and acquires limited information of objects, only range and intensity information can be used to extract the concealed cars. Due to the reflection from canopy, the intensity of points from concealed cars under vegetation is not distinguishable from that of their neighborhood points [9]. Thus, it is difficult to extract the concealed cars under vegetation using traditional discrete LiDAR point cloud data. Unlike traditional discrete LiDAR, waveform features characterizing the inherent attributes of targets can be extracted by processing full-waveform LiDAR data [10, 11]. The features, including number of peaks, Full Width at Half Maximum (FWHM), energy of the returned waveform, and backscattering cross section, can be extracted for targets classification [1215]. Moreover, compared to the points directly obtained by LiDAR systems, point density can be improved to a certain extent by employing waveform features [11]. In this way, the vertical structure information of targets can be obtained more accurately. Therefore, the performance of target extraction can be improved by using the point cloud data affiliated with waveform features. In this study, the full waveform data was first applied to the concealed target extraction which was in line with the current development trend of the target extraction.

Influenced by many factors, such as environment and characteristics of concealed targets, the number of backscattered waveform from concealed targets is less than that of uncovered targets, which results in inadequate waveform features of concealed targets. Therefore, VFH (Viewpoint Feature Histogram) descriptors combined with waveform features were used for concealed cars extraction. VFH [16] is a global descriptor for 3D point cloud data that encodes geometry and viewpoint. Alhamzi et al. [17] have used state-of-the-art 3D descriptors to recognize the objects and evaluated their relative performance. Based on the experimental results, the VFH had the best performance to recognize the objects among other global descriptors. Ceron and Prieto [18] evaluated different combination of three descriptors that are suitable for object recognition and classification: Spin Images, VFH, and NARF (Normal Aligned Radial Feature). The two experiments showed that the VFH outperformed other 3D Shape Descriptors when used alone. Because VFH has high recognition performance and fast computational properties, it has been widely used for objects recognition and classification [16, 1921]. In this paper, the VFH of the point cloud data affiliated with waveform features was calculated and employed for concealed cars extraction. The approach was verified using full-waveform LiDAR data. The results showed that concealed cars extraction can be realized using the approach.

2. Theoretical Background

The objective of this study is to extract concealed cars based on full-waveform LiDAR data. Considering the waveform characteristics of the concealed cars, spatial statistical analysis method was introduced for 3D point cloud segmentation. 3D geometry features and waveform features were combined for points classification.

2.1. Spatial Statistical Analysis

Due to the influence of terrain and environment, the spatial distribution of the waveform echo index from different targets varies. The waveform echo index denotes the position of the echo in the backscattered waveform [22]. The obtainment of such waveform echo index requires a preprocessing step. By waveform decomposition, a number of echoes can be extracted, and then waveform echo index can be obtained. The aim of this step is to extract all (or most of the) relevant peaks in order to generate a dense 3D point cloud [11].

In this paper, the concealed cars refer to the cars under the coverage of vegetation. In general, the echo index value of vegetation area is bigger than that of ground area. Thus spatial statistical analysis of echo index can be used to determine the vegetation areas. The spatial autocorrelation [23], which is one of the most important concepts in spatial statistics, is used to describe the correlation for echo index in different locations. The spatial autocorrelation has two indices, named global index and local index, respectively. The global index [24] is used to detect the spatial autocorrelation of the entire study area. A single value is used to outline the degree of spatial autocorrelation for the entire study area. The local index [25] assesses the extent to which observations of similar and nonsimilar values are clustered for each individual location. Currently the widely used global and local spatial autocorrelation indices are Moran index and coefficient [23, 26, 27]. Compared with the global Moran index which can only find the similarity value (positive correlation) or nonsimilarity values (negative correlation) in spatial aggregation model, the global coefficient is capable of detecting whether the unit belongs to high value aggregation or low value aggregation of echo index in spatial distribution pattern [23]. The detecting ability for spatial clustering of local Moran index and local coefficient also has significant differences. The Moran index can roughly detect the cluster center but is unable to discriminate whether a pattern is dominated by high values or low values. But coefficient can accurately detect the cluster [26]. The vegetation areas are isolated regions; thus the local coefficient is used for spatial statistical analysis on the waveform echo index of concealed cars areas in this work.

The local coefficient [24] measures local concentration by calculating the ratio of the sums of attribute values between local neighborhood and the entire region. The calculation formula of local coefficient is [26] where denotes the echo index value of the th point; is a symmetric one/zero spatial weight matrix with ones for all links defined as being within distance of a given point ; all other links are zero including the link of point to itself. The numerator is the sum of all within of but not including . The denominator is the sum of all within the entire region under investigation not including . The standardization of local coefficient is [26] where is the value of expectation and is the variance. large positive implies that points with high echo index are within of point . A large negative means that low echo index points are within of point [24, 26].

2.2. Description of 3D Point Features

As mentioned, the number of backscattered waveforms from concealed targets is less than that of uncovered targets. Less waveform features of concealed targets are obtained. Especially for concealed cars, even less waveform features can be acquired due to the small size. In order to more effectively extract the concealed cars, not only the waveform features but also the 3D point features were utilized. The Point Feature Histograms (PFH) descriptor [28, 29] is a 3D feature descriptor, the purpose of which is to encode the local neighborhood’s geometrical properties by generalizing the mean curvature at a point using a multidimensional histogram of values. In its most basic form, the computation of PFH at a point relies on the presence of 3D coordinates and estimated surface normals [30]. The computation steps are as follows: (i) For each point , all of ’s neighbors enclosed in the sphere with a given radius are selected (-neighborhood). (ii) For every pair of points and in the k-neighborhood of and their estimated normals and , a Darboux frame is defined and the angular variations of and are computed as follows:

These three features are measurements of the angles between the points’ normals and the distance vector between them. The histogram collects these three angles between every pair of normals on a surface patch [28].

The influence region in the calculation of PFH for the query point is shown in Figure 1 [30]. The query point is marked with red in the middle position of a circle (sphere in 3D) whose radius is , and all its -neighborhood (points with distances smaller than the radius ) is fully interconnected in a mesh [30]. When all possible pairs of points are considered, the computational complexity for -neighborhood is . Thus the theoretical computational complexity of the PFH for a given point cloud with n points is [29], where is the number of neighbors for each query point in a point cloud.

In order to simplify the calculation of PFH, the Fast Point Feature Histogram (FPFH) is introduced. The calculation complexity of FPFH is reduced to , but it still preserves most of the discriminative ability of PFH [16]. The specific calculation steps for FPFH are as follows:(i)For each query point , the relationships (see (3)) between and its neighbors are only calculated. The first step is called Simple Point Feature Histograms (SPFH).(ii)The neighbors for each point are redetermined, and the neighboring SPFH values are used to weight the final histogram of (called FPFH); the specific formula iswhere the weight means the distance between the query point and its neighbor point in a given metric space.

Viewpoint Feature Histograms (VFH) descriptor is derived from FPFH and is mainly used in 3D object recognition and classification [16]. The VFH adds a viewpoint variance while retaining its invariance to scale, as shown in Figure 2 [20]. Thus VFH descriptor consists of two parts: a viewpoint direction component and surface shape component comprised of an extended FPFH [16]. To compute the first part, the centroid of the object is found, which is the point that results from averaging the three coordinates of all points. Next, the vector between the viewpoint and the centroid is computed and normalized. Finally, for all points in the cluster, the angles between this vector and their normals are calculated. The result is represented by a histogram. Note that it does not mean the view angle to each normal as this would not be scale invariant. Instead, it means the angle in the central viewpoint direction translated to each normal. The second part is computed like the FPFH. The resulting four histograms (one for the viewpoint component and three for the extended FPFH component) are concatenated to build the final VFH descriptor [17].

2.3. Waveform Feature Extraction

By decomposing the returned waveforms, waveform features, which are used for concealed cars 3D point cloud segmentation and classification, can be extracted. The extracted waveform features in this paper include distance, intensity, pulse width, backscattering cross section, and echo index. The distance feature indicates the distance from laser transmitter to the target which is determined by estimating the waveform echo position [11]. Ideally, the peak position is considered as component position to calculate the distance. Intensity is not yet a clearly defined term. The echo amplitude is most commonly referred to as intensity [31]. The return intensity is a function related to the reflective capability of the targets. The waveform width denotes the depth extension of waveform in the laser incident direction, which is closely related to the geometry of targets, terrain slope, and targets material [10]. The backscattering cross section delineates the backscattering ability of the targets and is a comprehensive indicator of distance, intensity, and waveform width. It is not a practical geometric area, but the equivalent area that scatters laser pulse back to receiver after the transmitted laser pulse illuminates a target. It can be obtained by the laser energy transmission equation after calibration [32]. coefficient and echo ratio (ER) can be calculated based on echo index. coefficient was introduced in Section 2.1. Echo ratio refers to the ratio between the total number of points in a sphere with the radius of and the total number of points in a cylinder with the radius of [33]. The most suitable value of is two times the average distance between two points.

Some factors, such as angle of incidence, atmospheric correction, range, and surface characteristics, have influence on the waveform features. Therefore, these features can hardly be used without radiometric calibration [34]. To weaken such influence and further improve the effectiveness of waveform features for concealed cars extraction, this paper has made a comprehensive correction on the extracted waveform features. The detailed methodology was introduced in [35].

3. Concealed Cars Extraction Approach

3.1. 3D Point Cloud Segmentation

Using the characteristics of high value aggregation of the waveform echo index for the concealed cars areas, spatial autocorrelation coefficient and the echo ratio were introduced to determine the concealed cars area. Considering the differences of waveform features between ground and nonground points, ground information of concealed cars area was extracted firstly. Then the points within the elevation range of concealed cars were segmented. The workflow is illustrated in Figure 3.

3D point cloud segmentation consists of three steps:(1)Waveform features were first extracted through the decomposition of backscattered waveforms. Next coefficient and echo ratio were calculated. Then coefficient and echo ratio were combined to classify points so that the points in the concealed cars area can be extracted.(2)Ground point information in the concealed cars area was extracted using a split-and-merge segmentation algorithm based on an octree structure proposed by Wang and Tseng in [36]. Consecutively, the points within the elevation range of concealed cars were extracted. Considering the characteristics of the cars, the elevation range was set to 1 m–4 m from the ground. This reduced the influence of uninterested points on 3D point cloud segmentation.(3)The features for each extracted 3D point (one echo) were combined to form a feature vector , where , , and were 3D coordinates of the points and , , , , and were intensity, waveform width, backscattering cross section, coefficient, and echo ratio, respectively. The distance between the feature vectors of different points was calculated. Then KD-tree [37] which was an index structure for storing a finite set of points from a -dimensional space was used for nearest neighbor search in accordance with the distance. For a query point , radius searching was performed. All its neighbors that were within radius were put into a cluster. The same procedure was conducted for the searched neighborhood points until there was no new point that could be put in the cluster, or the number of points in the cluster reached threshold value. In this paper, the distance d was set to 1.5 times the average distance between two points of the experiment data. The minimum number of points for a cluster was set to 40 according to the actual experiment data. In this way, the points that remained after the second step could be clustered.

3.2. Classification

Due to the small size of concealed cars, only a few points on concealed cars can be captured. To classify concealed cars more effectively, more point cloud features reflecting the attributes of cars were required. In this paper, waveform features (distance, intensity, waveform width, and backscattering cross section) and VFH descriptors were combined for classification of point clusters. First, sample data was used for training and KD-tree was built. Then the nearest neighbor searching for the input cluster data was implemented based on the built KD-tree. The distances between the objects in the sample data and input cluster data were obtained and sorted. Then the matching result was determined according to the minimum distance. If the minimum distance was less than the threshold, then the input cluster data was classified as concealed car. The whole process consists of training stage and classification stage.

The flowchart of the training stage is given in Figure 4. The detailed steps are as follows: Collect the full waveform data of different types of cars. Mean, variance and maximum and minimum values of waveform features and VFH descriptors were calculated. The points of concealed cars containing the waveform features and VFH were used as sample data for training. The features of each sample were stored as point cloud feature files and added to a sample library. Based on the traversal of point cloud feature files in the sample library, a KD-tree index was built and saved for subsequent use.

The flowchart of classification stage is shown in Figure 5. This procedure consists of three steps: The point cloud feature files generated in training stage were loaded, and then the KD-tree index was rebuilt. The clusters in Section 3.1 were set as target sets for classification. Mean, variance, and maximum and minimum values of waveform features and VFH descriptors were calculated. Nearest neighbor searching was conducted using the waveform features and VFH descriptors based on KD-tree built in step . The sample data were searched, and feature distances between samples and target sets were calculated. One determined whether the minimum distance of features between the sample and target set was less than a preset threshold, which was determined by the variance of the features. If so, then the target set was classified as a concealed car. Otherwise the target set was classified as other targets, such as the dense thickets or bare cars. Finally the classification results were output.

4. Experiment and Results Analysis

4.1. Data Description

The data used in this paper was acquired by airborne small-footprint full-waveform LiDAR system LMS-Q560 from Riegl. The flight height is 700 meters, the wavelength of the laser is 1550 nm, the divergence angle of laser beam is 0.5 mrad, and the range measurement accuracy is 0.15 meters. The point cloud density is 4 points/m2. Figure 6 shows an example of concealed cars and the corresponding point cloud. Figure 6(a) is a photograph of cars under vegetation and Figure 6(b) is a side view of the original point cloud data of the same area. Three different experiment areas with different vegetation coverage and targets distribution, which were named as experiment area I, experiment area II, and experiment area III in Miyun in Beijing, were selected to test the presented concealed cars extraction approach. The CCD image and original point cloud data which is colorized by elevation are, respectively, shown in Figures 7(a), 7(b), 8(a), 8(b), 9(a), and 9(b). In order to show the position of concealed cars clearly, the point cloud data of concealed cars are visualized using the highlight red color, as shown in Figures 7(c), 8(c), and 9(c). There are 15, 15, and 20 concealed cars in experiment areas I, II, and III, respectively.

4.2. Experiment

The results of concealed cars areas are illustrated in Figures 10, 11, and 12. It shows that the coefficient can be used to extract the big area of trees and the echo ratio can be used to extract the boundary of the trees. Then points within the elevation range of concealed cars in the concealed cars areas were segmented, as illustrated in Figures 10(c), 11(c), and 12(c). A total number of 96, 28, and 82 potential concealed cars were segmented, respectively, for experiment areas I, II, and III. The concealed cars were all segmented, which were marked by numbers in Figures 10(c), 11(c), and 12(c).

In order to verify the effectiveness of waveform features on concealed cars classification, comparative classification experiments were performed on the potential concealed cars shown in Figures 10(c), 11(c), and 12(c). First the classification experiment was conducted using waveform features and VFH descriptors. The other experiment was carried out with only the VFH descriptor.

The classification results using waveform features (intensity, waveform width, and backscattering cross section) and VFH descriptors for experiment area I, experiment area II, and experiment area III are, respectively, shown in Figures 13(a), 14(a), and 15(a). The classified cars numbered 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, and 14 in Figure 13(a), numbered 1, 2, 4, 5, 6, 7, 8, 10, 11, 12, and 14 in Figure 14(a), and numbered 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 15, 16, 18, 19, and 20 in Figure 15(a) are the true concealed cars; the remaining classified cars are the wrong targets. The classification results with only VFH descriptors are, respectively, shown in Figures 13(b), 14(b), and 15(b). The classified cars numbered 1, 2, 3, 4, 5, 6, 8, 11, 12, and 15 in Figure 13(b), numbered 1, 2, 4, 5, 6, 8, 11, 12, and 15 in Figure 14(b), and numbered 1, 6, 10, 11, 12, 14, 15, 16, 18, 19, and 20 in Figure 15(b) are the true concealed cars, while other classified cars are the wrong targets.

The classification in this paper is a binary classification problem and the confusion matrix of binary classification problem is shown in Table 1. Suppose ALL is the total number of clusters obtained in segmentation step and is the number of true concealed cars in the clusters. TP (True Positive) is the number of concealed cars that are correctly classified as concealed cars. FP (False Positive) is the number of other objects (such as thickets beside the trees and bare cars beside the trees) that are falsely classified as concealed cars. TN (True Negative) is the number of other objects that are correctly classified as other objects. FN (False Negative) is the number of concealed cars that are falsely classified as other objects.

The confusion matrix of the classification results with waveform features and VFH descriptors in experiment area I, area II, and area III is shown in Table 2.

The confusion matrix of the classification results with only VFH descriptors in experiment area I, area II, and area III is shown in Table 3.

To evaluate the classification results, the indices including accuracy, recall, and precision [38] are most commonly used and defined as follows:

The evaluation results are shown in Table 4. By comparing the classification results with and without waveform features, we can see that, for the classification results using waveform features and VFH descriptors, the recall rate is improved by 13.3%, 13.3%, and 30% and the precision and the accuracy are improved by 22.6%, 9.6%, and 25.1% and 12.5%, 10.7%, and 17% for these three experiment areas, respectively.

4.3. Results Analysis

Concluding from the experimented results, when the number of points of a concealed target is small, the random error of the features weakens the ability to distinguish different targets. This will greatly increase the difficulty of determining concealed targets. The point density of point cloud data, the target attributes, and the vegetation coverage factor may affect the number of points for concealed targets and then affect the concealed target extraction.

4.3.1. The Point Density of the Point Cloud Data

In order to analyze the influence of point density on concealed target extraction, different degrees of point cloud data thinning for concealed cars were applied. The data after data thinning was used for concealed cars extraction experiments. Figure 16 shows an example of data thinning results for a car. The point density of the experiment data was 4 points/m2; thus the average distance between two points was 0.5 m. There were approximately 20–30 points obtained for each car in this case. After thinning at the rate of 1/2 or 1/4, the point density of the experiment data decreased to 2 points/m2 or 1 point/m2. The number of points obtained for each car also decreased as shown in Table 5. Thus the density of point cloud data influences concealed cars extraction.

4.3.2. The Car Attributes

The size of a car and reflectivity affect the amount of point data that can be returned. Under the same condition, the bigger the target size, the more the points that can be obtained. With the decrease of the size of the target, the difficulty of target extraction increases gradually.

The target’s reflectivity also affects the number of points. The numbers of points for the cars with different colors were analyzed by statistics, as shown in Table 6. The greater the reflectivity, the more the points of the target that can be obtained. The reflectivity of the white car is greater than that of the black car, and the number of points from white car is bigger than that of black car. Thus the car attributes influence concealed cars extraction.

4.3.3. Vegetation Coverage Factor

In order to analyze the vegetation cover effect on concealed cars extraction, the vegetation coverage factor was calculated for some concealed cars as follows:

From the results, as shown in Table 7, we can see that the number of points is bigger when vegetation coverage factor is smaller. It can be drawn that with the same targets’ attributes and point density, the smaller the vegetation cover factor is, the bigger the number of targets’ points is and the more likely it is to extract the concealed targets. Thus the vegetation cover factor also influences concealed cars extraction.

Considering these factors as mentioned above, the errors of the concealed cars extraction were analyzed. The errors mainly came from two aspects. One issue was that some concealed cars were missed. That might be because these concealed cars were black or the vegetation cover factor was big for these cars; thus the number of points was not enough. It was difficult to extract the concealed cars with only a few points. The other issue was that some objects were wrongly classified as concealed cars, such as the dense thickets beside the trees or the bare car next to the trees. If the dense thickets beside the trees happened to have the similar elevation and waveform features with concealed cars then they might be classified as concealed cars. If the bare cars were very close to the trees and had enough points, then they might be classified as concealed cars, as shown in Figures 8(a) and 14(a) with the red rectangles.

In the future, the density of point cloud in the experiment area can be increased so that more points of the concealed cars can be obtained. Then the probability of missing the concealed cars can be decreased. Furthermore, more waveform features and geometry features should be extracted for better distinguish between dense thickets and concealed cars. In addition, more features can be used to accurately extract the points in vegetation areas so that the bare cars can be removed.

5. Conclusion

A concealed cars extraction algorithm was presented in this study. First coefficient and echo ratio were combined to determine concealed cars area based on full-waveform LiDAR data. The elevation distribution of the points in the concealed cars area was further analyzed to extract the concealed cars. Then by sample training and pattern matching, the concealed cars classification approach combining waveform features and VFH descriptor was presented. Experiments showed that the algorithm could correctly extract concealed cars, and comparative experiments further indicated that the waveform features could improve the accuracy, recall, and precision for concealed cars classification. However, this study focuses on concealed cars under trees; whether the algorithm is feasible to other types of concealed targets needs to be further investigated.

Competing Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work was partially supported by National Natural Science Foundation of China (Grant no. 40901177), National High Technology Research and Development Program of China (863 Program) (Grant no. 2015AA123801), National International Science and Technology Cooperation Project (Grant no. 2015DFA70930), and Key Projects of Foreign Cooperation in the International Cooperation Bureau of Chinese Academy of Sciences (Grant no. 181811KYSB20130003).