Abstract
Complex pavement texture and noise impede the effectiveness of existing 3D pavement crack detection methods. To improve pavement crack detection accuracy, we propose a 3D asphalt pavement crack detection algorithm based on fruit fly optimisation density peak clustering (FO-DPC). Firstly, the 3D data of asphalt pavement are collected, and a 3D image acquisition system is built using Gocator3100 series binocular intelligent sensors. Then, the fruit fly optimisation algorithm is adopted to improve the density peak clustering algorithm. Clustering analysis that can accurately detect cracks is performed on the height characteristics of the 3D data of the asphalt pavement. Finally, the clustering results are projected onto a 2D space and compared with the results of other 2D crack detection methods. Following this comparison, it is established that the proposed algorithm outperforms existing methods in detecting asphalt pavement cracks.
1. Introduction
Globally, the economic framework and evolving networks of cities depend on road traffic [1]. The detection of pavement distress is an important part of pavement maintenance. Crack is one of the critical pavement defects, and crack length measurement is an important aspect of asphalt pavement crack detection. Now, there are already some crack detection methods [2]. However, most crack measurement techniques have low accuracy and efficiency. Crack detection technology, which provides significant technical support in pavement maintenance, is becoming more advanced.
Traditional artificial crack detection methods affect traffic and have low efficiency, strong subjectivity, and low accuracy; they have been unable to meet the growing pavement maintenance needs for a long time [3]. With the development of computer technology, the emergence of many pavement crack detection technologies based on digital image processing, such as Laplace, Sobel, Prewitt, Roberts, and Canny operators and other edge detection algorithms, has greatly improved pavement crack detection efficiency and accuracy. However, these detection algorithms are highly sensitive to noise. Other pavement crack detection methods are based on morphology [4–6], threshold segmentation, and other approaches [7]. These algorithms provide an important reference for improving crack detection accuracy [8]. Although there are some quite effective 2D crack detection methods [9], 2D crack detection methods are easily affected by light, shadow, pavement signs, and oil stains. Therefore, there is growing scholarly interest in the study of 3D detection of asphalt pavement cracks [10].
In recent years, 3D detection technology based on structural optics has gradually become a research hot spot [11]. There are some automated 3D road condition assessment methods based on liquid chromatography-mass spectrometry (LCMS) and road detection [12]. By using a camera to capture the contour of the pavement and imaging it through 3D reconstruction, more effective solutions can be found to the environment-related limitations of 2D images [13]. Other methods include the 3D pavement crack detection algorithm based on 2D empirical mode decomposition and the 3D pavement damage data surface-based filtering method. The deep learning algorithm can also been deployed for recognising crack images in 3D [14]. However, this form of algorithm does not process crack images that differ from the training samples effectively because it is highly dependent on feature selection and the richness of training samples, and the calculation cost is also very high [15].
In summary, although 3D images can avoid environmental interference more effectively, accurately detecting cracks is still impeded by significant challenges due to the influence of factors such as uneven background, varied morphology of cracks, and complex pavement texture [16]. 3D pavement crack detection is the subject of increasing scholarly attention. Clustering is an important unsupervised data mining method in the field of machine learning. Clustering by fast search and finding density peaks (DPC) is a new clustering algorithm proposed in 2014 that can cluster nonspherical data efficiently [17]. The clustering centre is selected according to this algorithm: (1) The local density is higher; that is, the density values of the neighbouring points of the central point are all less than that point. (2) Larger “distance” from other points with higher density [18]. The DPC algorithm is simple in principle, efficient in clustering, and has been widely used in various fields, such as expressway toll data analysis [19] and abnormal event mining [20]. However, the DPC algorithm also has very manifest shortcomings. It is necessary to generate a decision graph to select the clustering centre based on the calculated local density and distance values [21]; it is also necessary to set the cutoff distance according to the experience value. The points with the larger values are artificially selected as the clustering centre in the decision graph. However, the subsequent distribution and optimisation of off-centre points and the discovery of noise points ARE significantly affected by the high subjectivity and instability characterising this selection [22]. To solve both shortcomings, this study uses the fruit fly optimisation algorithm to optimise the parameters of the fast peak clustering algorithm and proposes an adaptive clustering algorithm. Then, the improved fruit fly optimisation density peak clustering (FO-DPC) algorithm was used to cluster the 3D asphalt pavement data, accurately identify the cracks, and obtain their parameters to provide support for the pavement distress assessment.
2. Materials and Methods
2.1. Components of 3D Pavement Crack Detection System
The asphalt pavement crack detection system based on the FO-DPC algorithm has three main components. The flow chart in Figure 1 shows how the system collects data and detects cracks. The proposed system involves the acquisition of the 3D data of the asphalt pavement, 3D pavement data clustering based on the FO-DPC algorithm, and pavement crack feature extraction.
The 3D data of the asphalt pavement are collected using the Gocator3100 series binocular intelligent sensors; and then the data are preprocessed using Lagrange interpolation and median filtering. The input of the clustering algorithm proposed in this paper is the distance matrix including the data number and the distance between each data point. In this study, the similarity measurement method used to calculate the distance matrix is the Manhattan distance. The clustering results give each data point a classification label. According to the different labels, the 3D pavement data are mapped to the 2D image using different colours. Then, crack features are extracted.
2.2. Acquisition and Preprocessing of Asphalt Pavement 3D Data
After investigation, most regions have approximately 1000 mm annual precipitation; the implication of this is that most pavement surfaces are quite wet over the years. Meanwhile, the daily precipitations of most regions exceed 10 mm, indicating extreme intense precipitation conditions. The presence of water stains will greatly affect 2D crack detection. This greatly increases the need for 3D detection. The Gocator3100 series binocular 3D smart scanner is an industrial inspection device that combines 3D point cloud data acquisition and measurement tools. The built-in variety of tools makes it ideal for a variety of contactless online inspection applications. The Gocator binocular 3D smart scanner is easy to install, easy to integrate, and highly reliable. The performance parameters of the Gocator3110 used in this study are shown in Table 1.
The Gocator3100 series binocular smart sensor, which is used for data collection, is shown in Figure 2; it is based on binocular stereo vision, and its 3D reconstruction principle is parallax. Two cameras are used to image the same object from different angles to obtain two perception images at different angles. Then, the three-dimensional information of the target object in the real environment is restored based on the principle of parallax. In this experiment, two cameras are placed at a certain point of view, as shown in Figure 3. The origin of coordinates of approximately two cameras is located at the camera light heart, with the assumption that the imaging plane is located in the front of the camera; f is around two-image coordinate system origin, and the optical axis peace intersection coordinate of both cameras meets at OL and OR below.
Figure 4 shows the working scene of collection of the asphalt pavement data in the author’s university and a top view of the data acquisition device. The binocular camera is mounted on a shelf. Both the 3D data and 2D images of the cracks, which are shown in Figure 5 and Table 1, were collected. The 2D images were acquired to facilitate the subsequent comparison of the effectiveness of the 2D detection method and the proposed algorithm at identifying pavement cracks.
2.3. Data Preprocessing
The 3D pavement data used in this study are mainly comprised of the collected pavement contour data and data formed through 3D reconstruction. In addition to capturing the contours of cracks, the 3D data will show the different heights of asphalt pavement, textures, ruts, road signs, potholes, or rubble.
Table 2 shows some 3D pavement data collected using the binocular cameras. The first row and the first column in the table are the horizontal and vertical coordinates of the pavement data. The table is the height value of the pavement; there are some missing values. The missing values are filled in using Lagrange interpolation. For the n + 1 different interpolation node, Lagrange interpolation polynomials of up to n times can be constructed to satisfy the interpolation condition: the function value of the interpolation polynomials of up to n times is 0 at the node of missing data. Point is referred to as the interpolation node. Equation (1) is the interpolation basis function of the interpolation polynomial. When the missing value is being filled, the interpolation polynomial is constructed first, and then the interpolation node is interpolated to obtain the missing value. In this study, n takes the value of 15:
To consider the crack identification speed and accuracy, the original data are preprocessed using median filtering. The basic principle of median filtering is to replace the value of a point in a digital image or digital sequence with the median value of the values of each point in the neighbourhood of the point so that the surrounding pixel value is close to the real value to eliminate the few isolated noise points, as represented in the following equation:
The purpose of filtering is to smooth the data and remove the influence of the pavement aggregate texture and depth of construction on crack detection.
The preprocessing results of the cross section of the 3D asphalt pavement data is shown in Figure 6. Preprocessing can remove the noise generated in the process of data acquisition.
2.4. 3D Crack Detection of Asphalt Pavement Using FO-DPC Algorithm
2.4.1. Asphalt Pavement Crack Detection Process
In this study, the clustering method is adopted to detect cracks. The height values of the asphalt pavement in the collected 3D crack data and the parameter calculation, clustering centre selection, noncentre point allocation, clustering result optimisation, and noise point identification are clustered. Then, the data points are identified as common pavement or pavement cracks. The clustering results of the 3D pavement data are presented as 2D images. Then, the flow chart of the pavement crack detection based on the clustering algorithm is shown in Figure 7.
2.4.2. Density Peak Clustering Algorithm Based on Fruit Fly Optimisation
(1) The DPC Algorithm. As a simple, easy-to-understand clustering algorithm, the DPC algorithm involves three steps: Step 1: calculation of the distance matrix Before the distance is calculated, it is necessary to normalise the different numeric fields of the original data to ensure the data of each dimension have the same order of magnitude. Following normalisation, the distance between data is calculated, and the distance matrix is finally outputted. In this study, the Euclidean distance method is used to calculate , and the calculated distance is outputted according to the ordinal number of two data points and the corresponding distance as a matrix with column number 3; this distance matrix is the input of the DPC algorithm. Step 2: selection of the cluster centre The clustering centre is characterised by high local density and large distance from other points with higher density. The local density of a point is the number of points whose distance to the point is less than the given cut off distance: where i and j are two different data points. When , ; otherwise, . In other words, if the distance from j to i is less than , the local density is increased by 1. is the distance between i and j, and is the cut off distance, which is set by the user according to experience. The value of data is defined as follows: For the point with the highest density, there is no point with higher density, the value of the highest density is defined as the maximum distance between this point and all other points: After both values are calculated for each point, all the points are visually outputted, with and as two dimensions. The output graph is referred to as the decision graph (Figure 8(b)). To show the selection principle of the clustering centre in this algorithm simply and intuitively, the selection process of the clustering centre is more vividly illustrated with 28 sample data. All the data points are sorted and numbered according to the principle of decreasing density; that is, the density of Point 1 is the largest, and the density of Point 28 is the smallest. The specific distribution is shown in Figure 8(a). and of the 28 points in the above sample are calculated one by one, according to equations (3)–(5). The decision diagram is shown in Figure 8(b). Although Points 26, 27, and 28 all have a large , their points are all small, and they have the lowest density of all the sample points. According to the distribution diagram in Figure 1, they are scattered around the main cluster and can be regarded as a cluster composed of single points, namely, noise points. Points 1 and 10 in the upper right corner of the coordinate system both have higher and higher ; thus, these two points are selected as the clustering centre. It can also be seen from the decision graph that although Points 9 and 10 have similar , the values are completely different. of Point 10 is significantly larger than that of Point 9, and of Point 9 is close to 0. According to the original data distribution diagram, Point 9 belongs to the first class cluster, and several other points with relatively high values close to it are also included in the first class cluster; Point 10 is the centre of another class. Therefore, it can be concluded that the points with large and are clustering centres. The points with small and large are the noise points, and the points with large and small are the points in the class cluster. Step 3: clustering and optimisation of other points After finding the clustering centre, the number of classes is determined, after which it becomes necessary to systematically allocate other points to the most suitable cluster. According to the distribution principle for these points, each point should be assigned to the cluster with higher density nearest to it one by one, and this operation will be carried out step-by-step, until all the points are allocated to the corresponding cluster. The DPC algorithm first finds a boundary region for each class. A boundary region exists where the distance between the points assigned to one kind of cluster and the points of another kind of cluster is less than the cut off distance . Then, the points with the highest density within the boundary region of each cluster are found, and their densities are represented as . All points within the cluster are traversed, and points with a density higher than are assigned to the cluster or marked as noise points. The DPC algorithm is a new clustering algorithm, and it has some shortcomings. It is necessary for the algorithm to manually set the cut off distance, according to the empirical value, and select the clustering centre. In this study, the fruit fly optimisation algorithm (FOA) is used to optimise the DPC algorithm and then used for pavement data clustering to realise 3D asphalt pavement crack detection.
(a)
(b)
(2) Fruit Fly Optimisation Algorithm. The FOA is a population swarm intelligence optimisation algorithm, a global optimisation method derived from the complex foraging behaviour of fruit flies. It is divided into the following steps: Step 1: initialisation The following parameters were determined: initial size of fruit fly , initial position , and maximum number of iterations . Step 2: olfactory search First, the number of iterations was specified, and the random flight direction and the search step value (distance value) of the individual fruit fly in the olfactory search phase were set during the iteration. The search direction is , and the search step size is ; then, Step 3: preliminary calculation Because it is impossible to know the specific location of the initial food source, the distance between the individual fruit fly and the origin of the coordinate was first calculated to obtain the determination value of taste concentration. The equation is as follows: Step 4: calculating the taste concentration value The judgment value of the taste concentration is substituted into the judgment function of smell concentration, and the current concentration value of each individual fruit fly is calculated as follows: Step 5: find out the coordinates of the individual fruit flies with the highest smell concentration in the current fruit fly population according to the smell-concentration value: Step 6: visual search Record and retain the and the optimal fly coordinates at this point; the entire fly population uses their sharp vision to fly to the optimal individual location: Step 7: iterative optimisation Termination condition is the basis of iterative optimization decision. When , continue Steps 2–5; compare the best taste density with the best taste density of the previous iteration. If the performance is better than the previous iteration, perform step 6. Otherwise continue to execute steps 2–5, loop the process until , and stop the iteration of the algorithm.
(3) FO-DPC Algorithm. Information entropy is the measure of the uncertainty of the system. Thus, the entropy is directly proportional to the uncertainty of the system. Given that the space contains the data set of n objects, let the density function value of each object be ; then, the density estimation entropy can be defined as follows:where is the normalised factor.
The FO-DPC algorithm flow is as follows: Step 1: iteration number—1000 and population size—10. Step 2: the fruit fly population search; calculate the individual concentrations to determine the distance from the origin of coordinates and taste value . Step 3: use the information entropy as the evaluation criterion to determine the effectiveness of each fruit fly population, locate the fruit fly population with the highest effectiveness in each generation, and record its optimisation results. Step 4: according to the principle of minimum information entropy, select the best position of fruit flies. Step 5: calculate the density and distance of each data point using the DPC algorithm. Obtain the final clustering result.
(4) 3D Asphalt Pavement Crack Detection Based on the FO-DPC Algorithm. Some of the collected 3D data of pavement are reconstructed, as shown in Figure 9. Different height values are displayed in different colours. During data collection in this study, 330 175 data points are collected for each group. The horizontal and ordinate coordinates in Figure 9 are the locations of data points on the asphalt pavement, and the vertical coordinates are the height values of the data points. The top view in Figure 10 provides a more intuitive view of the structure of the asphalt pavement.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
In this study, the proposed algorithm is used to cluster the height values of the 3D data of the asphalt pavement. In Figure 11, (a) is the 3D construction of the original data, (b) is the top view of the pavement, (c) is the 3D display of the clustering results, (d) is the photo of the crack, and (f) is the 2D display of the clustering results. As can be seen from Figure 11, due to the preprocessing, the noise in the data used for clustering is small, and there is a large difference between the cracks and pavement. In addition, the proposed clustering algorithm is highly efficient and has high detection accuracy.
(a)
(b)
(c)
(d)
(e)
Figure 12 is the clustering result of the pavement height value, where (a) is the clustering decision graph and (b) is the final result of clustering. In Figure 12(b), the one-dimensional data of the height value are clustered and analysed, and the clustering result is only displayed on the horizontal axis of the label. The horizontal and vertical coordinates of Figure 12 are the calculated local density and the distance to a point of higher density and have no practical physical meaning, so the unit is not marked. The cluster label (horizontal axis), the original data coordinate (vertical axis), and some crack clustering results are shown in Figure 13. Finally, the results of clustering detection are processed through the morphological treatment of expansion and corrosion, as shown in Figure 14. To improve the connectivity between pixels, this study uses the morphological closure operation, that is, expansion, followed by etching.
(a)
(b)
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
3. Results and Discussion
3.1. Benchmark Processing Algorithms
To compare the detection effect of the proposed algorithm with the traditional 2D detection algorithm, Canny edge detection and Otsu threshold segmentation are used to detect the 2D images of pavement cracks, which were collected simultaneously with the 3D data. The 3D data are detected using the k-means algorithm.
The Canny edge detection algorithm consists of five steps: smoothing the image using Gaussian filtering, finding the intensity gradient, eliminating edge false detection, determining the potential boundary using the double threshold method, and tracking the boundary.
The specific steps of the Otsu threshold segmentation algorithm are as follows: calculating the standard histogram, calculating the cumulative mean and the global grey mean, calculating the probability of assigning various classes, calculating the variance between classes, and identifying the maximum variance as the optimal threshold, which is then used to split the image.
The k-means algorithm comprises the following steps: setting the K value, randomly selecting the K-cluster centres, calculating the distance from each point to the cluster centre, assigning each point to the cluster where the nearest cluster centre is located, and recalculating each clustered cluster centre and looping until the cluster centre remains constant.
3.2. Performance Criteria
We use a label tool to mark the 2D images acquired simultaneously with the 3D data to obtain the pseudoground truth for subsequent evaluation.
The pavement data are identified using the Otsu threshold segmentation, Canny edge detection, and the proposed algorithm, and the real crack images are manually marked. The data results are evaluated according to four scenarios: true positive (TP: actually a positive example, same as expected), false positive (FP: actually a negative example, contrary to expectation), true negative (TN: actually a negative example, same as expected), and false negative (FN: actually a positive example, contrary to expectation). Then, the accuracy (P), recall rate (R), and F value of crack identification are calculated by comparing the detection results with the real crack images:
The DSC is the harmonic average of the accuracy and recall rate:
Averaging the results of each data used in the laboratory, Table 3 shows the accuracy rate (P), recall rate (R), and DSC of the three algorithms. It can be seen that the accuracy, recall rate, and F value of the Otsu threshold segmentation and Canny edge detection are all very low. The accuracy, recall rate, and F value of the proposed algorithm are the highest.
3.3. Illustrative Results
Four kinds of typical pavement crack image detection results were obtained, as shown in Figure 15. In the first image, the difference in the grey values of the asphalt pavement and crack is very small. Although the difference between the grey values of the asphalt pavement and crack in the second image is small, the 2D detection method still accurately detects the crack. However, due to the presence of water stains, the grey values of the asphalt pavement and cracks are quite different in the third image; thus, the water stains will be wrongly identified as cracks. The grey value difference between the aggregate and pavement in the fourth type of crack image is higher than that in the crack image, and detecting the crack in the figure is more difficult.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
In the second type of pavement crack image, for instance, first, the original 3D image of the pavement crack is labelled manually. Then, the marked graph, the detection result graph of the two 2D crack detection methods mentioned above, and the proposed detection method are divided into several square images of fixed size. Subsequently, the file name of the subblock image that contains the marked or detected crack is changed. Finally, the detection accuracy, recall rate, and F value of each method were calculated.
In Figure 15, (a) is the original cracks, (b) is the pseudoground (when labelling, our algorithm detects the cracks in the pavement and the surrounding area, so it labels an area larger than the crack of the skeleton), (c) is the grayscale histogram of the crack image, (d) is the processing results following Otsu threshold segmentation, (e) is the result of Canny edge detection, (f) is the detection result of the k-means algorithm, (g) is the detection result of the proposed algorithm; the morphological processing of the test result is in (h). (d) and (e) are 2D-based, and (f)–(h) are 3D-based. The linear features and region with low depth value are identified as cracks using Otsu threshold segmentation; thus, the recall rate is high and the accuracy rate is low. Canny edge detection can detect the boundary features of depth variations; however, it cannot detect crack pixels with slow depth changes. Therefore, the recall rate is low. At the same time, it is prone to noise interference, resulting in a decrease in its accuracy. Similarly, the 3D data are clustered using the k-means algorithm. The crack detection result of the k-means algorithm is better than that of the 2D method. However, due to the limitation of data distribution, the clustering effect is still not sufficiently effective. The proposed clustering algorithm is density based and can effectively circumvent the above challenges. It is based on height feature and distribution feature. The FO-DPC algorithm is used to identify cracks with high accuracy and high recall rate. In addition, due to the effective preprocessing, the crack identification accuracy is also high; thus, the final F value is as high as 89.97%.
The 2D image processing algorithms, such as Otsu threshold segmentation and Canny edge detection, can only segment and detect the cracks based on the grey value of the image; however, the grey value of asphalt is close to that of the cracks; the difference is small, which affects the 2D detection effect. There is a small amount of water in some of the cracks recorded in this study, and the colour is much more different from that of ordinary pavement. Although the 2D method can effectively detect the cracks, it cannot effectively detect the dry cracks. The proposed algorithm relies on the height of the road surface, and is thus unaffected by these factors.
4. Conclusions
This study proposed an FO-DPC algorithm-based pavement cracking detection approach. The use of binocular cameras to collect the asphalt pavement data ensured that more accurate and less noisy data were collected. The proposed FOA-DPC algorithm effectively solved the challenge posed by the necessity of determining the cut off distance based on empirical values and uses the fruit fly optimisation algorithm to obtain the optimal parameters. Moreover, by choosing the clustering centre adaptively, it effectively solves the instability caused by the artificial selection of clustering centre. Finally, the 3D asphalt pavement detection based on the proposed algorithm can effectively identify pavement cracks based on the height value and spatial distribution characteristics of 3D data. The algorithm outperforms the traditional Otsu threshold segmentation, Canny edge detection algorithm, and k-means algorithm in terms of accuracy, recall rate, and F value. The algorithm exploits the high precision of the 3D data of pavement to identify cracks, thus compensating for the defects of the quality of the 2D image, which is susceptible to noise relating to lane marking, shadows, and oil stains. In subsequent studies, the threshold and Otsu methods will be compared to the 2D versions of the k-means and FO-DPC algorithms, and the effectiveness of the same clustering method for 2D and 3D data will be compared.
Data Availability
The source data, models, or code are supposed to be optimised with respect to current versions of the developed models to minimise their limitations. The authors would like to point out that some of the data, models, or code generated or used during the study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This study was supported by the National Key Research and Development Plan “Integrated Transportation and Intelligent Transportation” Special Project (no. 2018YFB1600202), National Natural Science Foundation of China (no. 51978071), and Fundamental Research Funds for the Central Universities, CHD (nos. 300102249301 and 300102249306).