Abstract
Potholes are one type of pavement surface distresses whose assessment is essential for developing road network maintenance strategies. Existing methods for automatic pothole detection either rely on expensive and high-maintenance equipment or could not segment the pothole accurately. In this paper, an asphalt pavement pothole detection and segmentation method based on energy field is put forward. The proposed method mainly includes two processes. Firstly, the wavelet energy field of the pavement image is constructed to detect the pothole by morphological processing and geometric criterions. Secondly, the detected pothole is segmented by Markov random field model and the pothole edge is extracted accurately. This methodology has been implemented in a MATLAB prototype, trained, and tested on 120 pavement images. The results show that it can effectively distinguish potholes from cracks, patches, greasy dirt, shadows, and manhole covers and accurately segment the pothole. For pothole detection, the method reaches an overall accuracy of 86.7%, with 83.3% precision and 87.5% recall. For pothole segmentation, the overlap degree between the extracted pothole region and the original pothole region is mostly more than 85%, which accounts for 88.6% of the total detected pavement pothole images.
1. Introduction
The pavement maintenance is important and helpful for saving cost and ensuring driving safety, of which pavement condition assessment is crucial to developing repair and maintenance strategies. The process of pavement condition assessment can be divided into three parts: data collection, distress identification, and defect assessment. The data collection is to a large extent automatically performed, while the latter two are mostly performed manually. This paper focuses on pavement pothole identification, including pothole detection and segmentation.
Potholes are important clues that indicate the structural defects of the asphalt road. They are usually caused by aging, heavy traffic, poor drainage, thin asphalt surface, weak substructure, too little asphalt in the mix, and so forth. These localized failed areas would delay traffic, frustrate drivers, create a dangerous driving condition, and cause expensive repairs because of tire wear, wheel alignments, and structural fatigue [1]. A pothole is defined as a bowl-shaped depression in the pavement surface, and its minimum plane diameter is 150 mm [2]. Accurately detecting these potholes is an important task for asphalt-surfaced pavement maintenance and rehabilitation [3].
Because the commonly used manual detecting methods are expensive and time-consuming, several efforts have been made to automate the pothole detection process. At present, the automatic pothole detection methods include vibration-based methods, 3D reconstruction-based methods, and 2D vision-based methods [4]. These methods promote the automatic detection process significantly, but they have both advantages and disadvantages. Vibration-based methods [5–8] require small storage and can be used in real-time processing, but it can only get a rough assessment of pavement potholes or even provide wrong results. 3D reconstruction-based methods [9–15] can obtain detailed information of the potholes, but the drawback is that they cost too much equipment money or too much calculation time. 2D vision-based methods [16–22] can find a balance between the vibration-based methods and 3D reconstruction-based methods. It only needs on-board high-speed camera to obtain the road image and the processing time is little. However, the existing visual-based methods carry out the grayscale and texture information of the pothole separately in turn, which will affect the detection results.
In this paper, a pothole detection and segmentation method based on wavelet energy field is proposed. The wavelet energy field effectively integrates the grayscale and texture information together, which can accurately and efficiently detect the pavement pothole. Then, the constructed wavelet energy field is used as the label field of Markov random field model, which significantly promoted the pothole segmentation accuracy. The proposed method has been implemented in a MATLAB prototype and achieves a high detection and segmentation accuracy.
2. Literature Review
At present, automatic pavement pothole detection methods mainly include three different types: vibration-based methods, 3D reconstruction-based methods, and 2D vision-based methods [4], of which the latter two types belong to computer vision technology. Details of previous works about pothole detection are discussed below.
Vibration-based methods use accelerometers to record the mechanical responses imposed by the pavement surface and apply the responses to estimate the surface condition. B. X. Yu and X. Yu [5] have proposed a preliminary pavement condition survey. De Zoysa et al. [6] and Eriksson et al. [7] proposed a similar method that combines accelerometers and GPS to locate the position of the pothole. Rode et al. [8] integrated the detection and warning system that could help the driver avoid potholes. In general, vibration-based methods require small storage and can be used in real-time processing. However, it can only get a rough assessment of pavement potholes or even provide wrong results that the hinges and joints of road can be detected as potholes and potholes in the center of a lane cannot be detected using accelerometers due to no hit by any of the vehicle’s wheels [7].
3D reconstruction-based methods include 3D laser scanner methods, stereo vision methods, and visualization using Microsoft Kinect sensor. Chang et al. [9] applied time-of-flight scanners, with an accuracy of a millimeter, to get accurate 3D point-cloud points and extract specific distress features by means of a grid-based processing approach to quantify the severity of pavement distresses. Li et al. [10] developed a real-time, low-cost laser triangulation based inspection system, with at horizontal and vertical resolutions of around 2 mm, to achieve 3D geometric features to differentiate different distresses, like potholes and rutting. Wang [11] used two digital cameras, while Hou et al. [12] used four digital cameras to detect and classify any pavement distress, and the images of the same pavement distress surface are matched and reconstructed to establish 3D surface model. Joubert et al. [13] developed a low-cost vehicle mounted sensor system using Kinect sensor and a high-speed USB camera to detect and analyze potholes and pack the results together with the GPS coordinates of the pothole. Moazzam et al. [14] used Kinect sensor to capture depth images, generated meshes of the pothole, and calculated some geometrical information of the pothole like depth, area, and volume. Mahmoudzadeh et al. [15] provided a thorough literature review on usage of Kinect in pavement management and proposed the best approach which is cost-effective and precise. 3D reconstruction-based methods can obtain detailed information of the potholes. However, laser scanning systems cost too much to be applied at vehicle-level, and stereo vision methods need a high computational effort to reconstruct pavement surface through the procedure of matching feature points between different views. Although Microsoft Kinect sensor is cost-effective as compared to industrial lasers and cameras, the visualization method using Microsoft Kinect sensor is still a novel idea and further research is necessary.
2D vision-based methods can find a balance between the vibration-based methods and 3D reconstruction-based methods. The equipment cost is little, and the detection results can obtain high accuracy. 2D vision-based methods can be further distinguished into 2D image-based approaches and video-based approaches. For 2D image-based approaches, Lin and Liu [16] extracted the texture measure based on histogram as the features of the image region, and the nonlinear support vector machine was built up to identify whether a potential region is a pothole or not. The method proposed by Koch and Brilakis [17] firstly segmented the pavement images into defect and nondefect regions, then the potential pothole shape was approximated, and subsequently, the texture inside a potential defect shape was extracted and compared with the texture of the surrounding nondefect pavement to determine if the region of interest represents an actual pothole. Buza et al. [18] utilized image processing and spectral clustering method to detect potholes. Ryu et al. [19] presented a method that combines image processing with geometric information to extract the potential pothole region and compared the histogram similarity and standard deviation of the potential pothole region with the nondefect region to identify whether the potential region is a pothole or not. In video-based approaches, Huidrom et al. [20] used a heuristic decision logic approach that combines area, standard deviation, roundness, and diameter information to detect and classify potholes and cracks from video clips of Indian highways. Koch et al. [21] presented a method to evaluate the severity of potholes by incrementally updating texture signature for intact pavement regions and using a vision tracker to trace detected potholes over a sequence of frames. Tedeschi and Benedetto [22] proposed a new system using computational resources provided by a mobile Android-based device, which embedded the recognition system based on OpenCV library, to detect the potholes and cracks. All these 2D vision-based methods have achieved good detection results, but they carry out the grayscale and texture information of the pothole separately in turn, which will reduce the detection accuracy.
Considering the significance of the pothole exists in both the grayscale and texture aspects, a different 2D vision method based on wavelet energy field is proposed in this paper. The constructed wavelet energy field integrates the grayscale and texture information together to improve detection accuracy. And the wavelet energy field is used as the label field of Markov random field model to segment the pothole accurately.
3. Mathematical Models
As is shown in Figure 1, the method proposed by this paper mainly includes two aspects, that is, pothole detection and pothole segmentation. First of all, the pothole is detected by morphological processing of the constructed wavelet energy field. Secondly, the detected pothole is segmented using Markov random model and morphological operation. Above all, the wavelet energy field model and the Markov random field model are the keys for pothole detection and segmentation.
3.1. Wavelet Energy Field Model
The high-frequency coefficients of the image wavelet decomposition reflect the detailed information of the pavement image. In image segmentation, taking the wavelet coefficients of the same scale or different scales as texture features, and combining with FCM (Fuzzy -means) algorithm, artificial intelligent algorithms, or Markov random field, can achieve good segmentation results. For example, in the detection of pavement cracks, the method proposed by Nejad and Zakeri [23] has combined wavelet decomposition and neural network to obtain good detection and classification results.
However, the texture feature of the pothole is more complex than that of the crack, such as gravels outside the pothole, and fine grained texture inside the pothole would reduce texture significance that only taking the wavelet high-frequency coefficients as the characteristic is not appropriate. Fortunately, from the probability point of view, the gray value inside the pothole is generally lower than the external grayscale, and the internal texture is generally rough. The goal is to integrate these two characteristics together. Firstly, the wavelet high-frequency coefficients are synthesized to obtain the wavelet coefficient modules, which are then averaged and divided by the grayscale of the pixel to enhance the internal texture features while weakening the external texture features. In this way, the texture and the grayscale information of the pothole are well integrated into the wavelet energy field, and the significance of the pothole region is highlighted. The wavelet energy field is obtained according to represents the wavelet coefficient module obtained by combining the horizontal, vertical, and diagonal details in the wavelet domain of the grayscale image , represents the average value of wavelet coefficient modules at different level of wavelet domain, represents the pixel value of the grayscale image, and represents the obtained wavelet energy field.
The main steps of constructing wavelet energy field are as follows.
Step 1. Convert RGB image to grayscale image.
Step 2. Apply median filtering for image enhancement.
Step 3. Carry out wavelet decomposition, calculate wavelet modules by high-frequency subband coefficients, and average the wavelet coefficient modules.
Step 4. Construct wavelet energy field through dividing the averaged wavelet coefficient module by the pixel value of the grayscale image.
Step 5. Normalize the energy field.
Taking the pothole in Figure 3(a)(1) as an example, its energy field constructed by the wavelet coefficients in the first, second, third, and fourth level of wavelet domain is shown in Figures 2(a), 2(b), 2(c), and 2(d) respectively. As we can see, Figure 2(a) mainly describes the edge information of the pothole, and Figure 2(b) is a more accurate description of the pothole region, while Figures 2(c) and 2(d) highlight not only the pothole region, but also its surrounding area. It can be concluded that the pothole region described by the energy field is gradually enlarging with the increase of wavelet decomposition level.
(a) First level
(b) Second level
(c) Third level
(d) Fourth level
(a) Original images
(b) Wavelet modules
(c) Wavelet energy fields
In order to improve the accuracy and stability of the pothole region represented by the energy field, the grayscale image is decomposed by three-level wavelet decomposition, and the high-frequency coefficients of the second level and the third level are taken to construct the wavelet energy field. The constructed energy field of different types of pavement conditions is shown in Figure 3. From Figures 3(1)–3(7), each figure represents pothole, nondefect pavement, cracks, patches, greasy dirt, shadows, and manhole cover, respectively. The left column (a) shows the original pavement images, the middle column (b) shows the averaged wavelet modules, and the right column (c) shows the corresponding wavelet energy fields. Comparing energy fields of these seven typical pavement conditions, it is found that the wavelet energy field of the pothole is distinct from the energy field of other pavement conditions and can highlight the significance of the pothole region to improve the accuracy of pothole detection and segmentation.
3.2. Markov Random Field Model
Although using wavelet energy fields and morphological processing can accurately detect the pothole, it is not recommended to segment the pothole with only the wavelet energy field model because the energy fields of some pavement images can only describe part region of the pothole. Therefore, a method of segmenting potholes by Markov random field is presented. Specifically, the original image is taken as the feature field, and the wavelet energy field is taken as the label field, which can improve the segmentation accuracy.
Use the Potts model to describe the label field , which can have two or more values. Considering the first-order neighborhood system and the second-order neighborhood system, the potential function is defined as is the direction parameter of the two-point potential regiment and and represent the label value of of the energy field.
By the local correlation of the Markov random field, the local probability of the label field model can be expressed as represents the number of neighborhood points whose label value is not equal to .
Regarding the value of image pixels as the characteristic, the original image constitutes a feature field. Assuming that the probability distribution of each pixel is independent with each other under the given label, the Gaussian model can be used to represent the feature field, and the local conditional probability is and , respectively, represent the mean and covariance matrices of the original image pixels labeled .
As mentioned above, taking the given original image as the feature field and the constructed wavelet energy field as the labeled field , the maximum a posteriori probability of Bayesian theory can be expressed as is the prior probability of the label field, is called the likelihood function, which is the probability description of the original image observed from the label field , and is the probability of the feature field. For a given pavement image, can be regarded as a constant because the pixel of the image is certain. Therefore,
According to formula (6), formula (5) can be expressed as
The objective function is gained by taking logarithm of formula (7), as is shown below:
Because of the equivalence of MRF and Gibbs distribution, can be expressed as
Therefore, under the MRF-MAP framework, the objective function (8) can be transformed into the sum of the energy of the label field and the feature field, which is demonstrated below: is the set of all the pixel locations. is the direction parameter of the two-point potential regiment. represents that the label value at position is . represents the number of pixels whose label value is not equal to in the neighborhood of position . and , respectively, represent the mean and covariance matrices of the original image pixels labeled .
The Iterated Conditional Modes Algorithm is used to solve the transformed objective function (10). The required initial segmentation is obtained by applying -means clustering algorithm to the wavelet energy field. For the sake of convenience, taking the original image as both the feature field and the label field is called method A, while taking the energy field as the label field and the original image as the feature field is called method B. In order to illustrate that method B is better than method A, this paper makes a comparative analysis of these two methods. As shown in Figure 4, column (a) represents two original pothole images, column (b) represents the segmentation results of method A, and column (c) represents the segmentation results of method B. On the one hand, it can be seen from the comparison of Figures 4(b)(1) and 4(c)(1) that method B can eliminate the influence of asphalt strips and some other noise outside the pothole. On the other hand, in Figure 4(b)(2), the lower end of the segmented pothole is not connected, while in Figure 4(c)(2), the lower end of the segmented pothole is closed, and the internal region has less holes, which indicate that method B can enhance the segmentation inside the pothole. In other words, taking wavelet energy field as the label field can not only reduce noise, but also enhance internal characteristics of the pothole and improve the segmentation accuracy.
(a) Original images
(b) Segmentation results of method A
(c) Segmentation results of method B
4. Methodology and Implementation
4.1. Detection of Potholes Based on Wavelet Energy Field
The methods proposed by Ryu et al. [19] and Koch and Brilakis [17] not only need to calculate the geometric criterions of the pothole region, but also need to extract the texture information to identify whether it is a pothole or not. But for the method proposed by this paper, it only need the geometric criterions to judge whether the pavement image has a pothole or not because the wavelet energy field combines the grayscale and texture information together. The processing steps are as follows.
Step 1 (simplify wavelet energy field). Considering coefficients of the wavelet energy field is between 0 and 1 after the normalization process of Section 3.1, the coefficients less than 0.3 is set to 0 to discard the redundancy coefficients.
Step 2. Apply Otsu method to perform binarization of the wavelet energy field.
Step 3 (perform morphological operations). Firstly, morphological open operation is utilized to eliminate small objects, separate objects connected by fine points, and smooth the boundaries between larger objects. Then, morphological expansion is carried out to fill the holes inside the white pixels.
Step 4 (extract the potential pothole region). After the morphological operations, the area of the pothole region should be the largest among all the connected regions. So, the potential pothole region can be extracted through comparing the area of each connected region.
Step 5 (pothole recognition by calculating geometric criterions). The geometric criterions include ellipticity , density C, and area of the potential pothole region, with which the potholes can be separated from small shadows, greasy dirt, and cracks. The ellipticity and density are defined as represents the major axis’s length of the ellipse that has the same standard second-order central moments with the potential pothole region, and meanwhile, represents the minor axis’s length of the ellipse and represents the area of the ellipse.
Then, select appropriate thresholds , , and for ellipticity rate , density , and area , respectively, and identify whether the extracted potential pothole region is really a pothole or not.
The pothole detection process is shown in Figure 5. Figure 5(a) is the original pothole image. Figure 5(b) is the constructed wavelet energy field, which emphasizes the pothole region. Figure 5(c) is the result of the simplified energy field that the redundancy coefficients are eliminated. Figure 5(d) shows the result of binarization, and Figure 5(e) shows the result of morphological operations, which further highlights the pothole region. Figure 5(f) is the extracted potential pothole region, and with the geometric criterions, it can be concluded whether it is a pothole or not.
(a) Original image
(b) Wavelet energy field
(c) Eliminate redundancy coefficients
(d) Binarization
(e) Morphological operations
(f) Potential pothole region
In order to illustrate the advantage of the wavelet energy field in extracting the potential pothole region, the initial extraction result of Figure 5(d) is compared with the extraction results using texture or grayscale information alone shown in Figures 6 and 7. Figure 6 is the initial extraction result using GLCM (Gray Level Cooccurrence Matrix) texture and FCM (Fuzzy c-means) algorithm. The shadow inside the pothole is not extracted and the rough texture outside the pothole is extracted as messy regions. Figure 7 is the binarization result of the grayscale image using Otsu method. Most of the regions with low grayscale level inside the pothole are not extracted. On the contrast, Figure 5(d) extracts most of the pothole region because the wavelet energy field integrates texture and gray information together.
4.2. Segmentation of the Detected Pothole
With the detection process of Section 4.1 based on wavelet energy field, the pothole image is detected, and it needs to be accurately segmented for pavement assessment. The process includes 5 steps.
Step 1. Use Markov random field model of Section 3.2 to segment the pothole. However, as is shown in Figure 4(c)(2), the segmentation result contains scattered spots and holes, which need to be further improved by morphological operations.
Step 2. Fill the internal small holes by morphological close operation whose structure element is a small disc with a radius of 3, and after that, extract the connected region with the largest area, which is exactly the pothole region.
Step 3. Apply morphology close operation again to fill the internal big holes, but the difference is that the structural element is a big disc with a radius of 18.
Step 4 (remove the sharp corners and thin sections). Firstly, apply morphological erosion whose structural element is a disc with a radius of 10, and extract the connected region with the largest area. Then, apply morphological dilation to restore the size of the pothole region before morphological erosion.
Step 5. Extract the outer edge of the pothole with canny operator.
Figure 8 shows the segmentation results of five different pavement potholes. Column (a) represents the original pothole images, among which, both the texture and gray features of the pothole shown in Figure 8(a)(1) are significant, but the pothole shown in Figure 8(a)(2) lacks texture information, the pothole shown in Figure 8(a)(3) lacks gray information, and even worse the pothole shown in Figure 8(a)(4) lacks both the grayscale and texture information, and there is water in the pothole shown in Figure 8(a)(5). Although the conditions of these pavement potholes are complex, all of the segmentation results are ideal. Column (b) represents the initial segmentation results of each corresponding pothole, column (c) represents the results after morphological processing, and column (d) represents the extracted edges of each pothole. It is worth noting that the water inside the pothole shown in Figure 8(a)(5) will cause the wavelet high-frequency coefficients very small and lead the energy field’s coefficients to almost zero, which is shown in Figure 8(b)(5). Eventually, the segmentation result of the pothole appears a large hole, but this could be solved by extracting the outer edge of the pothole, as is shown in Figures 8(c)(5) and 8(d)(5).
(a) Original images
(b) Initial segmentation results
(c) Morphological processing
(d) Edge extraction
5. Results
5.1. Pothole Detection Results
The proposed method in this paper was implemented in a MATLAB prototype. 120 pavement images were selected using Google search engine as the database for pothole detection, of which 30 images were used for manual training, and the remaining 90 images were used to test the performance of the method proposed by this paper. As Koch’s method, some premises are made for the sample set to improve the reliability of image processing. Firstly, the collected sample set of this paper properly represents a common population of pavement images, which contain several kinds of typical pavement distresses and other objects. Secondly, the image resolution is not at a low level. Thirdly, the resolution of each collected image is not much different from the others.
For the training phase, the threshold values of ellipticity , density , and area were selected based on the change of precision, recall, and accuracy.
Precision describes the detection exactness or fidelity, recall is a measure for detection completeness, and accuracy is used to describe the average correctness of a classification process. TP (true positives) represents correctly detected shapes, FP (false positives) represents wrongly detected shapes, TN (true negatives) represents correctly detected as false shapes, and FN (false negatives) represents wrongly detected as false shapes.
In order to select thresholds for ellipticity , density , and area , we performed a simple optimization strategy. First of all, was tested without taking the other two parameters into account, since we consider the area has the major impact on the results. Then, combining the selected threshold , was tested without taking into account, while was tested without taking into account. The statistical results were depicted in Figure 9.
(a) Thresholds
(b) Thresholds
(c) Thresholds
As is shown in Figure 9(a), recall, precision, and accuracy are at their maximum value when the threshold is between 15000 and 30000. In order to improve the possibility of detect small potholes correctly, the smallest value 15000 is regarded as the threshold of area . Figure 9(b) demonstrates that, with the increase of the threshold , recall has a downward trend, precision is on the rise, and accuracy is showing the trend of first rising and then falling. When is increased to 0.12, recall and accuracy will reach their maximum value, and the value of precision is at a satisfying level. If we continue to increase , the precision will become larger, but both recall and accuracy will decline. The reasonable value for is 0.12. Figure 9(c) indicates that recall, precision, and accuracy are at their best level when the threshold is 0.17. Table 1 shows the selected threshold values.
After the training phase, the remaining 90 pavement images were tested. Precision, recall, and accuracy were calculated to measure the performance of the method proposed by this paper. The statistical results of the performance were shown in Table 2.
From the statistical results in Table 2, it can be seen that most of the pothole images and nonpothole images can be distinguished correctly. The pothole detection method proposed by Ryu et al. [19] reached an overall accuracy of 73.5%, with 80.0% precision and 73.3% recall, and the resulting accuracy by Koch and Brilakis’s method [17] reached 86%, coming along with 82% precision and 86% recall, while the method proposed by this paper reaches an overall accuracy of 86.7%, with 83.3% precision and 87.5% recall, which demonstrates that the method of this paper has a better detection result.
However, false detection exists both in pothole images and nonpothole images. For the pothole, the brighter inside will weaken the value of its wavelet energy field, and the smooth inside means small wavelet high-frequency coefficients, which will also cause the wavelet energy field at a low value. These two situations will affect the pothole detection accuracy. For the nonpothole images, the value of wavelet energy field is large at the edge of shadows, cracks and lane-marking, and so forth. Mesh shadows, alligator cracks, and dense lane-markings are very likely to be wrongly detected as a pothole. Besides, the lighting conditions and blur images will also impact the pothole detection results. For example, the shadow inside the pothole could always help to strengthen the wavelet energy field, but to a small possibility, if the shadow is too dark to hide the texture feature, it will weaken the wavelet energy field and cause wrong detection results. As to the blur image, it will reduce the capability of detecting potholes correctly, which means reducing TP, but it will also reduce the possibility of wrongly detecting nonpothole images as potholes, which means reducing FP. The impaction of blur images needs to be further studied.
5.2. Pothole Segmentation Results
After pothole detection process of 90 pavement images, 35 of the 40 pothole images were successfully detected. Then, the segmentation process was implemented. The overlap degree between the extracted pothole region and the real pothole region was used to measure the performance of the segmentation results, which is shown in Table 3. The overlap degree of more than 0.95 accounts for 74.3% of the pavement pothole images and the overlap degree of more than 0.85 accounts for 88.6%. For the few remaining, its overlap degree is less than 0.85. There are some reasons for this, such that cracks on the pothole edge will result in excessive segmentation, and too much water or dust inside the pothole will result in insufficient segmentation. But on the whole, the segmentation results are ideal that most (88.6%) of the segmented potholes’ overlap degree is more than 0.85.
5.3. Comments on Processing Time
Table 4 shows the computing time of pothole detection and segmentation time of five pavement pothole images that are mentioned in Section 4.2. The mean time to construct the wavelet energy field is 2.31 s. Because the constructed wavelet energy field integrates the texture and gray information of the pothole together, it only needs to calculate the shape of the potential pothole region to determine whether it is a pothole or not. The detection process is simplified, and the average time consumed by the detection process is 0.46 s. Overall, it takes about 2.77 s from the beginning of the program to the finish of pothole detection, which indicates that the speed of pothole detection process is acceptable.
However, the Markov random field segmentation is time-consuming during optimizing the energy function. The pothole segmentation result is effective at accuracy, but it takes too much time that the average segmentation time reaches 25.27 s. In the future, the computing performance of Discrete Two-Dimensional Wavelet Transform and Markov random field model can be improved by using GPU (Graphics Processing Unit) hardware [24–26], which can not only make the pothole detection perform in real time, but also speed up the pothole segmentation process.
6. Conclusions
Considering the present 2D visual-based methods apply grayscale and texture information of the potholes separately in turn to detect the pothole, this paper proposed a method for asphalt pavement pothole detection and segmentation based on wavelet energy field, which integrates the gray and texture features together to highlight the pothole region.
Although the pavement images contain cracks, patches, greasy dirt, shadows, and manhole covers, the method proposed by this paper only needs geometric criterions to detect the pothole in the image. The performance is good that it reaches an overall accuracy of 86.7%, with 83.3% precision and 87.5% recall, which is better than the methods proposed by Ryu and Koch. As for the Markov random model used for pothole segmentation, taking wavelet energy field is better than taking the original image as the label field. The proposed method achieves good pothole segmentation results for different kinds of potholes that most (88.6%) of the segmented potholes’ overlap degree is more than 0.85.
However, there are some limitations in the proposed method. For example, a pothole with bright or smooth inside is very likely to be detected as nonpothole, and mesh shadows, alligator cracks, and dense lane-markings are very likely to be wrongly detected as a pothole. Besides, the lighting conditions and blur images will also impact the pothole detection results. In terms of computing time, the speed of pothole detection process is acceptable, but the pothole segmentation based on Markov random field is time-consuming.
In the future, we will study a more effective mathematic model to integrate wavelet high-frequency coefficients and pixel grayscale and take more visual characteristics to improve the accuracy. For the processing speed, we plan to apply GPU hardware [24–26] to make the pothole detection perform in real time and speed up the pothole segmentation process. And, we will also study the solutions to nonuniform lighting conditions and blur images.
Competing Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This research was supported by the grant from Central University Fund Project of Chang’an University (Mechanical Intelligent Controller [310825153313], Pothole Spray Injection Technology [310825165028], and Green Intelligent Pavement Maintenance Robot [310825173314]) funded by the Chinese Ministry of Education.