Classification of Asphalt Pavement Cracks Using Laplacian Pyramid-Based Image Processing and a Hybrid Computational Approach

Hoang, Nhat-Duc

doi:https://doi.org/10.1155/2018/1312787

Computational Intelligence and Neuroscience

On this page

Abstract Introduction Experimental Results Conclusion Data Availability Conflicts of Interest Supplementary Materials References Copyright Related Articles

Research Article | Open Access

Volume 2018 | Article ID 1312787 | https://doi.org/10.1155/2018/1312787

Classification of Asphalt Pavement Cracks Using Laplacian Pyramid-Based Image Processing and a Hybrid Computational Approach

Nhat-Duc Hoang¹

Guest Editor: Alvaro Rodriguez

Received22 Jun 2018

Accepted07 Sept 2018

Published01 Oct 2018

Abstract

To improve the efficiency of the periodic surveys of the asphalt pavement condition, this study puts forward an intelligent method for automating the classification of pavement crack patterns. The new approach relies on image processing techniques and computational intelligence algorithms. The image processing techniques of Laplacian pyramid and projection integral are employed to extract numerical features from digital images. Least squares support vector machine (LSSVM) and Differential Flower Pollination (DFP) are the two computational intelligence algorithms that are employed to construct the crack classification model based on the extracted features. LSSVM is employed for data classification. In addition, the model construction phase of LSSVM requires a proper setting of the regularization and kernel function parameters. This study relies on DFP to fine-tune these two parameters of LSSVM. A dataset consisting of 500 image samples and five class labels of alligator crack, diagonal crack, longitudinal crack, no crack, and transverse crack has been collected to train and verify the established approach. The experimental results show that the Laplacian pyramid is really helpful to enhance the pavement images and reveal the crack patterns. Moreover, the hybridization of LSSVM and DFP, named as DFP-LSSVM, used with the Laplacian pyramid at the level 4 can help us to achieve the highest classification accuracy rate of 93.04%. Thus, the new hybrid approach of DFP-LSSVM is a promising tool to assist transportation agencies in the task of pavement condition surveying.

1. Introduction

Asphalt pavement is an important part of the national transportation network. Surveying tasks performed by local transportation agencies are very crucial for collecting the current condition of asphalt pavement. Based on the collected pavement condition, road distresses can be detected, classified, evaluated, and documented for determining the maintenance methods and priority. Since transportation agencies around the world are struggling to allocate their limiting resources to a large number of deteriorating road sections, acquiring the condition of road pavement in a timely manner has become indispensible to correctly rank rehabilitation projects which are currently competing for budget.

According to a recent report of the Central Intelligence Agency, the total road network length of the world has reached the figure of 64,285,009 km. Particularly, in Vietnam, based on the statistics done by the General Statistics Office of Viet Nam 2010 [1], the total length of asphalt pavement road has amounted to 93,535 km. Needless to say, this large number of road sections necessitates a huge cost for the rehabilitation operations yearly and makes road maintenance an increasingly challenging task.

As pointed out by Al-Suleiman et al. [2], the aging and downgrading processes of pavements are inevitable and can be caused by surface fatigue, shear developing in the subgrade, subbase, base, or surface layers. Among the various types of pavement distress, cracking is one of the most easily identified forms of defect. In pavement maintenance practice, cracks are often considered as the most important indicator of pavement deterioration. It is because cracks are generally associated with the structural integrity of the asphalt pavements and directly affect their serviceability [3]. Thus, timely and correct identification of cracks in pavement structure is crucial for appraising the structure condition, establishing proper rehabilitation approaches, and recovering the acceptable quality of pavement structures.

In developing countries including Vietnam, visual inspection and manual data processing carried out by human inspectors are the prevailing method of pavement survey. Although this approach can deliver accurate assessments, it is only appropriate for evaluating a small number of road sections. It is because manual surveying is notorious for low productivity as well as inconsistent results due to subjective judgments of road inspectors [4]. Accordingly, methods for automating this tedious surveying works, and expediting the data processing tasks is an urgent demand of transportation agencies in many countries especially Vietnam. Such methods not only help us to improve the productivity but also help us to ensure the consistency of evaluation outcomes. The assessment results obtained quickly by these automated methods can significantly help us to allocate the government budget reasonably to competing pavement maintenance projects.

Due to the aforementioned motivations, various studies have recently proposed intelligent models for automating the pavement survey process. In these models, signals acquired by two-dimensional digital images are the main objects under analysis. The reason is that pavement cracks can be generally recognized via the pixel intensities and the shape of cracks. Nevertheless, previous works have demonstrated that image-based crack detection is not an easy task. It is because pavement images often have complex background texture and diverse patterns of cracks. The images also feature heterogeneous pixel intensity and suffer from inconsistency of illumination condition [4].

To cope with such challenges, a large number of studies have resorted to advanced image processing techniques to enhance and extract numerical features which are useful for crack recognition from the digital images. Mahler et al. [5], Kirschke, Velinsky [6], and Cheng et al. [7] established intelligent methods relied on the image intensity thresholding for recognizing cracks; these methods are based on the assumption that crack objects usually have lower intensity values than those of the regions in the background. A simplified method for crack category realization based on the concept of crack type index was proposed by Lee and Kim [8]; this approach is also based on image thresholding techniques, and information computed from neighboring pixels is used to distinguish the state of cracks from no cracks.

Besides the widely used image thresholding methods, the beamlet transform [9], predesigned image filtering [10], the Gabor filter [11], weighted neighborhood segmentation [12], wavelet-morphology-based detection [13], fuzzy Hough transform [14], steerable matched filtering [15], probabilistic generative model [4], and optimized minimal path selection [16] have been investigated by various scholars. Deep learning approaches [17, 18] which automate the feature extraction process have also been proposed. Nevertheless, these complex models are also computational intensive and require capable machine to perform the computing tasks. For the case of deep learning, a large number of image samples are needed to construct robust pavement classification models. Moreover, the training process of deep learning-based models is also very time-consuming.

Recent literature review has shown a notable trend of combining advanced image processing techniques and computational intelligence approaches. The computational intelligence models are then employed for crack detection and classification based on the features extracted by image processing techniques. These hybrid models are capable of both detecting the appearance of cracks and classifying cracks into predefined patterns.

A study that compared the performance of multilayer perceptron neural network, genetic algorithms, and self-organizing maps in pavement crack classification was accomplished by Rababaah [19]. Mokhtari et al. [20] relied on neural network models for pavement crack detection; this study found that neural network models demonstrated better prediction performances than those of other learning strategies including decision tree and k-nearest neighbor. Banharnsakun [21] combined the artificial bee colony and the artificial neural network for pavement surface distress categorization; the former computational method was employed in the phase of image segmentation; the image classification was performed by the latter approach. Cubero-Fernandez et al. [22] recently employed various image processing techniques of logarithmic transformation, bilateral filter, Canny algorithm, and morphological filter to extract features from digital images; the extracted features were then analyzed by a classification tree.

Fujita et al. [23] and Wang et al. [24] relied on the support vector machine to establish crack classification models. An intelligent method based on steerable filter, support vector machine, neural network, and random forest has been described in the study of Hoang and Nguyen [25]. Recent reviewing works [26, 27] pointed out an increasing trend of applying a hybridization of image processing and computational approaches in pavement crack detection and categorization. Due to the criticality of the problem at hand, investigating other image processing and computational intelligence methods to develop models for pavement crack classification can greatly contribute to the body of knowledge and provide practitioners with other capable alternatives. This study first proposes to employ the image processing technique of Laplacian pyramid in the phase of feature extraction. Laplacian pyramid, as a pyramid-based image process, can be highly useful for representing and analyzing images to facilitate the task of recognizing pavement cracks. Furthermore, a hybrid model of least squares support vector machine and Differential Flower Pollination is then used to construct a pavement crack classification model based on the numerical features derived from the Laplacian pyramid.

The rest of the study is organized as follows: Section 2 describes the research methodology, followed by the section of image acquisition. Section 3 depicts the structure of the proposed computational intelligence, followed by the report of experimental result (Section 4). The final section summarizes this research work with several conclusions.

2. Research Methodology

2.1. Image Processing Techniques

2.1.1. Laplacian Pyramid

The pyramid-based image process is a popular method for solving basic problems in image analysis or manipulation including data compression and shape analysis [28–30]. This method has recently attracted many scholars in solving various tasks that involve image processing such as automatic segmentation [31], texture classification [32, 33], nonlinear image registration [34], and remote sensing [35].

Essentially, the image pyramid is a data structure devised to facilitate efficient scaled convolution by means of decreased image representation. This method includes a sequence of versions of an original image in which both sample density and resolution are reduced in regular steps [36]. In the case of the Gaussian pyramid, the next pyramid level is attained by low-pass filtering and subsampling the previous pyramid with a factor of two. In general, the levels of the pyramid are obtained iteratively as follows [36]:where the original image is denoted as G₀. The next pyramid levels are denoted as G₁, G₂, …, G_N.

In the task of pavement crack recognition, bandpass-filtered images are highly useful for revealing the patterns of cracks. The bandpass-filtered images of pavement images can be computed by subtracting each Gaussian pyramid level from the next lower level in the pyramid. Since the two consecutive levels of pyramid are different in the density of the sample, it is required to perform an image expansion process for the next lower level of the pyramid as follows [36]:with G_l,0 = G_l and k > 0.

Accordingly, the next level of the bandpass pyramid (L₁, …, L_N) can be computed as follows:

It is noted that each value of this bandpass pyramid could be attained by convolving a difference of two Gaussians with the original image. This bandpass pyramid is very similar to the Laplacian operation. Thus, the bandpass pyramid can be regarded as the Laplacian pyramid. The responses of the the Laplacian pyramid applied for a pavement image are presented in Figure 1. It is proper to note that the Laplacian pyramid provides a complete image representation. It is because the steps employed to construct the pyramid can be performed in a reversed order to obtain the original image as follows [36]:

2.1.2. Projection Integrals

Projection integral (PI) is a widely employed method for in shape and texture classification [37]. This method has been shown to be highly applicable in the field of pavement defect detection and classification [22, 25, 38]. The first step of the PI approach is to convert an original RGB image into a gray-scale one. Based on the gray-scale image, the average value of the gray intensity at each location of the image along an axis is computed to obtain a projection integral (PI).

PIs along the horizontal and vertical axes are denoted as HPI and VPI, respectively. They are computed as follows:where and denote the horizontal and vertical PIs, respectively. and are the set of horizontal pixels at the vertical pixel y and the set of vertical pixels at the horizontal pixel x of an image , respectively.

Besides the two commonly employed HPI and VPI, the diagonal PI (DPI) can also be useful for pavement crack classification, especially in dealing with diagonal cracks. It is proper to note that there are two DPIs in each image. In order to calculate the two DPIs (denoted as DPI1 and DPI2), the output of the Laplacian pyramid operation is rotated with the angles of +45 and −45 to create two rotated maps of salient cracks. The DPI1 and DPI2 are attained by calculating the HPIs of the two rotated maps of salient cracks. Figure 2 provides illustrations of PIs of pavement images consisting of alligator cracks (AC), diagonal cracks (DC), longitudinal cracks (LC), no cracks (NC), and transverse cracks (TC).

2.2. Computational Intelligence Methods

2.2.1. Least Squares Support Vector Machine

LSSVM is a popular tool for pattern classification. First proposed in the previous work of Suykens and Vandewalle [39], this classifier is essentially a least squares version of the standard SVM established by Vapnik [40]. It is noted that LSSVM employs a nonlinear mapping function to deal with nonlinear classification problems. The data in the original input space are mapped to a high-dimensional feature space within which a linear classification model can be constructed.

Moreover, it is only necessary to compute the dot product of nonlinear mapping functions during the model construction and prediction phase of LSSVM. The result of this dot product of nonlinear mapping functions is the kernel function K(.). Thus, a LSSVM model can be considered as a variant of a neural network model with two layers [41]; the first layer maps the input data into a high-dimensional space and the second layer performs data classification.

LSSVM has demonstrated its superior capability in constructing data-driven models used in various engineering applications [42]. Besides the predictive performance, a notable advantage of LSSVM is its fast training process. It is because the LSSVM model can be trained via a process of solving a system of linear equations instead of the quadratic programming problem required by the standard SVM. The learning phase of LSSVM can be stated in the following constrained optimization problem:where denotes a vector perpendicular to the classification hyperplane, is the bias coefficient and represents an error variable at the kth data instance, denotes a regularization parameter which affects the process of penalizing the discrepancy between the observed and the predicted outputs.

The Lagrangian is applied to solve the above constrained optimization problem:where is the Lagrange multiplier and denotes a nonlinear mapping function.

Moreover, based on the KKT conditions for optimality, the aforementioned optimization described in Equation (7) can be converted into a linear system [43]. Finally, the LSSVM model used for pattern classification can be compactly written as follows:where and denote the solutions of the problem described in Equation (7). represents the kernel function.

The radial-basis function (RBF) kernel is commonly employed; this kernel function is described as follows:where denotes the kernel function parameter which influences the smoothness of the classification boundary.

2.2.2. Differential Flower Pollination (DFP)

As mentioned in the previous section, the model construction phase as well as the prediction phase of a LSSVM model requires the specification of the regularization parameter (γ) and the kernel function parameter (σ). The problem of model hyperparameter selection, also known as the model selection problem, can be formulated as an optimization problem [44]. Hence, this work employs DFP as a metaheuristic method for optimizing the LSSVM model performance with respect to the two parameters of γ and σ.

DFP [45] is a population-based metaheuristic method designed for dealing with complex optimization problems in continuous domains. It is noted that the selection of the LSSVM tuning parameters is a complex problem. The first reason is that it is virtually impossible to know the characteristics of the objective function of the optimization problem. The second reason is that the tuning parameters (γ and σ) are optimized in continuous domains. Hence, there is an infinite number of the candidate solutions. Therefore, DFP is a suitable approach to the problem of the LSSVM model optimization.

Notably, the DFP operation is a combination of two individual metaheuristic methods of the differential evolution (DE) proposed by Storn and Price [46] and the Flower Pollination Algorithm (FPA) proposed by Yang [47]. Thus, DFP inherits the exploitative capability of the DE-based mutation-crossover operators and the explorative capability of the FPA-based global pollination operator. Notably, the Levy flight-based global pollination in FPA significantly helps us to enhance the explorative search by utilizing large step sizes [48, 49].

At the first iteration, all members within the population are randomly generated in the feasible regions. After that, each member modifies its current location through either the global pollination operator or the local pollination operator. According to the suggestion of Yang [47], a selecting probability is employed to specify the frequencies of the global and local pollination operations.

The global pollination operator of DFP, inherited from the FPA algorithm, is described in the following equation:where is the index of the current iteration. denotes a trial solution.

The local pollination process of DFP which is similar to the mutation and crossover of the DE, creates the mutated and crossed solutions as follows:(i)Mutated solution:where r1, r2, and r3 denote three random indices used to select three members in the current population. is the parameter of a mutation-scale factor. is generated from a normal distribution with the mean = 0.5 and the standard deviation = 0.15 as suggested in previous works [45, 50].(ii)Crossed flower:where is the crossover probability which is fixed to be 0.8 as recommended by Price et al. [51].

2.3. The Collected Dataset of Pavement Images

Because LSSVM belongs to the category of a supervised learning approach, a dataset of pavement images with ground truth surface conditions must be prepared for training and validating phases. Thus, this research has carried out a survey along several road sections in Hai Chau and Thanh Khe districts, Da Nang city (Vietnam), to collect asphalt pavement images. The images are taken by the human inspector with a digital camera held at the distance of about 1.2 m above the road surface.

To accelerate the phases of data processing, feature extraction, and data classification, the size of each image sample has been fixed to be 200 × 200 pixels. Moreover, each image sample is associated with one of the five classes of pavement conditions: alligator crack (AC), diagonal crack (DC), longitudinal crack (LC), noncrack (NC), and transverse crack (TC). It is noted that each individual class of images has 100 samples; thus, the total number of data samples is 500. Figure 3 illustrates collected image samples. It is noted that the original pavement images have also been enhanced by the widely employed median filter with the window size of 5 × 5 pixels to remove dot noise of the pavement background.

3. The Proposed Approach for Classifying Pavement Cracks

This section describes the structure of the proposed model that combines image processing and computational intelligence for pavement crack classification. The model relies on the Laplacian pyramid to generate a salient map of pavement cracks. This map is then processed by the PI technique to create numerical features used by computational intelligence approaches of LSSVM and DFP. As aforementioned, this study resorts to the DFP algorithm to optimize the performance of the LSSVM classification model. In the subsequent parts of the paper, the DFP-optimized LSSVM is denoted as DFP-LSSVM. The overall picture of the proposed model is presented in Figure 4.

It is noted that the proposed pavement classification model is developed in the MATLAB environment with the help of the Image Processing Toolbox [52] and the Statistics and Machine Learning Toolbox [53]. The LSSVM classification model is implemented via the LS-SVMlab Toolbox developed by De Brabanter et al. [54]. The model basically consists of the following two modules:(i)Feature extraction, which is based on the Laplacian pyramid and PI techniques(ii)Data classification based on DFP-LSSVM

In the feature extraction phase, pavement images are first processed by the Laplacian pyramid to generate salient crack maps. The PI technique is then employed to compute the four PIs based on such maps. As aforementioned, the image size is 200 × 200 pixels. Therefore, the number of IP-based features created by the first Laplacian pyramid is 200 × 4 = 800. The number of IP-based features created by the next level of the Laplacian pyramid is equal to the number of IP-based features generated by the previous level divided by two. Therefore, the number of features in the second level is 400; the number of features in the third level is 200, and so on. Figure 5 illustrates PIs of images corresponding to different Laplacian pyramid levels.

(a)

(b)

(c)

(d)

(e)

After the feature extraction phase, the numerical dataset used for pavement crack classification is constructed. To assess the predictive capability of DFP-LSSVM, the numerical dataset is divided into two different sets: training set (80%) and validating set (20%). In addition, the input variables of the dataset, which are the PI-based features, have been normalized by the Z-score transformation. This step aims at standardizing the ranges of variables. The equation of the Z-score data transformation is shown as follows:where and represent the normalized and the original features, respectively. and denote the mean value and the standard deviation of the original features, respectively.

Since the performance of LSSVM is strongly influenced by its parameters of γ and σ, this study relies on DFP to fine-tune these two parameters to achieve the most desired prediction accuracy. At the first iteration (), the two hyperparameters (γ and σ) of LSSVM are randomly created within the feasible regions according to the following equation:where represents the tuning parameter (either γ or σ) at the first iteration. denotes a uniform random number generated within the range of 0 and 1. The and of γ are set to be 0.01 and 1000; the and of σ are selected to be 0.01 and 10.

To identify the most suitable values of γ or σ, the training set is further divided into subset 1 and subset 2. The first subset is used for model construction, and the second set is used for model validating. These two subsets are only employed for optimizing the LSSVM’s parameters. Accordingly, the following objective cost function is used within the DFP algorithm:where denotes the classification accuracy rate (CAR) of LSSVM when this classifier predicts data instances in the cth class.

CAR used to express the model predictive capability is calculated as follows:where and denote the number of correctly classified data instances and the total number of data instances, respectively.

The inclusion of prediction results of the subset 1 and subset 2 in the cost function is to guide the population members of DFP to maximize the prediction accuracy of the LSSVM for all the five classes of interest (AC, DC, LC, NC, and TC). The involvement of the subset 2 as validating data is to alleviate model overfitting. Overfitting happens when the LSSVM model classifies the image samples in the training set well, but it classifies the image samples outside the training set incorrectly. Thus, to reduce the effect of this undesired phenomenon, it is beneficial to acquire the LSSVM model that has high CAR in both training set and validating phases. Moreover, it is noted that the LSSVM model has been equipped with the one-versus-one (OvO) strategy [55] to deal with the multilabel data classification. It is because the OvO strategy can help us to achieve good prediction results and also can help us to avoid the problem of imbalanced data classification [56].

During the optimization process, DFP carries out the optimization process until a sufficient number of searching iteration is reached. The maximum number of iteration is fixed to be 100 in this study. When the optimization process terminates, the optimized LSSVM model with the fine-tuned hyperparameters of γ and σ is ready for classifying the image samples stored in the testing dataset.

4. Experimental Results

As stated previously, the dataset including 500 image samples is used to create and verify the performance of the six machine learning models. The dataset is divided into a training set (80%) and a testing set (20%). The first set is employed in the model construction phase; the second set is used to demonstrate the model generalization capability. Since a single run may not reflect the true performance of each machine learning approach due to the randomness in the data selection process, this study repetitively performs the training and testing processes 30 times. The model performance is then evaluated by averaging the outcomes obtained from the 30 times of training and testing data samplings.

To demonstrate the capability of DFP-LSSVM, its performance is compared to those of the classification tree (CTree) [57], linear discriminant analysis (LDA) [58], naïve Bayesian classifier (NBC) [58], and backpropagation artificial neural network (BPANN) [59]. The CTree, LDA, NBC, and BPANN models are implemented in the MATLAB environment via the Statistics and Machine Learning Toolbox [53]. To employ the CTree, LDA, NBC, and BPANN models, it is required to specify their hyperparameters. In this section, the hyperparameters that lead to the best validating performance of models are selected. In the case of the CTree model, the minimal number of observations per tree leaf is chosen to be one as suggested by the MATLAB toolbox [53]. It is proper to note that for guaranteeing a fair comparison, the CTree, NBC, and LDA are also employed with the (OvO) strategy to deal with the multiclass classification problem at hand.

Furthermore, the number of neuron in the hidden layer (denoted as Nr) is an important parameter to be set. Based on the suggestion of Heaton [60], Nr is allowed to vary from 2/3D + O to 1.5D (where D denotes the number of input variables and O is the number of the output class). In this study, particularly for the datasets constructed from the Laplacian pyramid with levels of 1 and 2, the numbers of features are large. The numbers of features for the Laplacian pyramid with levels of 1 and 2 are 800 and 400, respectively. By several trial and error runs, it is found that the BPANN models with Nr > 300 require long training times and have poor classification performances. Therefore, in these cases, the values of Nr ≤ 300 are investigated to identify the most appropriate BPANN model structure. Moreover, the scaled conjugate gradient algorithm with the maximum number of training epochs = 3000 is employed to construct the BPANN model.

Besides the CAR for each individual class labels, the overall classification accuracy rate (CAR_Overall) for all the five class labels is calculated by the following equation:

Since the Laplacian pyramid is a crucial step in the feature extraction phase, this study has investigated the performances of DFP-LSSVM and other benchmark models according to different levels of the Laplacian pyramid. It is noted that the Laplacian pyramid does not only highlight the crack patterns by creating a salient crack map but also serves as a means of dimension reduction. It is because this technique helps us to reduce the image size and leads to fewer PI-based features. The outcomes of experiments are graphically presented in Figure 6. It can be seen that the increase in the Laplacian pyramid’s level imposes positive effects on the model performances. Evidently, CTree, LDA, and NBC achieve the highest overall CAR when the Laplacian pyramid’s level = 5. The BPANN and DFP-LSSVM attain the best overall CAR when the Laplacian pyramid’s level = 4. A notable observation is that when the Laplacian pyramid’s level >5, the model performances significantly drop. It is an understandable phenomenon because high level of the Laplacian pyramid leads to lower image resolutions and causes significant losses of information. If the Laplacian pyramid’s level = 6, the information presented in the images is not sufficient for the machine to construct classifiers capable of making accuracy crack pattern recognition.

(a)

(b)

(c)

(d)

(e)

The details of the model prediction results obtained from the repeated random subsampling process with 20 runs are summarized in Table 1. The CAR of all individual classes (AC, DC, LC, NC, and TC) and the overall CAR are reported with their mean and standard deviation values. It can be observed that DFP-LSSVM has achieved the highest overall CAR of 93.40%, followed by BPANN (87.25%), NBC (82.30%), LDA (81.75%), and CTree (76.45%). The hybrid computational intelligence model of DFP-LSSVM also has the most accurate outcomes in each individual class of AC (91.50%), DC (97.50%), LC (99.25%), NC (81.00%), and TC (97.75%).

In addition, Figure 7 presents the box plots of prediction results of the DFP-LSSVM, BPANN, NBC, LDA, and CTree. To confirm the statistical difference of the model performances, this study relies on the Wilcoxon signed-rank test (WSRT). WSRT is a popular nonparametric statistical hypothesis test which is often used for result comparison [61]. In this study, the significance level of WSRT is chosen to be 0.05. If the value computed from the test is smaller than 0.05, it is able to confirm that the pavement crack classification results of the employed models are statistically different. The results of WSRTs are reported in Table 2. With value < 0.05, it is confident to state that the performance of DFP-LSSVM is significantly better than those of other benchmark models.

In addition, to confirm that the Laplacian pyramid at the level of 4 is the most appropriate level for the feature extraction phase of the DFP-LSSVM model, and the prediction outcomes of the levels of 3, 4, and 5 are compared. The results of the repeated subsampling obtained from these three levels are illustrated by the box plots in Figure 8. WSRT is also used to inspect the statistical significance of the model performances. The results of WSRT are reported in Table 3. Observably, the values computed from the test indicate that the Laplacian pyramid at the level of 4 is significantly better than that at the levels of 3 and 5. Based on the experimental results and the employed statistical test, the DFP-LSSVM that uses the Laplacian pyramid at the level of 4 deems best suited for the collected dataset at hand.

Hence, the overall finding of this study is that the Laplacian pyramid can be an effective method for achieving a high CAR of pavement crack detection and categorization. The model accuracy of 93.40% is significantly better than the previously constructed models that employ the steerable filtering method. The steerable filtering-based models proposed in [22] and [25] achieved the average CAR of 80.25% and 87.50%, respectively. The model based on steerable filters and support vector machine described in [62] demonstrated good predictive performance (CAR is about 96%); however, this model was incapable of recognizing diagonal cracks. Image processing-based crack recognition approaches based on Gabor filter [11] and steerable matched filtering [15] also demonstrated promising results with the precision being up to 95% and 93%, respectively; however, this approach is only appropriate for assigning crack or noncrack labels to image samples.

A novel method based on the beamlet transform technique has been proposed in [9]; nevertheless, this method was unable to recognize diagonal and alligator cracks. Moreover, intelligent models based on convolution neural networks (CNNs) established in [17, 18, 63] have significant advantages over the models based on image processing algorithms (e.g., Laplacian pyramid and steerable filters); that is, the feature extraction and data classification can be integrated and performed autonomously. These models based on CNNs also attained positive classification accuracy which can be up to 94%; however, they have rarely been performed in multiclass pavement recognition. In the aspect of data classification based on machine learning, the previous works [20, 64–66] have pointed out the appropriateness of neural networks for pavement crack detection. However, this current study has experimentally shown that LSSVM can be more capable than neural networks in the task of interest.

5. Conclusion

This study constructs an image processing-based method for classifying pavement crack patterns. The Laplacian pyramid and the PI technique are used to extract numerical features from pavement images. The Laplacian pyramid generates a salient crack map which highlights the patterns of cracks. The PI technique produces the features of the four PIs including HPI, VPI, and two DPIs. These PIs are specifically designed to recognize the four patterns of cracks (AC, DC, LC, and TC) as well as the state of intact pavement (NC). A prediction model that combines the LSSVM and DFP algorithms has been constructed. LSSVM is used to generalize classification boundaries that separate the learning space into five subspaces of AC, DC, LC, NC, and TC. DFP acts as an optimization tool for assisting the model construction phase of LSSVM.

A dataset of 500 images has been collected to train and test the proposed methods of feature extraction and classification. Notably, since the image size processed by the Laplacian pyramid is reduced after each level of analysis, the Laplacian pyramid also helps us to alleviate the computational burden of DFP-LSSVM by decreasing the size of the feature set. Experiments relied on a random subsampling process, and WSRT confirms that DFP-LSSVM used with the Laplacian pyramid at the level 4 can deliver the most desired prediction outcome with the overall CAR = 93.4%. Thus, the newly established model can be a promising tool to ease the labor-intensive periodic pavement survey. The future development of the current work may include the employment of other advanced image processing techniques in the tasks of crack identification. Additionally, comparing different methods used for crack analysis and feature extraction can also be a worth investigating research direction. It is also desirable to investigate other effective classifiers in the task of crack pattern recognition as well as to expand the current dataset by collected more types of cracks (e.g., reflective cracks).

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The author confirms that there are no conflicts of interest regarding the publication of this manuscript.

Supplementary Materials

The supplementary file contains the dataset used in this study. In this file, the first 100 columns are the input features of the data (which are the projection integrals); the last column is the class labels (1 = alligator crack, 2 = diagonal crack, 3 = longitudinal crack, 4 = noncrack, and 5 = transverse crack). (Supplementary Materials)

References

GSO, Transport, Postal Services and Telecommunications, Some Main Indicators on Capacity of Transportation, General Statistics Office of Viet Nam, Hanoi, Vietnam, 2010, http://www.gso.gov.vn/default_en.aspx?tabid=473&idmid=&ItemID=12788.
T. I. Al-Suleiman, Z. M. Hamici, S. M. Bazlamit, and H. S. Ahmad, “Assessment of the effect of alligator cracking on pavement condition using WSN-image processing,” in Proceedings of 8th International Conference on Engineering, Project, and Product Management (EPPM 2017), pp. 265–274, Springer International Publishing, Cham, Switzerland, March 2018.
View at: Publisher Site | Google Scholar
W. Wang, A. Zhang, K. C. P. Wang, A. F. Braham, and S. Qiu, “Pavement crack width measurement based on Laplace’s equation for continuity and unambiguity,” Computer-Aided Civil and Infrastructure Engineering, vol. 33, no. 2, pp. 110–123, 2018.
View at: Publisher Site | Google Scholar
D. Ai, G. Jiang, S. K. Lam, and C. Li, “Automatic pixel-level pavement crack detection using information of multi-scale neighborhoods,” IEEE Access, vol. 6, pp. 24452–24463, 2018.
View at: Publisher Site | Google Scholar
D. S. Mahler, Z. B. Kharoufa, E. K. Wong, and L. G. Shaw, “Pavement distress analysis using image processing techniques,” Computer-Aided Civil and Infrastructure Engineering, vol. 6, no. 1, pp. 1–14, 1991.
View at: Publisher Site | Google Scholar
K. R. Kirschke and S. A. Velinsky, “Histogram-based approach for automated pavement-crack sensing,” Journal of Transportation Engineering, vol. 118, no. 5, pp. 700–710, 1992.
View at: Publisher Site | Google Scholar
H. D. Cheng, X. J. Shi, and C. Glazier, “Real-time image thresholding based on sample space reduction and interpolation approach,” Journal of Computing in Civil Engineering, vol. 17, no. 4, pp. 264–272, 2003.
View at: Publisher Site | Google Scholar
H. Lee and J. Kim, “Development of a crack type index,” Transportation Research Record: Journal of the Transportation Research Board, vol. 1940, pp. 99–109, 2005.
View at: Publisher Site | Google Scholar
L. Ying and E. Salari, “Beamlet Transform-based technique for pavement crack detection and classification,” Computer-Aided Civil and Infrastructure Engineering, vol. 25, no. 8, pp. 572–580, 2010.
View at: Publisher Site | Google Scholar
A. Zhang, Q. Li, K. Wang, and S. Qiu, “Matched filtering algorithm for pavement cracking detection,” Transportation Research Record: Journal of the Transportation Research Board, vol. 2367, no. 1, pp. 30–42, 2013.
View at: Publisher Site | Google Scholar
M. Salman, S. Mathavan, K. Kamal, and M. Rahman, “Pavement crack detection using the Gabor filter,” in Proceedings of 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013), pp. 2039–2044, The Hague, Netherlands, October 2013.
View at: Publisher Site | Google Scholar
L. Sun, M. Kamaliardakani, and Y. Zhang, “Weighted neighborhood pixels segmentation method for automated detection of cracks on pavement surface images,” Journal of Computing in Civil Engineering, vol. 30, no. 2, Article ID 04015021, 2016.
View at: Publisher Site | Google Scholar
Y. O. Ouma and M. Hahn, “Wavelet-morphology based detection of incipient linear cracks in asphalt pavements from RGB camera imagery and classification using circular Radon transform,” Advanced Engineering Informatics, vol. 30, no. 3, pp. 481–499, 2016.
View at: Publisher Site | Google Scholar
S. Mathavan, K. Vaheesan, A. Kumar et al., “Detection of pavement cracks using tiled fuzzy Hough transform,” Journal of Electronic Imaging, vol. 26, no. 5, p. 1, 2017.
View at: Google Scholar
S. Li, Y. Cao, and H. Cai, “Automatic pavement-crack detection and segmentation based on steerable matched filtering and an active contour model,” Journal of Computing in Civil Engineering, vol. 31, no. 5, Article ID 04017045, 2017.
View at: Publisher Site | Google Scholar
W. Kaddah, M. Elbouz, Y. Ouerhani, V. Baltazart, M. Desthieux, and A. Alfalou, “Optimized minimal path selection (OMPS) method for automatic and unsupervised crack segmentation within two-dimensional pavement images,” The Visual Computer, 2018.
View at: Publisher Site | Google Scholar
Z. Tong, J. Gao, Z. Han, and Z. Wang, “Recognition of asphalt pavement crack length using deep convolutional neural networks,” Road Materials and Pavement Design, vol. 19, no. 6, pp. 1334–1349, 2018.
View at: Publisher Site | Google Scholar
N. D. Hoang, Q. L. Nguyen, and V. D. Tran, “Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network,” Automation in Construction, vol. 94, pp. 203–213, 2018.
View at: Publisher Site | Google Scholar
H. Rababaah, “Asphalt pavement crack classification: a comparative study of three AI approaches: multilayer perceptron, genetic algorithms, and self-organizing maps,” Indiana University South Bend, South Bend, IN, USA, 2005, M.S. Thesis.
View at: Google Scholar
S. Mokhtari, L. Wu, and H. B. Yun, “Comparison of supervised classification techniques for vision-based pavement crack detection,” Transportation Research Record: Journal of the Transportation Research Board, vol. 2595, pp. 119–127, 2016.
View at: Publisher Site | Google Scholar
A. Banharnsakun, “Hybrid ABC-ANN for pavement surface distress detection and classification,” International Journal of Machine Learning and Cybernetics, vol. 8, no. 2, pp. 699–710, 2017.
View at: Publisher Site | Google Scholar
A. Cubero-Fernandez, F. J. Rodriguez-Lozano, R. Villatoro, J. Olivares, and J. M. Palomares, “Efficient pavement crack detection and classification,” EURASIP Journal on Image and Video Processing, vol. 2017, no. 1, 39 pages, 2017.
View at: Publisher Site | Google Scholar
Y. Fujita, K. Shimada, M. Ichihara, and Y. Hamamoto, “A method based on machine learning using hand-crafted features for crack detection from asphalt pavement surface images,” in Proceedings of International Conference on Quality Control by Artificial Vision 2017, p. 8, SPIE, Tokyo, Japan, May 2017.
View at: Google Scholar
S. Wang, S. Qiu, W. Wang, D. Xiao, and K. C. P. Wang, “Cracking classification using minimum rectangular cover–based support vector machine,” Journal of Computing in Civil Engineering, vol. 31, no. 5, Article ID 04017027, 2017.
View at: Publisher Site | Google Scholar
N. D. Hoang and Q. L. Nguyen, “A novel method for asphalt pavement crack classification based on image processing and machine learning,” Engineering with Computers, 2018.
View at: Publisher Site | Google Scholar
H. Zakeri, F. M. Nejad, and A. Fahimifar, “Image based techniques for crack detection, classification and quantification in asphalt pavement: a review,” Archives of Computational Methods in Engineering, vol. 24, no. 4, pp. 935–977, 2017.
View at: Publisher Site | Google Scholar
T. B. J. Coenen and A. Golroo, “A review on automated pavement distress detection methods,” Cogent Engineering, vol. 4, no. 1, 2017.
View at: Publisher Site | Google Scholar
H. J. Antonisse, “Image segmentation in pyramids,” Computer Graphics and Image Processing, vol. 19, no. 4, pp. 367–383, 1982.
View at: Publisher Site | Google Scholar
J. L. Crowley and A. C. Parker, “A representation for shape based on peaks and ridges in the difference of low-pass transform,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 6, no. 2, pp. 156–170, 1984.
View at: Publisher Site | Google Scholar
D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.
View at: Publisher Site | Google Scholar
A. Sarkar and S. Padmavathi, “Image pyramid for automatic segmentation of fabric defects,” in Proceedings of IEEE International Conference On Computational Vision and Bio Inspired Computing, pp. 569–578, Springer International Publishing, Coimbatore, India, September 2018.
View at: Google Scholar
A. R. Yadav, R. S. Anand, M. L. Dewal, and S. Gupta, “Gaussian image pyramid based texture features for classification of microscopic images of hardwood species,” Optik-International Journal for Light and Electron Optics, vol. 126, no. 24, pp. 5570–5578, 2015.
View at: Publisher Site | Google Scholar
S. Dash and U. R. Jena, “Gaussian pyramid based laws' mask descriptor for texture classification,” in Proceedings of International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), pp. 654–658, Chennai, India, March 2017.
View at: Publisher Site | Google Scholar
L. Gan and G. Agam, “Gradient enhanced image pyramid for improved nonlinear image registration,” Electronic Imaging, vol. 2016, no. 19, pp. 1–8, 2016.
View at: Publisher Site | Google Scholar
B. Zou, J. Sun, Y. Jin, and Y. Cheng, “Large area land cover mapping based on pyramid transformation with high-resolution PolSAR image,” in Proceedings of 2016 IEEE Radar Conference (RadarConf), pp. 1–4, Philadelphia, PA, USA, May 2016.
View at: Publisher Site | Google Scholar
E. H. Adelson, C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden, “Pyramid methods in image processing,” RCA Engineer, vol. 29, no. 6, pp. 33–41, 1984.
View at: Google Scholar
A. Chouchane, M. Belahcene, and S. Bourennane, “3D and 2D face recognition using integral projection curves based depth and intensity images,” International Journal of Intelligent Systems Technologies and Applications, vol. 14, no. 1, pp. 50–69, 2015.
View at: Publisher Site | Google Scholar
N.-D. Hoang, “An artificial intelligence method for asphalt pavement pothole detection using least squares support vector machine and neural network with steerable filter-based feature extraction,” Advances in Civil Engineering, vol. 2018, Article ID 7419058, 12 pages, 2018.
View at: Publisher Site | Google Scholar
J. A. K. Suykens and J. Vandewalle, “Least squares support vector machine classifiers,” Neural Processing Letters, vol. 9, no. 3, pp. 293–300, 1999.
View at: Publisher Site | Google Scholar
V. N. Vapnik, Statistical Learning Theory, John Wiley & Sons, Hoboken, NJ, USA, 1998, ISBN-10: 0471030031.
J. A. K. Suykens, I. Horvath, S. Basu, C. Micchelli, and J. Vandewalle, Advances in Learning Theory: Methods, Models and Applications, IOS Press, Amsterdam, Netherlands, 2003.
Y.-H. Wu and H. Shen, “Grey-related least squares support vector machine optimization model and its application in predicting natural gas consumption demand,” Journal of Computational and Applied Mathematics, vol. 338, pp. 212–220, 2018.
View at: Publisher Site | Google Scholar
J. Suykens, J. V. Gestel, J. D. Brabanter, B. D. Moor, and J. Vandewalle, Least Square Support Vector Machines, World Scientific Publishing Co. Pte. Ltd., Singapore, 2002, ISBN-13: 978-9812381514.
D. Prayogo and Y. T. T. Susanto, “Optimizing the prediction accuracy of friction capacity of driven piles in cohesive soil using a novel self-tuning least squares support vector machine,” Advances in Civil Engineering, vol. 2018, Article ID 6490169, 9 pages, 2018.
View at: Publisher Site | Google Scholar
N.-D. Hoang, D. Tien Bui, and K. W. Liao, “Groutability estimation of grouting processes with cement grouts using differential flower pollination optimized support vector machine,” Applied Soft Computing, vol. 45, pp. 173–186, 2016.
View at: Publisher Site | Google Scholar
R. Storn and K. Price, “Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces,” Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, 1997.
View at: Publisher Site | Google Scholar
X. S. Yang, “Flower pollination algorithm for global optimization,” in Unconventional Computation and Natural Computation: Lecture Notes in Computer Science, J. Durand-Lose and N. Jonoska, Eds., vol. 7445, pp. 240–249, Springer Berlin Heidelberg, Heidelberg, Germany, 2012.
View at: Publisher Site | Google Scholar
W. Cui and Y. He, “Biological flower pollination algorithm with orthogonal learning strategy and catfish effect mechanism for global optimization problems,” Mathematical Problems in Engineering, vol. 2018, Article ID 6906295, 16 pages, 2018.
View at: Publisher Site | Google Scholar
Y. Zhou, S. Zhang, Q. Luo, and C. Wen, “Using flower pollination algorithm and atomic potential function for shape matching,” Neural Computing and Applications, vol. 29, no. 6, pp. 21–40, 2018.
View at: Publisher Site | Google Scholar
J. Zhang and A. C. Sanderson, “JADE: adaptive differential evolution with optional external archive,” IEEE Transactions on Evolutionary Computation, vol. 13, no. 5, pp. 945–958, 2009.
View at: Publisher Site | Google Scholar
K. V. Price, R. M. Storn, and J. A. Lampinen, Differential Evolution A Practical Approach to Global Optimization, Springer-Verlag, Berlin, Germany, 2005.
Matwork, Image Processing Toolbox User’s Guide, MathWorks, Inc., Natick, MA, USA, 2017, https://www.mathworks.com/help/pdf_doc/images/images_tb.pdf.
Matwork, Statistics and Machine Learning Toolbox User’s Guide, MathWork Inc., Natick, MA, USA, 2017, https://www.mathworks.com/help/pdf_doc/stats/stats.pdf.
K. De Brabanter, P. Karsmakers, F. Ojeda et al., LS-SVMlab Toolbox User's Guide version 1.8, 2010.
A. Rocha and S. K. Goldenstein, “Multiclass from binary: expanding one-versus-all, one-versus-one and ECOC-based approaches,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 2, pp. 289–302, 2014.
View at: Publisher Site | Google Scholar
C. M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics), Springer, Berlin, Germany, 2011, ISBN-10: 0387310738.
L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classifcation and Regression Trees, Wadsworth and Brooks, Montery, Calif, USA, 1984, ISBN-13: 978-0412048418.
R. Duda, P. Hart, and D. Stock, Pattern Classification, John Wiley & Sons, 2001, ISBN-13: 978-0471056690.
M. T. Hagan, H. B. Demuth, M. H. Beale, and O. D. Jesús, Neural Network Design, Martin Hagan, 2 edition, 2014, ISBN-10: 0971732116.
J. Heaton, Introduction to Neural Networks for C#, Heaton Research, Inc., Chesterfield, MI, United States, 2008.
N. T. Nguyen, B. T. N. Dang, X. C. Pham et al., “Spatial pattern assessment of tropical forest fire danger at Thuan Chau area (Vietnam) using GIS-based advanced machine learning algorithms: a comparative study,” Ecological Informatics, 2018.
View at: Publisher Site | Google Scholar
N. D. Hoang, Q. L. Nguyen, and D. T. Bui, “Image processing-based classification of asphalt pavement cracks using support vector machine optimized by artificial bee colony,” Journal of Computing in Civil Engineering, vol. 32, no. 5, Article ID 04018037, 2018.
View at: Publisher Site | Google Scholar
L. Zhang, F. Yang, Y. D. Zhang, and Y. J. Zhu, “Road crack detection using deep convolutional neural network,” in Proceedings of 2016 IEEE International Conference on Image Processing (ICIP), pp. 3708–3712, Phoenix, AZ, USA, September 2016.
View at: Publisher Site | Google Scholar
M. S. Kaseko, Z. P. Lo, and S. G. Ritchie, “Comparison of traditional and neural classifiers for pavement-crack detection,” Journal of Transportation Engineering, vol. 120, no. 4, pp. 552–569, 1994.
View at: Publisher Site | Google Scholar
L. Li, L. Sun, G. Ning, and S. Tan, “Automatic pavement crack recognition based on BP neural network,” Promet-Traffic and Transportation, vol. 26, no. 1, pp. 11–22, 2014.
View at: Publisher Site | Google Scholar
H. Cheng, J. Wang, Y. Hu, C. Glazier, X. Shi, and X. Chen, “Novel approach to pavement cracking detection based on neural network,” Transportation Research Record: Journal of the Transportation Research Board, vol. 1764, pp. 119–127, 2001.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2018 Nhat-Duc Hoang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1147

Downloads

1246

Citations