Abstract

Adverse road condition is the main cause of traffic accidents. Road surface condition recognition based on video image has become a central issue. However, hybrid road surface and road surface under different lighting environments are two crucial problems. In this paper, the road surface states are categorized into 5 types including dry, wet, snow, ice, and water. Then, according to the original image size, images are segmented; 9-dimensional color eigenvectors and 4 texture eigenvectors are extracted to construct road surface state characteristics database. Next, a recognition method of road surface state based on SVM (Support Vector Machine) is proposed. In order to improve the recognition accuracy and the universality, a grid searching algorithm and PSO (Particle Swarm Optimization) algorithm are used to optimize the kernel function factor and penalty factor of SVM. Finally, a large number of actual road surface images in different environments are tested. The results show that the method based on SVM and image segmentation is feasible. The accuracy of PSO algorithm is more than 90%, which effectively solves the problem of road surface state recognition under the condition of hybrid or different video scenes.

1. Introduction

According to statistics, 16.12% of traffic accidents on the highway are ascribed to slippery road conditions [1] since 2007 in China. By analysis of accidents’ characteristics, it can be concluded that the traffic accident rate increases under the water, snow, ice, and freezing road surface conditions and that road surface conditions greatly affect the highway traffic safety and transport efficiency. Therefore, it is urgent to carry out research on the road surface state recognition and provide reference and theoretical basis for traffic control and meteorological management to ensure traffic safety [2].

In the field of traffic meteorology, the road surface state can be categorized in dry, wet, water, snow, and ice types according to different forms of liquid on road surface. At present, the road surface detection sensor is the main entrance to obtain the information of road surface slippery conditions. Cai et al. [3] used underground embedded road surface condition detector to realize the recognition. Gailius and Jacenas [4] collected the frictional noise between the tire and the road surface and obtained the road surface characteristics based on the noise spectrum analysis method. Qi et al. [5] extracted road surface characteristics and anti-hold-process parameters, according to the principle of maximum proximity to identify the state of the road surface. Alonso et al. [6] proposed a real-time acoustics road surface state recognition system based on tire-road noise and used the noise measurement system and the signal processing algorithm for road surface state classification, and finally, precise classification of dry and wet road state was realized. Wang et al. [7] proposed D-S evidence theory and artificial neural network method for recognition and prediction of traffic state level under adverse weather conditions. However, the road surface sensor can only obtain the information of the section and the maintenance is extremely inconvenient; hence the actual effect is not ideal.

With the widespread application of road surveillance cameras, more and more scholars pay attention to the image processing technology of road surface slippery condition recognition. Andreas and Wilco [8] extracted the gray scale feature of road surface image and designed the neural network classifier for road surface state recognition. Anis et al. [9] analyzed the set and spectrum of road surface reflection image and described the correlation between the surface texture and the friction coefficient, by which the reflection image of designated location can be monitored and identified. Chen [10] extracted low-order statistical features of road surface images including gray level cooccurrence matrix texture feature parameters and used linear discriminant function to determine the road surface state. Ueda et al. [11] and Yoda et al. [12] measured and analyzed the road surface roughness, the proportion of low-frequency reflection signal components and high-frequency components, and the average reflection intensity to determine the state of the road surface based on CCD camera technology. Yamamoto et al. [13] applied the human-computer interaction method to extract the gray scale value and temperature characteristics parameters of the road surface for the road surface state prediction, and measurement accuracy was tested to be more than 80%. Muneo et al. [14] used vehicle camera to collect traffic information; the parameters of road surface image polarization characteristics were utilized to establish the road slippery condition evaluation model. Becchi et al. [15] obtained the water condition video of the road surface; the rain density judge values and image analysis results were combined to evaluate the depth of water film on the road surface, by which the road condition evolution pattern can be forecasted. Fukui et al. [16] analyzed the slippery condition of the road by calculating the brightness and spatial spectrum of road surface images. Li et al. [17] extracted RGB, HIS, and YUV of road surface images and established the road surface state recognition model based on improved BP neural network. The recognition accuracy rate of this model could reach more than 85%. However, it was still on the theoretical research stage with the small size of training sample. Liu and Huang [18] collected wet road images and then designed a SVM classifier of road slippery state classification. Among them, the misjudgment rate of the dry state is slightly high, while the recognition accuracy of snow state is slightly high. Besides, the identification of hybrid road surface state was still to be studied. Wan et al. [19] used the RBF neural network model to discriminate the slippery conditions of different roads, and the recognition accuracy was 78.4%. Among them, the recognition accuracy of dry and silt state is low, and the recognition accuracy of snow and ice state is high.

Image feature extraction is a key step in image recognition. Zhang et al. [20] extracted the eigenvector of RGB color moment and the Munsell color moment from the images. The results show that the color moment feature can describe the color characteristic of the image well. Shinde et al. [21] extracted a variety of color features of images to form a preprocessing database of color eigenvector and then used machine learning to perform image classification experiments. The experimental results show that the classification accuracy can be achieved based on multiple color feature databases. Bhave et al. [22] extracted the color feature by calculating the average value of each color component and then used gray level cooccurrence matrix to extract texture eigenvectors. Based on the feature values above, image state can be classified. Haralick et al. [23] proposed some easy-to-calculate texture eigenvectors based on the gray level cooccurrence matrix. The texture feature is used to identify the aerial images. Experimental results show that the texture feature is the applicability of image classification. Mohanaiah et al. [24] extracted the four image texture eigenvectors based on the gray level cooccurrence matrix, including the second moment, correlation, inverse moment, and entropy. The recognition experiments show that calculation time can be saved and the recognition accuracy is high via these texture features.

Reviewing the above literatures, it is found that the existing problems and development trends of image recognition technology are as follows:(1)Image recognition technology is the main technology of road surface recognition. However, due to the complexity of the road scene and the weak adaptability of the vision system to the illumination change, the road condition detection method based on machine vision has the problem of weak adaptability, the poor robustness of illumination, and low recognition accuracy at present.(2)The identification of the hybrid road surface state is one of the main problems in this study.(3)Using SVM, neural network, and other machine learning methods to identify the road surface state is the development trend.(4)Extracting appropriate multidimensional color and texture eigenvectors can help to improve the accuracy of road surface state recognition.

Therefore, this paper presents a new method based on SVM classifier and image segmentation processing to solve the problem of the small size of the sample and nonlinear and high-dimension pattern recognition. First of all, the comprehensive sample database of road surface state is established by collecting road surface images in different scenes through a variety of ways. Then, 13-dimensional color and texture eigenvectors are extracted to build the training database of road surface state. Next, the optimal parameters of the SVM classifier are trained by the grid searching optimization algorithm and the PSO algorithm, respectively. Thus two kinds of road surface state classification models are built and the performances of the two optimization classification models are compared. For the hybrid road surface state recognition, the road surface state image is segmented into blocks and the overall state of road surface state is presented. Finally, the algorithm proposed is tested and the ideal recognition results are obtained based on the large-scale samples.

2. Eigenvectors Extraction of Road Surface State from Images

The road surface image information mainly includes color, texture, shape, and other characteristics. In this paper, representatively typical road surface state image samples are selected and color and texture eigenvectors are extracted, and the road surface state image feature database can be formed by researching color and the texture characteristics of road state.

2.1. Extraction of Color Eigenvectors

Color eigenvectors of road surface image are usually stable and not sensitive to size or direction. Among them, the color moment feature has the characteristics of translation invariance, rotation invariance, and scale invariance, which can ensure the integrity of image color information [20, 21, 25]. Therefore, this paper adopts the third-order color moment method to extract the road surface image color feature. The definitions are as follows:where is the color channel and is the gray value of image. is the first-order color moment of image . is the second-order color moment. is the third-order color moment. is the probability of the pixels with gray scale occurrences in the color channel of image. is the total number of pixels in the image. Equation (2) is a 9-dimensional color moment vector, indicating the color feature of images, based on the HSV (hue, saturation, and brightness) color model:

2.2. Extraction of Texture Eigenvectors

Gray level cooccurrence matrix can better represent the texture information [23, 26, 27]. In this paper, we choose the gray level cooccurrence matrix method to extract four commonly used texture features of road surface images.

(1) Energy

Energy reflects the texture thickness of image. When the texture is coarse relatively, is larger; on the contrary, is smaller, where , are gray scale values of pixels. is the spatial relationship between the two pixels. is the generated direction of the gray level cooccurrence matrix. is the number of occurrences of and pixels with the spatial relationship .

(2) Entropy

Entropy reflects the amount of the image information. When the image has more textures, the entropy value is larger. If the image contains fewer textures, the entropy value is smaller. If the image has no textures, the entropy value is close to zero.

(3) Contrast

The contrast reflects the clarity of the image texture. In images, the deeper the texture groove, the greater the contrast, and the clearer the image texture visual effect.

(4) Correlationwhere , , , and .

Correlation value reflects the correlation of local gray scale in images. When the values of the matrix elements are evenly equal, the correlation value is large. On the contrary, when the values of the matrix elements are very different, the correlation value is small.

Based on the research above, a set of 13-dimensional road surface state eigenvectors is determined as

3. Database Construction of Road Surface State Feature

3.1. Image Samples Collection of Road Surface State

As shown in Figure 1, we set up a road surface image acquisition experimental system including the road surface image acquisition camera, the hard disk video recorder, and the computer. This system can cover the entire road and achieve all-weather road image acquisition.

The basis of road surface state recognition is to establish the road surface state feature database, which needs to collect a large number of road surface state image samples through various ways. Because of the simplicity of the road surface images collected by the experimental system, we also use the highway video surveillance resources, network resources, and other video resources to collect road images to expand the sample database.

3.2. Image Samples Database Construction of Road Surface State

The road surface state is divided into five types including dry, wet, water, ice, and snow. According to the influence of original images to samples database under the condition of different images size and lighting scenes, the original image segmentation principle is proposed as shown in Table 1. According to Table 1, original images are divided into blocks, and then the single state blocks are selected to construct the road surface state samples, which effectively guarantee the quality and purity of the road surface image database.

In this paper, 500 dry images, 500 wet images, 500 water images, 500 snow images, and 500 ice images totaling 2500 images were collected to construct the sample database. Some of the image samples are shown in Figure 2.

3.3. Database Construction of Road Surface State Feature

Based on the road surface state image sample database, 500 samples were collected for each state, and the color and texture eigenvectors were extracted to build the road surface state feature database. Figures 36 show part of the color and texture feature curves of 200 samples for each state.

As shown in Figure 3, the range of first-order moment of dry samples is , the range of first-order moment of the wet samples is , the range of first-order moment of water samples is , the range of first-order moment of snow samples is , and the range of first-order moment of ice samples is . It can be seen that there is a large difference in first-order moment values between dry samples and snow samples, while the first-order moment curves of the wet, water, and ice samples show characteristics of overlapping.

As shown in Figure 4, the range of second-order moment of dry samples is [0.03, 0.21], the range of second-order moment of wet samples is [0.04, 0.20], the range of second-order moment of water samples is [0.01, 0.25], the range of second-order moment of snow samples is [0.01, 0.07], and the range of second moment of ice samples is [0.04, 0.16]. It can be seen that the second moment values of snow samples are small, and there is a big difference with the other four samples. The second moment curves of dry, wet, water, and ice samples are hard to distinguish because of obvious overlapping.

As shown in Figure 5, the range of energy values of the dry samples is [1.52, 4.74], the range of energy values of the wet samples is [1.71, 4.92], the range of energy values of the water samples is [0.18, 2.48], the range of energy values of the snow samples is [0.01, 2.13], and the range of energy values of ice samples is [1.86, 4.94]. It can be seen that the energy value curves of water and snow samples are overlapped, while the curves of energy values for dry, wet, and ice samples show characteristics of overlap.

As shown in Figure 6, the range of entropy of dry samples is [0.01, 0.35], the entropy of wet samples is [0.01, 0.36], the range of entropy of water samples is [0.04, 0.98], the range of entropy of snow samples is [0.14, 0.99], and the range of entropy of the ice samples is [0.03, 0.25]. It can be seen that the entropy curves of wet and ice samples are overlapped, and the entropy curves of dry, water, and snow samples are overlapped.

It can be concluded that the single feature curves of the five states have an overlapping area, but there are obvious differences in the feature vectors between at least two kinds of states. The 13-dimensional feature mentioned in this paper can help to accurately identify the road surface state.

4. Design of SVM Classification Optimization

4.1. Design of Classifier Based on SVM

The principle of SVM [28, 29] is to find the optimal hyperplane, which ensures the accuracy of the hyperplane classification, while the distance on both sides of the hyperplane can be maximized. A nonlinear multiclass SVM classifier is designed for the recognition of hybrid road surface states. The nonlinear-to-linear transformation depends on the nonlinear transformation from the kernel function to input space. Classifier design algorithm is as follows.

Linear SVM classification function is as follows:where is the input vector. is the vector type. is the number of input vectors. is the optimal weight vector. is the optimal bias. is the multiplier for the Lagrangian function. is the support vector. is the number of support vectors.

For the nonlinear classification function, the existence of misclassified samples is allowed by introducing nonnegative slack variable , and the classification hyperplane is

In this case, the reciprocal of the maximum classification interval is , where is the penalty factor for SVM.

After constructing the optimal hyperplane, the most widely used Gaussian kernel function is used [28, 29], and the input vector is transformed from the input space to the high-dimensional feature space with transformation,

Then the input vector is replaced by the eigenvector , and the nonlinear optimal classification function is obtained as

Based on the nonlinear optimal classification function, the main idea of multiclassification can be explained as follows: Assuming that a SVM classifier is designed between every two types of samples, SVM classifiers need to be designed for samples [28]. Therefore, ten SVM classifiers are designed for the five road states. When classifying an unknown sample, each classifier evaluates and counts its type, and the most statistical result can be regarded as the type of the test sample.

4.2. Parameter Optimization of SVM

In the process of SVM classification and identification, the penalty factor and Gaussian kernel function factor have a great impact on the accuracy of training [30, 31]. The higher can result in overlearning state, which means training set classification accuracy is high while test set classification accuracy is too low. The higher can lead to excessive support vectors and interfere with the efficiency of training and learning [31]. In order to solve the problems above, the grid searching algorithm and Particle Swarm Optimization algorithm are used to obtain the optimal parameters of and and improve the recognition efficiency and accuracy of SVM.

4.2.1. Parameters Optimization Based on Grid Searching Algorithm

Based on the grid searching algorithm, the principle of parameter optimization [30] is to make the SVM penalty factor and Gaussian kernel function factor divide the image into grids in a certain range and then traverse all the points in the grids to obtain the values. For the defined and , the K-CV (cross-validation) method is used to get the training set of this group to verify the classification accuracy. Finally, the best combination of and with the highest classification accuracy of verified raining set is obtained. Where the range of is set to [2−8, 28], the range of is set to [2−8, 28].

Among them, there will be many combinations corresponding to the highest verification classification accuracy. The combination of the smallest is selected as the best one, and if the corresponding are more than one, the firstly searched combination can be selected as the best one.

4.2.2. Parameters Optimization Based on Particle Swarm Algorithm

The basic principle of Particle Swarm Optimization (PSO) [3133] is as follows: suppose that an ethnic group consists of particles in a -dimensional search space, where the position of the particle (the optimal solution) is , the velocity is , and the optimal position of the particle is denoted as . The globally optimal solution of the ethnic group is denoted as . After finding the two optimal solutions, the particle velocity and position vector are updated based on where and . is the inertia weight. , are acceleration constants, generally set as 2. , are random numbers ranging between 0 and 1. is the number of iterations.

Parameters optimization based on the particle swarm algorithm is as follows.

Step 1. Initialize the size and initial velocity of the particle , and initialize the parameters , and the maximum number of iterations .

Step 2. The fitness value of each particle is calculated, and the classification accuracy, = number of samples correctly classified/total number of samples, trained by cross-validation of SVM is used to evaluate the fitness value of each particle.

Step 3. The fitness value of each particle and its optimal position are compared, respectively, and the optimal value is obtained. If the current value is better than , is set as the current value, which means the location is set as the current location.

Step 4. Comparing the fitness value of each particle and the optimal value of the ethnic group, if the current value is better than , the subscript and fitness value of the current particle are set as the subscript and the fitness value of .

Step 5. According to (12), the particle velocity and position are updated.

Step 6. When the end condition is reached, the times of iterations are completed, and the optimal value is output and the best parameter can be obtained.

5. Image Blocks Validation of Road Surface State

Firstly, two SVM parameters optimization algorithms are used to obtain two groups of optimal training parameters . Then 80% of the samples in the road surface state feature database are trained based on the best training parameters , and two road surface state classification models are obtained. After that, the remaining 20% of the data samples are tested to examine the performance of the two classification models. Finally, the road surface state samples in an actual environment are selected for experimental validation.

5.1. Establishment of Training Model

(1) Mark the surface state conditions: dry as D, wet as Wt, water as Wr, snow as S, and ice as I. The eigenvectors of 400 samples of each road state were extracted to form the training database.

(2) The training data is inputted into SVM classifier; the best training parameters are gotten. And then, two kinds of classification models are established.

From Table 2, it can be seen that the training accuracies of the two classification models are almost the same. However, the PSO algorithm is significantly less time-consuming and with better applicability than the grid searching algorithm.

(3) 20% of the sample data were tested by the classification model to verify the recognition performance of the two classification models. The test results are shown in Table 3.

From Table 3, it can be seen that the accuracy of the PSO model is higher than that of the grid searching algorithm, and the performance of the PSO model is better.

5.2. Image Segmentation Recognition of Actual Road Surface

Firstly, the actual road surface image is divided into blocks according to the segmentation principle. Next, the 13-dimensional feature of each block is extracted. Then the road surface block feature vectors are input into two classification models mentioned above. And the state of each block will be recognized. When all the blocks are recognized, the proportion of each state will be counted.

5.2.1. Image Validation of Dry State

The recognition results of the dry road surface state under good illumination condition (from the experimental system) are shown in Figure 7 and Tables 4 and 5.

Table 6 shows the statistic results of each road surface state.

From Table 6, it can be seen that the test ratio of dry state is 93.33% after the grid searching optimization. After the optimization by PSO, the ratio of dry state is 100% and increases by 6.67%.

The recognition results of dry road surface state under adverse lighting conditions (from the experimental system) are shown in Figure 8 and Tables 7 and 8.

Table 9 shows the statistic results of each road surface state.

From Table 9, it can be seen that the proportion of the test images identified as dry is 77.78% after optimization by the grid search algorithm. After the PSO optimization, the test image recognition rate is 95.56% and increases by 17.78%.

5.2.2. Image Validation of Wet State

The recognition results of the wet road surface state under good illumination condition (from the surveillance system) are shown in Figure 9 and Tables 10 and 11.

Table 12 shows the statistic recognition results of each road surface state.

From Table 12, it can be seen that the proportion of the test images identified as wet is 81.25% after optimization by the grid search algorithm. After the PSO optimization, the test image recognition rate is 93.75% and increases by 12.50%.

The recognition results of the wet road surface state under adverse illumination condition (from the experimental system) are shown in Figure 10 and Tables 13 and 14.

Table 15 shows the recognition results of each road surface state.

From Table 15, it can be seen that the proportion of the test images identified as wet is 80.00% after optimization by the grid search algorithm. After the PSO optimization, the test image recognition rate is 93.33% and increases by 13.33%.

5.2.3. Image Validation of Water State

The recognition results of the water road surface state under good illumination condition (from the mobile camera) are shown in Figure 11 and Tables 16 and 17.

Table 18 shows the recognition results of each road surface state.

From Table 18, it can be seen that the proportion of the test images identified as water is 78.57% after optimization by the grid search algorithm. After the PSO optimization, the test image recognition rate is 96.42% and increases by 17.85%.

The recognition results of the water road surface state with reflection (from the Internet images) are shown in Figure 12 and Tables 19 and 20.

Table 21 shows the recognition results of each road surface state.

From Table 21, it can be seen that the proportion of the test images identified as water is 75.00% after optimization by the grid search algorithm. After the PSO is optimized, the test image recognition rate is 89.59% and increases by 14.59%.

5.2.4. Image Validation of Snow State

The recognition results of the snow road surface state (from the Internet images) are shown in Figure 13 and Tables 22 and 23.

Table 24 shows the recognition results of each road surface state.

From Table 24, it can be seen that the proportion of the test images identified as water is 67.35% after optimization by the grid search algorithm. After the PSO optimization, the test image recognition rate is 85.71% and increases by 18.36%.

The recognition results of the snow road surface state (from the surveillance system) are shown in Figure 14 and Tables 25 and 26.

Table 27 shows the recognition results of each road surface state.

From Table 27, it can be seen that the proportion of the test images identified as water is 77.55% after optimization by the grid search algorithm. After the PSO optimization, the test image recognition rate is 91.84% and increases by 14.29%.

5.2.5. Image Validation of Ice State

The recognition results of the ice road surface state under good illumination condition (from the experimental system) are shown in Figure 15 and Tables 28 and 29.

Table 30 shows the recognition results of each road surface state.

From Table 30, it can be seen that the proportion of the test images identified as ice is 93.33% after optimization by the grid search algorithm. After the PSO optimization, the test image recognition rate is 97.78% and increases by 4.45%.

The recognition results of the ice road surface state with snow (from the Internet image) are shown in Figure 16 and Tables 31 and 32.

Table 33 shows the recognition results of each road surface state.

From Table 33, it can be seen that the proportion of the test images identified as ice is 71.11% after optimization by the grid search algorithm. After the PSO optimization, the test image recognition rate is 77.78% and increases by 6.67%.

5.2.6. Image Validation of Hybrid State

The recognition results of the ice, wet, and water hybrid state (from the mobile image) are shown in Figure 17 and Tables 34 and 35.

Table 36 shows the recognition results of each road surface state.

From Table 36, it can be seen that the error rate of each image block is relatively high on hybrid road condition after the grid searching optimization. After PSO optimization, the recognition accuracy of each block of the test image is improved, and the distribution of road conditions can be given accurately.

The recognition results of the ice, wet, and water hybrid state (from the surveillance system) are shown in Figure 18 and Tables 37 and 38.

Table 39 shows the recognition results of each road surface state.

From Table 39, it can be seen that the error rate of each image block is relatively high on hybrid road condition after the grid searching optimization. After PSO optimization, the recognition accuracy of each block of the test image is improved, and the distribution of road conditions can be given accurately.

6. Conclusions

There are a large number of traffic accidents caused by bad weather condition or slippery road condition. Therefore, road states greatly affect the traffic safety and transport efficiency on highway. It is of great social significance to study the classification of wet and slippery road condition, which can provide reference and theoretical basis for traffic control and meteorological management and ensure traffic safety.

There are many limitations in using instrument to recognize road surface conditions, and image recognition is becoming the main technology for recognizing road surface state. However, recognition under hybrid road conditions and different lighting conditions are two problems that need to be solved.

Based on SVM algorithm and image segmentation processing technology, we propose a method of video image processing technology for road surface state recognition. First of all, according to the segmentation principle, the road surface samples are divided into blocks and the road surface state sample database is constructed. Then, 9-dimensional color eigenvectors and 4-dimensional texture eigenvectors are extracted to form a 13-dimensional eigenvectors database which can describe the road surface state. After that, the SVM classifier is trained by using grid searching optimization and PSO optimization to obtain the road surface state classification model. And then, the performances of two classification models are tested. Finally, a road surface state recognition program was developed to test the actual road surface state images in a variety of environments.

The test results show that (1) the establishment of a perfect sample database is the basis for accurate recognition of road surface state. The quality and purity of the sample database can be ensured by dealing with single state image blocks. (2) Each feature value of the five states has overlapping parts, while 13-dimensional eigenvectors can satisfy the need of state recognition accurately. (3) After the SVM parameter optimization, the performance of road state classification model is superior, in which the performance of the PSO algorithm is better than that of the grid searching optimization algorithm, and the accuracy of state recognition is improved. (4) Image segmentation method can be used to obtain the distribution of road surface state, which solves the problem of hybrid road surface state and road surface under different light conditions. The recognition accuracy of single state is above 90%, and the recognition accuracy of hybrid state is more than 85%.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this article.

Authors’ Contributions

Jiandong Zhao and Hongqiang Wu presented the algorithms, analyzed the data, and cowrote the paper; Liangliang Chen installed the experimental system and performed the experiments.

Acknowledgments

This work is supported by the Fundamental Research Funds for the Central Universities (2016JBM053).