Multispectral, Hyperspectral, and Polarimetric Imaging TechnologyView this Special Issue
Research Article | Open Access
Use of GMM and SCMS for Accurate Road Centerline Extraction from the Classified Image
The extraction of road centerline from the classified image is a fundamental image analysis technology. Common problems encountered in road centerline extraction include low ability for coping with the general case, production of undesired objects, and inefficiency. To tackle these limitations, this paper presents a novel accurate centerline extraction method using Gaussian mixture model (GMM) and subspace constraint mean shift (SCMS). The proposed method consists of three main steps. GMM is first used to partition the classified image into several clusters. The major axis of the ellipsoid of each cluster is extracted and deemed to be taken as the initial centerline. Finally, the initial result is adjusted using SCMS to produce precise road centerline. Both simulated and real datasets are used to validate the proposed method. Preliminary results demonstrate that the proposed method provides a comparatively robust solution for accurate centerline extraction from a classified image.
Very High Resolution (VHR) satellite images have become increasingly available in recent year with the advent of modern sensors, such as QuickBird, Pleiades, and WorldView. Road centerline delineation from these images has a variety of real applications (e.g., change detection, Geographic Information System (GIS) database updating, image registration, etc.). Massive approaches have been developed over the past decades to address the road extraction issue. Mena  and Das et al.  have presented good reviews of road extraction methods. Generally speaking, road centerline extraction commonly consists of two main steps: road areas extraction and centerline extraction from the road areas.
The first step focuses on the extraction of road areas from the satellite images. In this field, most commonly used methods rely on the classification technology. The most traditional classification methods are pixelwise  that rely on spectral information only. Such pixelwise methods always meet the “salt-and-pepper” phenomenon  and there is large room to improve classification accuracy. In recent years, with the development of VHR satellite images, road geometrical features are derived and explored to integrate with spectral information to meet the challenge of the “salt-and-pepper” phenomenon. To do so, a variety of road shape features are integrated with spectral information to improve road extraction accuracy, including smoothness, compactness , shape index [6, 7], second order moment [2, 8], and mathematical morphology [9, 10]. To improve the classification, numerous algorithms have been proposed, such as sparse representation and low-rank constraint  and band subset-based clustering and fusion method . Except the classification methods, there are also some other road areas extraction methods, including the segmentation method [13–16], active contour (i.e., “snake” model) [17, 18], and level set [19, 20].
The second step involves centerline extraction from road areas extracted in the first step. The commonly used method is the thinning algorithm . Despite its computational efficiency, the result of the thinning algorithm always produces undesired objects (i.e., spurs) that reduce the smoothness and accuracy of the centerline. To tackle this limitation to some extent, a self-organized clustering method  was designed. This method firstly extracted initial points using -medians clustering, followed by the linking algorithm to create the central line. Radon and Hough transform [23, 24] were also introduced to extract the centerline from the classified image. Strong peak locations in the transform matrix are taken to be straight line pixels in the original image. Radon and Hough transform are suitable for straight line extraction and generally fail to process curvilinear cases. Recently, in the area of computer vision, a so-called subspace constrained mean shift (SCMS) method [25–27] was proposed to extract accurate centerline from point clouds. First, the probability that a pixel is located on the centerline is computed using the kernel density estimation (KDE). Then, all discrete points are iteratively projected onto the centerline based on mean shift. Despite its good performance, when the number of points is large, SCMS becomes very slow, which limits its real applications. To improve the computational efficiency of SCMS, feature points-based SCMS (F-SCMS)  was proposed. F-SCMS firstly detected feature points (i.e., end and junction points) from the classified image, followed by the connection of feature points by the geodesic method to create the central lines to formulate the road network. Although F-SCMS can largely improve the speed of SCMS, it cannot process the classified image with closed form, such as circle. The aforementioned discussions indicate that the extraction of accurate road centerline from the classified image is not well resolved and this topic is still open and challenging.
There are also many other interesting studies on road extraction from a different viewpoint, such as Kalman filtering , road footprint , and dynamic programming . Although most of the aforementioned studies focus on multispectral satellite images, other sources of information can also be used to extract road networks, such as Synthetic Aperture Radar (SAR) [18, 32, 33], LiDAR [34–36], and hyperspectral images [6, 37, 38]. This paper focuses on the accurate road centerline extraction from the classified image. It is worthy to point out that this focus will not weaken the proposed method, which is also suitable for centerline extraction from road areas extracted from other sources.
The main objectives of this paper are as follows:(i)improving the efficiency of SCMS;(ii)proposing a method for accurate road centerline extraction from the classified image with high generalization ability.
The objective of this study is to devise a computationally efficient approach with high general ability to extract accurate centerlines from classified images. Figure 1 summarizes the proposed method.
The proposed method consists of the three following steps.(1)The classified image is partitioned using the Gaussian mixture model (GMM) method.(2)The major axis is subsequently extracted from the ellipsoid of each Gaussian.(3)The major axis is adjusted based on SCMS to produce accurate road centerline.
In the following subsections, details of each step are described.
2.1. Image Partition Using GMM
Road pixels in the classified map can be represented as 2D discrete joint random variables. We can partition the classified image into multiple segments by taking road pixels as observations. To do so, the Gaussian mixture model (GMM) is used to implement image clustering. The reason of selecting GMM is that the Gaussian models used by the expectation-maximization (EM) algorithm are flexible and the EM result is able to accommodate clusters of variable size. This is because EM benefits from the Gaussian distribution present in the dataset. Therefore, GMM is able to capture the complex spatial structure. In other words, GMM is able to simplify the classified road map as well as retain its spatial topology. The concepts of GMM are briefly introduced as follows.
GMM can be written as a linear superposition of Gaussians that leads to a probability density in the formwhere are mixture weights subject to , , and is a multivariate Gaussian probability distribution of mean and covariance matrix . Consider
The expected value of the indicator variables under the posterior distribution is then given by
The log-likelihood function is defined as
The parameters of each component are updated using the expectation-maximization (EM) algorithm as follows:
The EM step is iteratively repeated until convergence. It is worthy of pointing out that initial estimates for the parameters () are necessary for the first EM iteration. To this end, the -Means algorithm is selected to initialize such parameters involving running. The spatial cluster number for -Means is estimated bywhere means to round the number to the nearest integer greater than or equal to this number, is the area of the classified image, is the average road width of the classified image, and is the minimum area of a spatial cluster.
Details of the image partition using GMM are illustrated in Algorithm 1. Figure 2 gives a conceptual example. Figure 2(a) is the classified image and its corresponding spatial partition result using GMM is shown in Figure 2(b).
2.2. Extraction of Initial Road Centerlines
After the spatial partition using GMM in Section 2.1, the mean and covariance matrix for each mixture are produced. The ellipsoids of each Gaussian can be defined using and . In this study, the major axis of each ellipsoid is taken as the approximate initial centerline. is a square matrix that can be factorized aswhere is the eigenvector matrix and is the diagonal matrix whose diagonal elements are the corresponding eigenvalues. The inverse matrix of is given by
The inverse matrix can be rewritten aswhere . The variables define a new coordinate system that is shifted (by ) and rotated (by ) with respect to the original coordinates: . Recall that the equation for an ellipse in 2D is
For each Gaussian mixture, the major axis of the ellipse is estimated. After that, all major axis will compose of a coarse road centerline network that has a similar spatial topology with the true one, as shown in Figure 4.
2.3. Adjustment of Initial Road Centerline Using SCMS
The major axis produced by GMM in Section 2.2 can be regarded as the initial road centerline that is commonly coarse. For the purpose of accurate road centerline result, it is promising to obtain accurate road centerline using the adjustment techniques. To this end, subspace constrained mean shift (SCMS) is selected to align the major axis. However, SCMS is iteratively implemented on the whole classified road pixels, which leads to high computational load. On the other hand, the major axis can also be taken as the simplification case of the classified road map that only accounts for small size pixels. In other words, GMM finds the representative points of the classified road pixels to be used in principal curve projection; and therefore, GMM followed by SCMS, termed as GMM-SCMS, provides a new means to lower the computational load. The SCMS method is briefly introduced as follows.
Considering a -variate random sample drawn from a population with its density function , the kernel density estimator (KDE) is defined aswhere and , . In addition, the symmetric positive definite matrix is the bandwidth matrix, which critically determines the performance of the KDE function . is a -variate kernel function defined aswhich satisfies . Here, indicates the determinant operation, and the common choice of Gaussian KDE functions is adopted; .
By generalizing Scott’s rule of thumb, bandwidth matrix is chosen here (see Ahamada and Flachaire  for details). Consequently, the kernel density estimator is constructed above as
In consideration of the numerical stability when solving the principle curve, the logarithm of the density function is involved:
The corresponding gradient and Hessian matrices for KDE are
Thus, we get the fixed-point update rule for adjusting the coarse initial point:
On account of the similarity of the above iteration formula to the mean-shift algorithm, thus it has the name of subspace constrained mean shift (SCMS) method, which is presented in Algorithm 2. An example of adjusting coarse points to produce accurate centerline utilizing SCMS is illustrated in Figure 5.
In this section, several experiments that test the proposed method are described. The proposed method is also compared with other methods in the literature to show the advantages and disadvantages of the proposed method. In this study, MATLAB was used as the coding environment on a PC that has an Intel Core2Quad processor with 2.83 GHz clock speed.
3.1. Tests of Parameters
To test the minimum area influence on the road centerline extraction accuracy, we adjusted its value automatically from 5 to 30 with an increment of 5 for each step. The test results are presented in Figure 6. The performance of the influence is quantitatively evaluated in terms of the computational time and accuracy. Here, three accuracy measures, Completeness, Correctness, and Quality , were used.where TP, FN, and FP represent true positive, false negative, and false positive, respectively. In this study, the ground truth dataset is produced by hand drawing method and the buffer width is fixed to 8 pixels.
Figure 7 reports quantitative evaluation results with different minimum area values. Figure 7(a) indicates that the computational efficiency steadily improved with the increase of the minimum area value. This is because large will result in initial road centerline with small number of points, which in turn reduce the computational load of the adjustment using SCMS. Figure 7(b) illustrates the influence of on the accuracy. It can be seen that when is increased from 100 to 180, there is a slight change of the accuracy, which indicates that has a weak effect on the accuracy in this range of values. However, the accuracy declines dramatically when is exceeding 180. The reason for this phenomenon is that the large cannot produce the initial road centerline with the correct spatial topology as the classified image, as shown in Figure 6(f). To achieve the best balance between the computational efficiency and the accuracy, this study fixed to 180 throughout the tests.
3.2. Experiments on Simulated Images
The proposed method is tested on three simulated images, including circle image, spiral image, and snow image, as shown in Figure 8(a). The intermediate results using GMM are shown in Figure 8(b) and the corresponding precise centerline results using SCMS are given in Figure 8(c).
The proposed method is compared with two existing methods in literature: SCMS and F-SCMS. The comparison results are illustrated by Figure 9. From Figure 9, it can be seen that F-SCMS well works in spiral and snow cases but fails to process circle case. By contrast, both SCMS and GMM-SCMS are capable of extracting centerline from three test images, which indicates that SCMS and GMM-SCMS are more general than F-SCMS.
The comparison results are quantitatively evaluated in terms of computation time and three accuracy measures. Table 1 reports the computational time of three methods on test images. It can be seen that F-SCMS and GMM-SCMS generally achieve higher computational efficiency than SCMS. F-SCMS is more efficient than GMM-SCMS in snow case. In spiral case, GMM-SCMS, however, is more efficient than F-SCMS. It seems that the efficiency of GMM-SCMS and F-SCMS is influenced by points number. F-SCMS is more efficient with the large points number while GMM-SCMS in small points number. The accuracy results are given in Table 2. From the practical prospective, circle and spiral cases are too easy and the snow case is more similar to the real-world issue. Therefore, the accuracy is just compared in snow case. It can be seen that F-SCMS produces the highest accuracy among these three methods and SCMS and GMM-SCMS achieve a similar accuracy.
3.3. Experiments on Real Satellite Images
The proposed method was tested on four multispectral satellite images used in a previous work . The test images are shown in Figure 10, where the ground truth datasets are shown in red. The corresponding classified images are presented in Figure 11. The proposed GMM-SCMS method is compared with two existing methods in literature: SCMS and F-SCMS. The comparison results are reported in Figure 12. The performances of three methods are quantitatively evaluated in terms of the computational efficiency and the accuracy, as shown in Table 3 and Figure 13.
From Table 3, it can be seen that the computational load of SCMS is highest among these three methods. By contrast, both F-SCMS and GMM-SCMS produce higher computation efficiency than SCMS. In general, F-SCMS achieves better computational performance than GMM-SCMS, such as Cases –3. However, Table 3 also shows an interesting phenomenon that GMM-SCMS shows higher computational efficiency than F-SCMS  in Case . The phenomenon indicates that the number of connected components (CCs) influences the F-SCMS performance. For instance, the numbers of CCs in Cases –4 are 3, 1, 1, and 8, respectively. It seems that when the number of connected components is high, GMM-SCMS is more efficient than F-SCMS. The main reason is that F-SCMS detects feature points (i.e., end and junction points) and then links these points iteratively for each connected component. Detection of feature points from the classified image costs a lot of time. Therefore, high number of CCs decreases the performance of F-SCMS.
Figure 13 demonstrates the accuracy of three methods. As can be seen, SCMS produces the lowest accuracy while F-SCMS the highest. In general, GMM-SCMS achieves the modest accuracy among these three methods.
By comparing the proposed method with two existing methods on simulated and real datasets, it can be seen that the proposed method produces the best balance between the computational complexity, accuracy, and generalization ability among these three methods. Thus, GMM-SCMS is more practical for accurate and efficient road centerline extraction from classified images.
In conclusions, an automatic method, so-called Gaussian mixture model based subspace constrained mean shift (GMM-SCMS), was developed for road centerline extraction from the classified image. The GMM-SCMS method can extract smooth road centerlines with high accuracy. Benefited from GMM, the proposed method can represent the classified images using a small number of points while retaining the correct spatial topology, which in turn improves the computational efficiency. By using SCMS, the approximate points are iteratively adjusted to produce the accurate road centerline.
The proposed method was compared with cutting-edge SCMS and feature points-based SCMS (F-SCMS). The experimental results indicate that GMM-SCMS is more efficient than SCMS and GMM-SCMS has a higher generalization ability than F-SCMS. In other words, GMM-SCMS can process the classified image with arbitrary shape. By full consideration of efficiency, accuracy, and generalization ability, it can be concluded that GMM-SCMS can provide a practical solution for delineating road centerlines from classified images.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
The authors would like to thank Professor D. Erdogmus from Northwestern University for his kind help on the subspace constrained mean shift method, Prof. H. Mayer from the Universität der Bundeswehr München for kind discussion on road extraction, and Dr. C. Ünsala from Yeditepe University for kindly providing GeoEye-1 images. This work was supported in part by the National Natural Science Foundation of China (41201451, 40901214), Ministry of Science and Technology of China (Project nos. 2012BAJ15B04, 2012AA12A305).
- J. B. Mena, “State of the art on automatic road extraction for GIS update: a novel classification,” Pattern Recognition Letters, vol. 24, no. 16, pp. 3037–3058, 2003.
- S. Das, T. T. Mirnalinee, and K. Varghese, “Use of salient features for the design of a multistage framework to extract roads from high-resolution multispectral satellite images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 10, pp. 3906–3931, 2011.
- Y. Tarabalka, J. A. Benediktsson, and J. Chanussot, “Spectral-spatial classification of hyperspectral imagery based on partitional clustering techniques,” IEEE Transactions on Geoscience and Remote Sensing, vol. 47, no. 8, pp. 2973–2987, 2009.
- M. Fauvel, Y. Tarabalka, J. A. Benediktsson, J. Chanussot, and J. C. Tilton, “Advances in spectral-spatial classification of hyperspectral images,” Proceedings of the IEEE, vol. 101, no. 3, pp. 652–675, 2013.
- M. Song and D. Civco, “Road extraction using SVM and image segmentation,” Photogrammetric Engineering and Remote Sensing, vol. 70, no. 12, pp. 1365–1371, 2004.
- Z. Miao, W. Shi, H. Zhang, and X. Wang, “Road centerline extraction from high-resolution imagery based on shape features and multivariate adaptive regression splines,” IEEE Geoscience and Remote Sensing Letters, vol. 10, no. 3, pp. 583–587, 2013.
- X. Huang and L. Zhang, “Road centreline extraction from high-resolution imagery based on multiscale structural features and support vector machines,” International Journal of Remote Sensing, vol. 30, no. 8, pp. 1977–1987, 2009.
- W. Shi, Z. Miao, and J. Debayle, “An integrated method for urban main-road centerline extraction from optical remotely sensed imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 6, pp. 3359–3372, 2014.
- S. Valero, J. Chanussot, J. A. Benediktsson, H. Talbot, and B. Waske, “Advanced directional mathematical morphology for the detection of the road network in very high resolution remote sensing images,” Pattern Recognition Letters, vol. 31, no. 10, pp. 1120–1127, 2010.
- W. Shi, Z. Miao, Q. Wang, and H. Zhang, “Spectral-spatial classification and shape features for urban road centerline extraction,” IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 4, pp. 788–792, 2014.
- Y. Q. Zhao and J. Yang, “Hyperspectral image denoising via sparse representation and low-rank constraint,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 1, pp. 296–308, 2015.
- Y.-Q. Zhao, L. Zhang, and S. G. Kong, “Band-subset-based clustering and fusion for hyperspectral imagery classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 2, pp. 747–756, 2011.
- D. Chaudhuri, N. K. Kushwaha, and A. Samal, “Semi-automated road detection from high resolution satellite images by directional morphological enhancement and segmentation techniques,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 5, no. 5, pp. 1538–1544, 2012.
- D. Guo, A. Weeks, and H. Klee, “Robust approach for suburban road segmentation in high-resolution aerial images,” International Journal of Remote Sensing, vol. 28, no. 2, pp. 307–318, 2007.
- A. Mohammadzadeh and M. J. V. Zoej, “A Self-Organizing Fuzzy Segmentation (SOFS) method for road detection from high resolution satellite images,” Photogrammetric Engineering and Remote Sensing, vol. 76, no. 1, pp. 27–35, 2010.
- W. Wei and Y. Xin, “Feature extraction for man-made objects segmentation in aerial images,” Machine Vision and Applications, vol. 19, no. 1, pp. 57–64, 2008.
- I. Laptev, H. Mayer, T. Lindeberg, W. Eckstein, C. Steger, and A. Baumgartner, “Automatic extraction of roads from aerial images based on scale space and snakes,” Machine Vision and Applications, vol. 12, no. 1, pp. 23–31, 2000.
- L. Bentabet, S. Jodouin, D. Ziou, and J. Vaillancourt, “Road vectors update using SAR imagery: a snake-based method,” IEEE Transactions on Geoscience and Remote Sensing, vol. 41, no. 8, pp. 1785–1803, 2003.
- K. Karantzalos and D. Argialas, “A region-based level set segmentation for automatic detection of man-made objects from aerial and satellite images,” Photogrammetric Engineering and Remote Sensing, vol. 75, no. 6, pp. 667–677, 2009.
- M. Rochery, I. H. Jermyn, and J. Zerubia, “Higher order active contours,” International Journal of Computer Vision, vol. 69, no. 1, pp. 27–42, 2006.
- R. C. Gonzalez, R. E. Woods, and S. L. Eddins, Digital Image Processing Using MATLAB, Prentice-Hall, 2003.
- P. Doucette, P. Agouris, A. Stefanidis, and M. Musavi, “Self-organised clustering for road extraction in classified imagery,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 55, no. 5-6, pp. 347–358, 2001.
- C. Poullis and S. You, “Delineation and geometric modeling of road networks,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 65, no. 2, pp. 165–181, 2010.
- Q. Zhang and I. Couloigner, “Accurate centerline detection and line width estimation of thick lines using the radon transform,” IEEE Transactions on Image Processing, vol. 16, no. 2, pp. 310–316, 2007.
- U. Ozertem and D. Erdogmus, “Nonparametric snakes,” IEEE Transactions on Image Processing, vol. 16, no. 9, pp. 2361–2368, 2007.
- U. Ozertem and D. Erdogmus, “Principal curve time warping,” IEEE Transactions on Signal Processing, vol. 57, no. 6, pp. 2041–2049, 2009.
- U. Ozertem and D. Erdogmus, “Locally defined principal curves and surfaces,” Journal of Machine Learning Research, vol. 12, pp. 1249–1286, 2011.
- Z. Miao, B. Wang, W. Shi, and H. Wu, “A method for accurate road centerline extraction from a classified image,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014.
- S. Movaghati, A. Moghaddamjoo, and A. Tavakoli, “Road extraction from satellite images using particle filtering and extended Kalman filtering,” IEEE Transactions on Geoscience and Remote Sensing, vol. 48, no. 7, pp. 2807–2817, 2010.
- J. Hu, A. Razdan, J. C. Femiani, M. Cui, and P. Wonka, “Road network extraction and intersection detection from aerial images by tracking road footprints,” IEEE Transactions on Geoscience and Remote Sensing, vol. 45, no. 12, pp. 4144–4157, 2007.
- A. P. Dal Poz, R. A. B. Gallis, J. F. C. da Silva, and É. F. O. Martins, “Object-space road extraction in rural areas using stereoscopic aerial images,” IEEE Geoscience and Remote Sensing Letters, vol. 9, no. 4, pp. 654–658, 2012.
- F. Dell'Acqua and P. Gamba, “Detection of urban structures in SAR images by robust fuzzy clustering algorithms: the example of street tracking,” IEEE Transactions on Geoscience and Remote Sensing, vol. 39, no. 10, pp. 2287–2297, 2001.
- M. Negri, P. Gamba, G. Lisini, and F. Tupin, “Junction-aware extraction and regularization of urban road networks in high-resolution SAR images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 44, no. 10, pp. 2962–2971, 2006.
- Y.-W. Choi, Y.-W. Jang, H.-J. Lee, and G.-S. Cho, “Three-Dimensional LiDAR data classifying to extract road point in urban area,” IEEE Geoscience and Remote Sensing Letters, vol. 5, no. 4, pp. 725–729, 2008.
- S. Clode, F. Rottensteiner, P. Kootsookos, and E. Zelniker, “Detection and vectorization of roads from lidar data,” Photogrammetric Engineering and Remote Sensing, vol. 73, no. 5, pp. 517–535, 2007.
- P. Kumar, C. P. McElhinney, P. Lewis, and T. McCarthy, “An automated algorithm for extracting road edges from terrestrial mobile LiDAR data,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 85, pp. 44–55, 2013.
- R. B. Gomez, “Hyperspectral imaging: a useful technology for transportation analysis,” Optical Engineering, vol. 41, no. 9, pp. 2137–2143, 2002.
- S. Roessner, K. Segl, U. Heiden, and H. Kaufmann, “Automated differentiation of urban surfaces based on airborne hyperspectral imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 39, no. 7, pp. 1525–1532, 2001.
- I. Ahamada and E. Flachaire, Non-Parametric Econometrics, Oxford University Press, Oxford, UK, 2010.
- C. Wiedemann, C. Heipke, and H. Mayer, “Empirical evaluation of automatically extracted road axes,” in Proceedings of the CVPR Workshop on Empirical Evaluation Techniques in Computer Vision, Los Alamitos, Calif, USA, 1998.
Copyright © 2015 Zelang Miao et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.