Advanced Intelligent Fuzzy Systems Modeling Technologies for Smart CitiesView this Special Issue
A Lane Extraction Algorithm Based on Fuzzy Set
Lanes are difficult to be extracted completely. A lane extraction algorithm is proposed according to vehicle driving rules. Vehicles are moving constantly, so the foreground area and background area cannot be defined effectively in the image. Therefore, based on the theory of fuzzy set, multidimensional degree is used to judge the membership degree of target and foreground in order to extract the moving area accurately. Then, the logistic regression model is established to determine the moving vehicles. Finally, based on the vehicle track, the lane extraction is realized by regional growth. The results show that the proposed algorithm can extract the road effectively.
With the rapid development of video processing technology, it promotes the development of transportation towards the direction of intelligence and automation [1,2]. Among them, road extraction is an important research direction. Niu  analyzed the vehicle operation by marking the road. Pongpaipool et al.  studied road congestion. Grote et al.  established a model based on the prior knowledge of road and extracted the road area. Grote and Heipke  established a regional model to extract roads. Lin et al.  constructed a semiautomatic tracker to realize lane line extraction. Grote et al.  constructed a model based on connectivity to realize the continuity of road extraction. Senthilnath et al.  comprehensively considered the structural, spectral, and geometric characteristics of the highway to establish a model to realize road extraction. Guan et al.  improved the Hough model to extract road centerline and estimate road width. Zhao and Wang  implemented road extraction based on mathematical morphology and snake model. Rajeswari et al.  integrated level set, normalized cuts, and mean shift methods to extract the road. Kirthika and Mookambiga  constructed a neural network model to extract roads. Karaman et al.  established a model from the perspective of multispectral to realize road extraction. Grote et al.  identified road supporting facilities and extracted road information. Miao et al.  established a model according to the road shape to determine the road centerline. Wegner et al.  built a higher-order CRF model to solve the problem that it is difficult to extract complex roads. Shi et al.  considered the road extraction problem as a binary classification problem to distinguish the road and nonroad parts. Singh and Garg  extracted trees to reduce the impact of shadow on road extraction results. Maboudi and Amini  established a multiresolution model to extract roads. Saati et al.  obtained SAR images and extracted road information. Li et al.  established a hierarchical model to extract road regions step by step. Saba et al.  established a cost function to optimize the genetic algorithm to obtain the best segmentation parameters. Wei et al.  improved CNN to realize road extraction from aerial images. Zang et al.  built joint enhancing filtering to achieve road enhancement. Sameen and Pradhan  established a segmentation and classification model to realize road extraction. Xu et al.  analyzed the road composition and extracted the road from the perspective of deep learning. Gao et al.  established a multifeature pyramid model to extract suburban roads. Bastani et al.  established a model based on road connectivity to predict the road extension direction. Hong et al.  established a novel FMH model to extract urban roads. Li et al.  fused multiple features into CNN to segment roads accurately. Qiu and Li  proposed moving target extraction and background reconstruction algorithm.
Generally, the main difficulties of road extraction are as follows: (1) it is difficult to establish a unified road segmentation model because of the limited features of the road in the image, and (2) due to the disorder of road extension, it is easy to be affected by shadow and other factors, which can result in incomplete segmentation. Therefore, we propose a complete road extraction process: (1) from the perspective of the image sequence, the driving direction of moving vehicles is analyzed to determine the road area indirectly, and (2) according to the trajectory of moving vehicles, the road is extracted completely.
2. Road Extraction Algorithm
According to the above analysis, the algorithm flowchart is established, as shown in Figure 1, and the images are input sequentially. (1) A multidimensional moving object detection method based on a fuzzy set is proposed to realize the accurate extraction of moving objects. (2) The vehicle detection module is established to detect and track the vehicle. (3) According to the vehicle trajectory, road extraction is realized.
2.1. Multidimensional Degree Moving Object Detection Based on Fuzzy Set
In the complex background, it is difficult to distinguish the moving object from the background. For this reason, scholars put forward the theory of fuzzy set to study the data with ambiguous types. Firstly, we define the fuzzy subset as A, and μA (x) is called the membership of x to A. In moving object detection, every attribute is a fuzzy set, and the selection of its membership function is very important. According to the difference between the intraclass similarity and the interclass similarity, we design the membership function:where a < c, b = (a + c)/2. The slope of this function is small near the boundary of 0.5, but large far away from the boundary. Because of the fuzziness of the features, the middle part cannot judge the foreground and background clearly, so it needs to be determined by combining with other membership relations. Therefore, we carry out the research from the dimensions of color (C), time (T), and space (S). The process is as shown in Figure 2.
The pyramid model is constructed to solve the membership functions under different resolutions and form the fuzzy vector set cluster. The upsampling is used to generate the foreground membership μc0, μc1, and μc2 (RGB three channels) of the same dimension as the original resolution. According to the membership principle of fuzzy vector set cluster, the membership degree of each point to the foreground is calculated from the color vector C, which is μc = (μc0 + μc1 + μc2)/3. Similarly, the membership of T and S is calculated.
The object motion will not produce mutation because of the continuity of the video. If a certain point is foreground in the current frame, it is more likely to be foreground in the next frame, and its probability is recorded as PF. Similarly, if the current frame is background, the possibility of the next frame as the background is also greatly recorded as PB. The background and foreground will be switched because of the uncertainty of object motion. Given that the number of foreground pixels and background pixels in T−n frame is and , when and , then PF and PB can be represented by fuzzy set. For the point in frame T−1, the membership degree of the vector to foreground and background is and within the time of T frames.
Spatial feature refers to the concentration of points. In a small area, the attributes of points should be consistent. Most of the outliers are noise, which should be removed. Based on the preliminary results of color membership, the number of foreground pixels with membership greater than 0.5 in the 5 × 5 pixel area is counted which is expressed as n. The membership degree of the space vector of the point to the foreground can be expressed as .
As the real moving object should have the following characteristics: the color difference between the object and the background point is large, the position of the object changes continuously, and the concentration is strong, then the final membership function is as follows:
2.2. Lane Extraction Algorithm Based on Vehicle Trajectory
According to the characteristics of vehicles only driving in the lane, we will extract vehicles from all moving objects and then indirectly determine the lane. LRM (logistic region model)  is a classical classification algorithm in statistics, whose process includes the following: the cost function is established, and then the optimal model parameters are obtained. Finally, the model effect is verified. The conditional probability distribution of the traditional binomial regression model is as follows:where ω is the parameter of the weight vector and · is the inner product. At this time,
The maximum likelihood estimation method is used to estimate the parameters
On the basis of L (ω), in order to avoid overfitting, the cost function is defined aswhere λ is the regularization coefficient. When J (ω) is minimum, the corresponding ω is the optimal parameter. The gradient descent method is used to get the minimum value of the cost function.
The update process is as follows:
According to the above algorithm, the vehicle area is extracted, and the vehicle trajectory set is determined by taking the vehicle gravity center as the center. Taking the trajectory set point as the seed point, the region growth method is used to realize the complete extraction of lanes.
3. Experiment Result and Analysis
20 groups of aerial videos are used in the experiment to form a database. 8 groups of representative data (43109 frames) are selected from it to verify the effectiveness of the proposed algorithm as shown in Table 1. The running environment of the program is Win 7. All images are calibrated at the pixel level by a dedicated professional person in the area where the lane is located. We use the trilinear difference to normalize the image resolution. And in this case, the frequency does not need to be normalized.
3.1. Comparison of Moving Object Extraction Algorithms
In order to verify the accuracy of moving object extraction, area overlap segmentation (AOM) and combination measure (CM) are introduced :where is the gold standard, Rs is the segmentation result, and AOM and CM are proportional to the segmentation result.
Comparing the segmentation results of different algorithms, as shown in Table 2, all algorithms show good performance for the data that the object has been moving. Because the dynamic learning factor of GMM  is fixed, the object extraction is incomplete. FD  algorithm fixes background frame, which can better extract large moving objects, but the quality of the background frame will directly affect the subsequent detection effect. EFD  reduces the influence of the background frame by fusing boundary information. DL  integrates prior knowledge, which can better restrain tailing. EGMM  simulates the light distribution to establish a model, which has a good inhibition effect on the shadow area. However, on the basis of the traditional GMM, our algorithm updates the learning factor dynamically and introduces the frame difference method to realize the complete extraction of the motion region. Although the effect is slightly lower than that of DL  and EGMM  when the first frame contains moving object and data with shadow, the proposed algorithm has high performance. Since the algorithm proposed in this paper is based on the area where the vehicle is traveling, when there is no vehicle passing through the shadow area, there will be a risk of missed detection. The follow-up research will continue to strengthen the study in this area.
3.2. Lane Extraction Result
A lane extraction algorithm is proposed in this paper. On the premise of moving vehicle detection, the vehicle trajectory graph is constructed to finally realize the lane extraction, as shown in Figure 3. Our algorithm can effectively extract the lane and effectively distinguish the pedestrian road and vehicle road.
As it is difficult to extract the lane completely, a lane extraction model is established based on moving vehicles. Based on the fuzzy set theory, the algorithm is proposed from the color, time, and space dimensions and takes the vehicle route as the benchmark to achieve accurate road extraction. On this basis, road condition analysis and road construction research can be carried out in the future.
All data used to support the findings of this study are available within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
This work was supported by Yulin Science and Technology Association Young Talents Promotion Program Project under Grant no. 20200214 and Yulin City Science and Technology Plan Project under Grant no. CXY-2020-012-02.
X. Niu, “A geometric active contour model for highway extraction,” in Proceedings of ASPRS 2006 Annual Conference, Reno, Nevada, May 2006.View at: Google Scholar
A. Grote, M. Butenuth, and C. Heipke, “Road extraction in suburban areas based on normalized cuts,” International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 36, no. 3, p. W49A, 2007.View at: Google Scholar
A. Grote and C. Heipke, “Road extraction for the update of road databases in suburban areas,” International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 37, no. 3, pp. 563–568, 2008.View at: Google Scholar
X. Lin, J. Zhang, Z. Liu, and J. Shen, “Integration method of profile matching and template matching for road extraction from high resolution remotely sensed imagery,” in Proceedings of the 2008 International Workshop on Earth Observation and Remote Sensing Applications, pp. 1–6, IEEE, Beijing, China, July 2008.View at: Publisher Site | Google Scholar
J. Senthilnath, M. Rajeshwari, and S. N. Omkar, “Automatic road extraction using high resolution satellite image based on texture progressive analysis and normalized cut method,” Journal of the Indian Society of Remote Sensing, vol. 37, no. 3, pp. 351–361, 2009.View at: Publisher Site | Google Scholar
L. Zhao and X. Wang, “Road extraction in high resolution remote sensing images based on mathematic morphology and snake model,” in Proceedings of the 2010 3rd International Congress on Image and Signal Processing, vol. 3, pp. 1436–1440, IEEE, Yantai, China, October 2010.View at: Publisher Site | Google Scholar
M. Rajeswari, K. Gurumurthy, L. Reddy, S. N. Omkar, and J. Senthilnath, “Automatic road extraction based on level set, normalized cuts and mean shift methods,” International Journal of Computer Science Issues (IJCSI), vol. 8, no. 3, p. 250, 2011.View at: Google Scholar
E. Karaman, U. Cinar, E. Gedik, Y. Yardemci, and U. Halici, “A new algorithm for automatic road network extraction in multispectral satellite images,” in Proceedings of the 4th GEOBIA, pp. 455–459, Rio de Janeiro, Brazil, May 2012.View at: Google Scholar
M. Maboudi and J. Amini, “Object based segmentation effect on road network extraction from satellite images,” in Proceedings of the 36th Asian Conference on Remote Sensing, pp. 19–23, Manila, Philippines, October 2015.View at: Google Scholar
X. Gao, X. Sun, M. Yan et al., “Road extraction from remote sensing images by multiple feature pyramid network,” in Proceedings of the IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, pp. 6907–6910, IEEE, Valencia, Spain, July 2018.View at: Publisher Site | Google Scholar
S. Qiu and X. Li, “Moving target extraction and background reconstruction algorithm,” Journal of Ambient Intelligence and Humanized Computing, pp. 1–9, 2020.View at: Google Scholar
C. Zhan, X. Duan, S. Xu, Z. Song, and M. Luo, “An improved moving object detection algorithm based on frame difference and edge detection,” in Proceedings of the Fourth international conference on image and graphics (ICIG), pp. 519–523, IEEE, Chengdu, China, August 2007.View at: Publisher Site | Google Scholar
J. Cheng, Y. Gang, S. Bai, Y.-n. Guo, and D. Wang, “An improved GMM-based moving object detection method under sudden illumination change,” in Proceedings of the International Conference on Bio-Inspired Computing: Theories and Applications, pp. 178–187, Springer, Beijing, China, November 2018.View at: Publisher Site | Google Scholar