Abstract

Vision-based multivehicle detection plays an important role in Forward Collision Warning Systems (FCWS) and Blind Spot Detection Systems (BSDS). The performance of these systems depends on the real-time capability, accuracy, and robustness of vehicle detection methods. To improve the accuracy of vehicle detection algorithm, we propose a multifeature fusion vehicle detection algorithm based on Choquet integral. This algorithm divides the vehicle detection problem into two phases: feature similarity measure and multifeature fusion. In the feature similarity measure phase, we first propose a taillight-based vehicle detection method, and then vehicle taillight feature similarity measure is defined. Second, combining with the definition of Choquet integral, the vehicle symmetry similarity measure and the HOG + AdaBoost feature similarity measure are defined. Finally, these three features are fused together by Choquet integral. Being evaluated on public test collections and our own test images, the experimental results show that our method has achieved effective and robust multivehicle detection in complicated environments. Our method can not only improve the detection rate but also reduce the false alarm rate, which meets the engineering requirements of Advanced Driving Assistance Systems (ADAS).

1. Introduction

As an important part of the intelligent transportation system (ITS), the Advanced Driving Assistance Systems (ADAS) can significantly improve the driving safety. Forward Collision Warning Systems (FCWS) and Blind Spot Detection Systems (BSDS) are principal portions of ADAS, and their performance depends on the real-time capability, accuracy, and robustness of the vehicle detection method. Recently, with the increasing maturity of visual sensors, vision-based vehicle detection has become a hot topic in the field of intelligent vehicle. There are plenty of approaches proposed for the day time vehicle detection. These methods can be divided into the following categories: methods based on prior knowledge, such as shadow-based [1, 2], taillight-based [1, 2], horizontal (vertical) edge-based [24], and symmetry-based vehicle detection method [2]; methods based on stereo vision; this type of method detects vehicles by using the three-dimensional information. The most widely used methods are inverse perspective transformation (IPM) based method and disparity map based method [1]; template-based methods use predefined patterns of vehicle class and perform correlation between the image and the template [1]; the main detection steps of appearance-based methods are as follows: the appropriate descriptors are first used for representing vehicles in the image; then the machine learning methods are used to train these descriptors. Much processes have been made in appearance-based vehicle detection, such as algorithm based on HOG + AdaBoost [5], Haar + HMM [6], Haar + AdaBoost [79], HOG + SVM [10], PCA – ICA + GMM [11], and minimum Mahalanobis distance classifier [12]. The method based on the motion information detects vehicles by using the motion information between vehicles and scenes, such as finding out vehicles by calculating the change of optical flow information which is caused by the relative motion of vehicles or scenes [13].

To improve the accuracy of vehicle detection methods, several of above methods are combined together to detect vehicles. Lin et al. [3] applied the SURF and edge features to represent the vehicle and, combining with probabilistic methods, their methods have achieved vehicle detection in the blind spot area. Chen et al. [6] first used a road modelling method to confine detection regions, and then Haar-like features and eigencolours were used for detecting vehicles. Finally, a tackling method was used. Tehrani Niknejad et al. [10] proposed a deformable vehicle model based on HOG feature; the method can achieve the adaptive threshold vehicle detection under urban roads. Wang and Lien [11] proposed a vehicle detection method based on a statistical model of local feature. They applied the public dataset Caltech Cars (Rear) [16] to test their method. Alonso et al. [12] proposed a vehicle detection method based on multidimensional classification. They represented vehicles in form of rectangular subregions based on the robust classification of features vectors result of a combination of multiple morphological vehicle features. Their method can detect vehicles with very different aspect-ratio, color, and size. Chang and Cho [8] presented a vehicle detection algorithm based on combination of Haar feature and online boosting. Their algorithm has realized vehicle detection in various environments. Sivaraman and Trivedi [9] proposed an active-learning framework based on Haar feature and AdaBoost for vehicle detection on the highway. Jazayeri et al. [13] proposed an optical flow and hidden Markov model-based vehicle detection method which mode the locations and motion information of vehicles in the image level. Their method can deal with the vehicle identification problem under the scene of changing illumination and environment well.

Vehicle detection method based on a single feature can quickly detect vehicles in images. However, using single feature method may cause a lot of false alarms, because single feature only describes one certain characteristic of vehicles. Most of the appearance-based vehicle detection methods' performance excessively depends on the number and scale of training samples. Various samples in different situations are needed to generate more powerful classifiers. In addition, detecting vehicles in images using appearance-based methods which has to scan the whole image requires excessive calculation and cannot meet the real-time requirement of FCW. To solve the above-mentioned problem, a widely used method is multifeature fusion which combines several single feature-based algorithms together by using voting method. This can significantly reduce the false alarm rate, but the detection rate is reduced either. In recent years, mathematical theory has been widely used for improving the performance of complex vehicle systems. Much process has been made in the field of mathematical modeling and control methods [1723], such as adaptive back stepping control for active suspension systems with hard constraints [17], saturated adaptive robust control for active suspension systems [18], and adaptive robust vibration control of full-car active suspensions with electrohydraulic actuators [19]. Choquet integral is a widely used method in data fusion [2426]; it can seek the maximum consistency of decision from the consistency and conflict detection results of multiple features. To improve the performance of vehicle detection algorithm and to solve problems above, we propose a multifeature fusion vehicle detection algorithm based on Choquet integral. Experiment results show that our multifeature fusion method will not only improve the detection rate but also reduce the false alarm rate.

Figure 1 illustrates the workflow of our approach. The rest of the paper is organized as follows. Section 2 briefly introduces the shadow-based vehicle region of interest (ROI) detection method. In Section 3, vehicle taillight feature similarity measure, vehicle symmetry feature similarity measure, and HOG + AdaBoost feature similarity measure are presented, respectively. Then our multifeature fusion vehicle detection algorithm based on Choquet integral is introduced in Section 4. Experiment results for the proposed method are shown in Section 5; finally Section 6 draws conclusions.

2. Shadow-Based Vehicle ROI Detection

The shadow-based vehicle detection algorithm is usually applied to extract the vehicle ROIs in the whole images for reducing computation complexity [1]. We have developed a shadow-based vehicle detection method, and the basic principle of the method is that regions underneath vehicles are distinctly darker than any other regions on an asphalt road. The grayscale of pixels in shadow regions is much lower than that in any other regions in the same image. Grayscale histogram (GH) can reflect the whole image grayscale distribution well. The grayscale of vehicle shadow pixels belongs to the lower parts of GH. So we can detect the shadow regions underneath vehicles by segmenting GH with a threshold th_BW. Figure 2(a) is a vehicle image from Caltech Cars (Rear) [27]. Black regions in Figure 2(b) are shadow regions segmented by setting th_BW to 0.1. The green lines in Figure 2(c) are vehicle shadow lines detected by shadow-based vehicle detection method.

3. Feature Similarity Measure

To make full use of the Choquet integral in our multifeature fusion vehicle detection framework, each single feature should be first represented in form of fuzzification before calculating the Choquet integral. After this phase, the algorithm can fuzz the output of each single feature; then the result can be determined by using the fuzzy judgment instead of direct judgment. Therefore, in this section, we detailedly introduce three feature-based vehicle detection methods and their feature similarity measure functions.

3.1. Vehicle Taillight Feature Similarity Measure

The red taillights and braking lights are important features for detecting the rear-view vehicle. Taillight-based feature provides an important criterion for our multifeature fusion vehicle detection framework. The RGB components of pixels in taillight regions are obviously different from the other parts of vehicle ROI (except red cars). Therefore, by following this rule, we present a similarity measure method based on color feature of vehicle taillights. First, taillight regions in the vehicle ROI are detected by threshold value segmentation method. The key threshold of method can be acquired by analyzing RGB components from images of taillights and the other parts of vehicle. The collection of images for setting the threshold should be large enough and includes different vehicles and various scenes. We acquire the , , and components distributions by analyzing the public collection [27] and images captured by our camera. As shown in Figure 3(b), differences between the component and the component of the other regions of vehicle are mainly distributed on the range of . It is different from the values of in taillight regions illustrated in Figure 3(a); therefore, the taillight regions of vehicle ROI can be detected by setting a certain threshold :

Figure 4(a) is the vehicle ROI detected by the shadow-based vehicle detection method. Figure 4(b) is the binary image of taillights detected by employing (1) on the vehicle ROI. Canny-based edge detection method is used to detect the edges of taillights in Figure 4(b), and Figure 4(c) is the edge image of Figure 4(b). Then the connected domains in Figure 4(c) are extracted. The input images of connected domain extraction method are binary and edge image which are illustrated as Figures 4(b) and 4(c), respectively. Finally, the minimum circumscribed rectangles (MCR) of connection domains are calculated. The detected MCRs are illustrated as the red rectangles in Figure 4(d).

Each MCR of connected domain is represented by the left top point and the right top point of MCR. Two left top points in vehicle ROI can form a straight line; the slope of straight line is defined as The distance between each MCR is represented as (3). The two taillights of vehicle are usually on a horizontal line, and the thresholds and can get rid of the straight lines that are not horizontal or almost horizontal:

Definition 1. The taillight feature similarity measure function is defined as

3.2. Vehicle Symmetry Feature Similarity Measure

The symmetry measure is a statistic to describe the symmetry of target. Vehicles are obviously symmetrical objects; therefore, we use the symmetry feature as a similarity measure in our algorithm. According to the symmetry-based method described in [28], we use the symmetry measure method based on normalized entropy to calculate the symmetry value of each vehicle ROI. The symmetry measure is described as (5), where is the symmetry measure of target. is the information entropy, which is also the mathematical expectation of information content. is the max value of information entropy. Consider

Definition 2. The symmetry feature similarity measure function is defined as

3.3. HOG and AdaBoost Classifier Feature Similarity Measure

The histogram of oriented gradient (HOG) is a descriptor of feature which has been widely used in object detection. Zhu et al. [29] introduced an efficient pedestrian detection method based on HOG and AdaBoost. In our previous work, we use HOG feature to detect pedestrian [14]. The HOG feature is represented by calculating the histogram of oriented gradient of local region in the image. First, the image is divided into a plurality of grids according to a certain size; these grids are called BLOCK which are illustrated as in Figure 5(a). Then each BLOCK is divided into four regions which are called CELL. Each CELL projects an orientation-based histogram which includes nine bins. In this histogram, the horizontal ordinate is a range of direction angles which divide 180° into nine equal parts, and the vertical coordinates are an accumulation of each angle range. Finally, a 36D feature vector named BLOCK is formed. Due to the strong edge feature of vehicles, we employ the HOG feature to represent vehicles; then the AdaBoost-based algorithm [30] is applied to generate weak classifiers.

In this paper, the training samples of generating HOG + AdaBoost classifiers are images captured from actual driving environments. Vehicle regions of these images are positive samples, and other regions of images are negative samples. The amount of positive samples and negative samples are both 10000. These samples are normalized to the same size (30 × 30). Screenshots of samples are shown in Figures 5(b) and 5(c). There are two phases to employ the HOG + AdaBoost classifier and the training and the detection phase. In the training phase, we extract HOG features by applying CELL size of 5 × 5, 10 × 10, and 15 × 15, respectively; the scanning step size is three pixels, and the weak classifiers are selected by AdaBoost algorithm. After training, we use the samples (positive 10000 and negative 10000) which are different from that of the training phase to test the weak classifiers. The ROC curves of HOG + AdaBoost algorithm under three different CELL sizes are illustrated as in Figure 6; the performance of HOG + AdaBoost classifiers whose CELL size is 15 × 15 is the best among these three types; therefore, we set the CELL size to 15 × 15 in our further experiments.

To enhance the performance of HOG + AdaBoost classifiers, inspired by method in [9], the active-learning based HOG + AdaBoost framework is used by following the steps in the Active-Learning Framework. The advantage of this framework is that you are only adding negative samples that would otherwise be causing false positives. There is no point in adding more negative samples that are handled by the original training anyways.

Active-Learning Framework.Step  1. Train HOG + AdaBoost classifiers using the 10000 positive samples and 30000 negative samples.Step  2. Run the algorithm by using well-trained HOG + AdaBoost classifiers on a large video set (not the training set from Step  1).Step  3. Any false positives from the run in Step  2 can be put in the negative set.Step  4. Retrain the algorithm using the original true positive set and the updated negative set (negatives from both Step  1 and Step  3).Step  5. This can be repeated as many times as appropriate, using new video on each iteration.

In detection phase, each vehicle ROI detected by shadow-based vehicle detection method is resized to the same size of the training sample; the HOG feature is extracted in the same way of training phase. Then use the well-trained classifiers to identify the vehicle ROI; the classification value of each vehicle ROI is calculated by

Most of AdaBoost-based object detection methods decide whether the ROI is object or interference by judging whether the symbol of the hogadbCoeff is positive or negative. This way is not suitable to employ the HOG and AdaBoost-based classifiers in our Choquet integral-based multifeature fusion vehicle detection framework. To represent the hogadbCoeff in form of probability, we first test the well-trained classifiers by using the testing sample set which is different from the training sample set. And then the statistic distribution of hogadbCoeff is calculated. Finally the mapping table between the HOG and AdaBoost classifier feature similarity measure value and the hogadbCoeff is formed. The statistic distributions of hogadbCoeff belonging to vehicles and interferences are illustrated in Figure 7; we use the algorithm precision corresponding to interval of hogadbCoeff to be the ; the precision is defined as (16) in this paper; the mapping table is created as Table 1.

Definition 3. The HOG + AdaBoost classifier feature similarity measure function is defined as Table 1.

4. Multifeature Fusion Vehicle Detection Algorithm Based on Choquet Integral

In this paper, fuzzy integral theory is applied to vehicle detection in complex scenarios. First, the basic theory of Choquet integral is introduced here. And then the fuzzy measure of each feature is defined. Finally, the features of taillight, symmetry, and HOG + AdaBoost classifier are fused by Choquet integral of fuzzy theory. The brief concepts of Choquet integral and the fuzzy measure used in our algorithm are followed from the concepts in [2426].

Definition 4. Let be a finite set, and is a power set which is composed of subsets of is the mapping function from the power set to the range of . If satisfies the following three conditions, is a fuzzy measure on .(1)Boundedness: .(2)Monotonicity: , if , then .(3)Continuity: if , and is monotonous. This is also represented in the form of or , then .
The fuzzy measure which is widely applied in multifeature fusion is the regular fuzzy measure: if and , the fuzzy measure is regular.

Definition 5. If the fuzzy measure satisfies the following conditions: , , if there exists a constant value , satisfying , then is a -fuzzy measure. can be calculated by (8), where ; it is used to indicate the importance of a single feature classifier for the final evaluation, where . Consider

Definition 6. is a nonnegative function defined on , is a fuzzy measure defined on power set , and then Choquet integral of function on with respect to fuzzy measure is defined by where ; the main idea of (9) is determining the value of Choquet integral using Riemann integral by an infinite approximation method. The definition of Choquet integral is as follows when is a finite set: where is a permutation of the indices such that
When fuzzy measure is a -fuzzy measure, any subset is defined by
To apply the Choquet integral to detect vehicles in complex environments, is first initialized as the vehicle ROI detected by shadow-based vehicle detection algorithm. is a classification framework. is the feature set for detecting vehicle, where , , and represent the vehicle symmetry feature, the vehicle taillight feature, and vehicle HOG + AdaBoost classifiers feature, respectively. Let be the fuzzy density of vehicle ROI belonging to the class ; define to be the degree of importance of the feature in deciding whether vehicle ROI is vehicle or interference. Define , , and ; the higher the is, the more important the feature is. The fuzzy function is defined in so that , , and . To calculate the value of Choquet integral for each vehicle ROI, the features in the set are needed to be rearranged with respect to the order .

Main steps of our multifeature fusion vehicle detection algorithm based on Choquet integral are as follows.

Multifeature Fusion Vehicle Detection Algorithm Based on Choquet Integral.Step  1. Calculate the fuzzy measure of each feature. We test each feature-based vehicle detection method on the same vehicle sample set, and, according to (16), the precision of each vehicle detection method can be acquired. Let the precision be the fuzzy measure corresponding to each feature-based method.Step  2. Calculate by (8).Step  3. Estimate the -fuzzy measure by (12).Step  4. The Choquet integral value of each ROI temp_choquet can be calculated by (10) combining with three feature similarity measures.Step  5. Decide whether the vehicle ROI is vehicle according to (13). As it is illustrated in Figure 8, the temp_choquet belonging to the vehicle and that belonging to the interference are much more different; the threshold Th_vehicle can be set according to Figure 8:

5. Experiment Results

To verify the performance of the algorithm, experimental platform has been built in c using OpenCV 1.0 library and Visual Studio 2010. The vehicle detection algorithm is performed on an Intel Core i7-3770GHZ PC. A part of vehicle images for testing are from the public test library Caltech Cars (Rear) [27]. The rest of vehicle images are captured in the real environments (parking lot and urban road) by using DEWETRON DEWE2-M4 (camera: DEWE-CAM-01, lens: computar M3Z1228C) and SAMSUNG GT-S7562 camera (5,000,000 pixels). There are 5 video sequences in our test datasets; the frames of our datasets are 1500, and the number of vehicles in datasets is 3219. The test images include single vehicle, multivehicle, and illumination changing in the scene. We use three indicators to measure the performance of algorithms: the detection rate , the false alarm rate , and the algorithm precision . The criterion to determine a “good” detection in this paper is the overlap of the detected bounding box versus the annotated bounding box. If the overlap is larger than a certain threshold, the detection is a “good” detection. Consider

Experiment 1 (calculate the fuzzy measure of each algorithm). In our multifeature fusion vehicle detection algorithm, fuzzy measure of each feature-based algorithm is set according to the performance of its own. We test each feature-based vehicle detection method on the same vehicle sample set named JVTL. The images in JVTV are vehicle ROIs detected by shadow-based method which is introduced in Section 2. The positive samples of JVTL are vehicles, and the negative samples are interferences in JVTL. The numbers of positive and negative samples are 3219 and 6000. According to (16), the precision of each vehicle detection method can be acquired. Let the precision be the fuzzy measure corresponding to each method. According to Figure 9, we can set the fuzzy measure of every algorithm.

Experiment 2 (performance of our multifeature fusion vehicle detection algorithm). After setting fuzzy measure of each feature-based algorithm, we apply the sample set JVTL to test our method and every feature-based algorithm. As shown in Figure 10, the single feature cannot meet the requirement of high detection rate and low false alarm rate. Our algorithm fuzzes the output of each single feature, and the result is determined by using the fuzzy judgment instead of direct judgment. At the same time, the use of fuzzy integral can give full consideration to the cooperation of multifeatures and the importance degree of each feature in the recognition phase. Therefore, our method outperforms each single feature. In our experiment, the average processing time (AVT) of our method can achieve 50 ms per frame when processing on the Caltech Rear public test images whose resolutions are , which basically achieve real-time processing. And the processing time is 36 ms per frame on images whose resolutions are . Part of results of our algorithm are shown in Figures 11 and 12. Figure 11 is the result of algorithm on Caltech Rear public vehicle images; we set the main thresholds as follows: th_BW = 0.1 and Th_vehicle = 0.9. Experimental results show that our method can detect well vehicles in different distances. The distances are different in Figures 12(a) and 12(b); the distances between vehicles and camera are from 3 m to 50 m. Figure 12(c) shows that our method can not only detect the single vehicle, but also handle the multivehicle detection problem. Figure 12(d) is the detection result on urban road.

Experiment 3 (algorithm comparison). To verify the performance of our method, we compare our method to three feature-based methods, the voting method of these three feature-based methods, and vehicle detection methods in [11, 15, 16]. Algorithms used for comparison are all tested on the same collection (the public test library Caltech Cars (Rear) [27]). There are two ways to get the algorithms’ results. On one hand, we download the source code from the websites which have been provided in their articles to get the testing results. On the other hand, we directly use the testing results illustrated in the articles. Comparison result is shown in Table 2; it shows that the single feature-based methods can detect vehicle better, but the false alarm rate is also the highest. Although the voting method can reduce the false alarm rate, the detection rate is reduced either. Processing time is another indicator to measure the performance of algorithms; Ali and Wang’s methods outperform our method in terms of accuracy, but the processing time of their methods is above 500 ms. Considering both the accuracy and the processing time of algorithms, our method outperforms the other methods.

6. Conclusions

In this paper, we propose a multifeature fusion vehicle detection algorithm based on Choquet integral. There are two major contributions in this paper. First, we propose a taillight-based vehicle detection method, and a vehicle taillight feature similarity measure is defined. In addition, the vehicle symmetry and HOG + AdaBoost feature similarity measures are introduced combining with the definition of Choquet integral. Second, these three feature similarity measures are fused by Choquet integral to detect vehicles in both static test images and videos. In experiment part, our algorithm has been evaluated by using public collections and our own test images, and the experiment results are encouraging. But, to generalize our algorithm, there are still several problems to solve, such as improving accuracy of HOG + AdaBoost feature. To improve the performance of vehicle detection methods, we will address these issues and improve the multivehicle detection to an upper level.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The authors would like to thank the reviewers and editors for their comments regarding enhancing the quality of the paper. This work is supported by Grants from Jilin Planned Projects for Science Technology Development (Grant no. 20120305 and no. 20130522119JH) and Ph.D. Programs Foundation of Ministry of Education of China (Grant no. 20130061110054).