Abstract
Dark channel prior (DCP) has advantages in image enhancement and image haze removal and is explored to detect highway visibility according to the physical relationship between transmittance and extinction coefficient. However, there are three major error sources in calculating transmittance. The first is that sky regions do not satisfy the assumptions of DCP algorithm. So the optimization algorithms combined with region growing and coefficient correction method are proposed. When extracting atmospheric brightness, different values lead to the second error. Therefore, according to different visibility conditions, a multimode classification method is designed. Image blocky effect causes the third error. Then guided image filtering is introduced to obtain accurate transmittance of each pixel of image. Next, according to the definition meteorological optical visual range and the relationship between transmittance and extinction coefficient of Lambert-Beer’s Law, accurate visibility value can be calculated. A comparative experimental system including visibility detector and video camera was set up to verify the accuracy of these optimization algorithms. Finally, a large number of highway section videos were selected to test the validity of DCP method in different models. The results indicate that these detection visibility methods are feasible and reliable for the smooth operation of highways.
1. Introduction
According to the road traffic accidents annual reports of China from 2007 to 2014, more than 50 percent of accidents and deaths are caused by low visibility which is below 200 meters [1]. Particularly on highway, severe congestion and major accident can easily occur in low visibility conditions. Therefore, timely and accurate detection of low visibility and early warning response are important to ensure the smoothness of highway system operation.
In the present of expensive test equipment, one often can be used to obtain accurate visibility value nowadays. However, on Chinese highway, the displacement distance between these devices is very sparse, about 30 km, which results in limited detection range and failing to meet the demand in reality. In recent years, with the development of video surveillance technologies, using image processing methods to detect low visibility has attracted a vast amount of attention and obtained numerous extensive achievements. By reviewing other literatures, current video visibility detection algorithms contain four main fields: template matching, camera model calibration, dual differential luminance, and DCP method.
For template matching method, Robert et al. [2, 3] comparatively analyzed the detected image and the images of which the visibility had been known in advance and then obtained the relative visibility. Bäumer et al. [4] used time-series images and detected the farthest object to estimate meteorological visibility. Bronte et al. [5] defined three kinds of weather patterns including mist, fog, and heavy fog; moreover, they used camera projection method to estimate visibility based on the degree of fog.
For camera calibration model based on the contrast method, Steffens [6] photographed black object image and calculated visibility value according to the relative brightness ratio of object and its background. According to the contrast values of multiple targets in an image, Taek [7] fitted the nonlinear visibility curve and got daytime visibility. Babari et al. [8, 9] presented a mapping model between image contrast and atmospheric visibility to estimate visibility base on the assumption that objects in the scene distributed continuously with the changing of distance. By selecting a virtual target area in an image marked with distance information on the road, Chen et al. [10–12] obtained the corresponding function of spatial brightness contrast and visibility by using image matching and edge feature extraction. By extracting image feature parameters which could reflect visibility changes, An et al. [13] used the least squares method and inverse transform to find a fitting function between image features and distance to determine visibility value.
For camera model calibration method based on vanishing point, by using sky and road as references, Hautiére et al. [14] determined atmospheric extinction coefficient with camera model calibration and real-time graphics processing program and calculated atmospheric visibility through the Koschmieder’s law. Li et al. [15] figured out the interested region by detecting lane boundaries and analyzed contrast values of each pixel and its neighbors. When the maximum contrast value exceeded the preset threshold, it would be considered as the distinguishable pixel by human operator. The maximum distinguishable pixel was called vanishing point; the distance between this point and camera was regarded as the current visibility. By collecting the distinguishable pixels by human operator when the contrast value is greater than the threshold of luminance contrast, Li and Zhou [16] established a camera parameters model to calculate the distances between these points and camera. The maximum distance is visibility value. Based on region growing algorithm, Zhang et al. [17] figured out the feature points in precise road region. The contrast curve of these points could reflect road luminance variation. Moreover, they found the maximum distinguishable pixels to calculate the maximum visibility distance.
For dual differential luminance algorithm, by setting two targets with different distances from observation station, Lv et al. [18] deduced daytime visibility according to the ratio of two measured luminance differences between targets and their horizon sky background. Chang et al. [19] calculated daylight visibility by dual differential luminance algorithm and gained night time visibility with double light sources. These algorithms could solve the problem that precision was affected by uneven distribution of sky brightness. Zhao et al. [20] proposed optimization dual brightness difference algorithm with correction coefficient and multimode division method, and she developed a special visibility detection system.
Based on statistical analysis of outdoor fog-free images, the concept of DCP was put forward by He et al. [21]. In the vast majority of local area of outdoor fog-free images, at least one color channel intensity value of some pixels is very low. When their minimum intensity values are close to zero, these pixels are known as the dark pixels. The DCP algorithm is widely used in image enhancement and image defogging for calculating the transmittance [22]. Guo et al. [23] obtained transmittance from building to camera by using DCP and guided filter. And then, they calculated extinction coefficient based on Lambert-Beer law and got visibility according to the relationship between transmittance and extinction coefficient.
Comparative analysis of the advantages and disadvantages of the method described above is as follows:(1)The principle of template matching method is simple. However, this method requires a lot of images calibrated with visibility information as a matching database and is not suitable for practical applications.(2)Using camera model calibration method, one can directly consider road as object to determine the extinction coefficient through camera model calibration and real-time graphics processing procedures. Then, the atmospheric visibility can be calculated with Koschmieder’s law. The advantages of this method are low cost, high stability, and high precision for moving vehicles and fixed cameras. However, self-calibration model is complicated and easily disturbed by weather and environment. Also, this method is difficult to work at night and cannot meet the demand for all-weather real-time detection.(3)Dual brightness difference method has great advantages in night visibility detection and meets the demand for all-weather visibility detection with better reliability, but the detection conditions of this method are high. Therefore, the additional artificial target should be set up. So it is not insufficient for highway application.(4)DCP method combines the advantages of digital imaging method and the transmittance method. The distinguishing features are simple principle and low cost. However it is still in theoretical stage and requires higher accuracy during the calculation of transmittance.
On the majority of Chinese highways, a video surveillance camera is implemented every 2 Km along the side of the shoulder. These devices are used to monitor traffic flow and traffic environment and can provide a wealth of video resources for the development of video detection of low visibility. Taking into account the advantages of DCP method, we attempt to apply this method to detect highway visibility. According to assumptions that there are some dark pixels in local area of image, DCP calculates the transmittance. However, when this method is applied directly to the highway visibility detection, large errors are produced due to the complexity of real highway condition. Therefore, an optimized DCP algorithm is presented which combines region growing, correction coefficient and multimode detection method to obtain more accurate transmittance.
2. Visibility Detection Principle of Dark Channel Prior Method
The digital image is defined as follows in computer vision and digital picture processing:where is image intensity, is actual scene intensity, is atmosphere light intensity, is the pixel of digital image, and is transmittance describing the proportion of the light that is reflected from an object and transmitted directly to camera without scattering.
2.1. Theoretical Basis of Dark Channel Prior
According to the literature [23], the corresponding dark channel of dark pixel is given inwhere is an arbitrary channel in RGB color space, is pixel value of channel at pixel , and is a local patch of which center pixel is .
Based on DCP, the transmittance of each pixel can be calculated. Main steps are shown as follows.
Step 1. For the channel , assuming that the transmittance of each pixel in the patch is the same as a constant , the minimum value of pitch is taken as formula
Step 2. Calculate the minimum value of three channels:
Step 3. Without considering DCP, based on the atmosphere light regularity of , can be described as formula
Step 4. According to DCP, formula (4) can be presented as follows:
Step 5. The estimated transmittance of each pixel in an image is obtained as formula
2.2. Visibility Detection Principle with Dark Channel Prior Algorithm
The essence of DCP algorithm is to calculate the transmittance of image pixel, and based on the physical relationship between transmittance and extinction coefficient, the latter can be obtained; therefore, the visibility value is carried out easily.
Bouguer-Lambert law describes the relationship between incident light flux and transmittance light flux , under the condition of a beam of light source transmits from a certain distance . Thus, mathematical relationship between atmospheric transmittance and meteorological optical visual range can be deduced. This law provides a theoretical basis for visibility detection with the method of DCP.
Bouguer-Lambert law is expressed as follows:where is the transmittance with light propagation distance and is the atmospheric extinction coefficient.
The transmittance can be derived as follows:
According to the definition of in meteorological optical range, select for Bouguer-Lambert law; there is
Thus, the visibility can be obtained:
It is worth noting that we should choose the objects which contain dark channel when using DCP to detect visibility. Then the extinction coefficient can be deduced based on the accurate transmittance of this region. When the transmittance is 0.05, we can calculate the meteorological optical range, namely, the atmospheric visibility.
As shown in Figure 1, video camera, variable message sign, and traffic sign are generally installed along the side of Chinese highway. Firstly, a fixed marker, such as signs, is selected as the target in the camera image collection range. Secondly, then based on DCP, camera images are used to detect region visibility. Thirdly, this visibility information will be displayed on the upstream variable message sign. Thus, the drives in the downstream section can be able to perceive the visibility information in advance. The right device in Figure 1 is the visibility detector whose data will be used to compare with the detection results of DCP method in this paper.
2.3. Error Analysis and Optimization of Dark Channel Prior
Because of the unpredictable environment in different highway segments, when DCP method is directly used to detect visibility, great number of errors will be generated. With preliminary analysis, the main errors come from sky area transmittance calculation [24], different atmosphere brightness values [25], and image block effect [21]. Therefore, we need to make the appropriate optimization algorithm for different error sources as our first step.
2.3.1. Error Analysis and Optimization in Sky Region
Based on the assumption that there are dark pixels in local image region, transmittance can be calculated by DCP algorithm. Video camera images can be divided into sky region and nonsky region. However, as the sky region does not conform to this assumption, a greater deviation in the calculation of transmittance will be generated [24].
When dark channel is not considered, the actual transmittance is calculated by formula (5). In the regions without dark channel, such as sky region, there is . So the actual transmittance is larger than the estimated one calculated by formula (7). In contrast, in the dark channel area, as is close to zero, the difference between actual transmittances and estimated one is very small.
Analyzing pixel characteristics of the sky region, the gray value changes are small and different from that of other pixels of nonsky region. By researching literatures [26–28], region growing method gathers the pixels based on similar characteristics in the same pixel area. One region starts from an initial pixel. And these pixels with same characteristics of the initial pixel will be incorporated into with this region. So this region gradually grows until there are no other pixels can be merged. Thus, during the growth process, region growing method requires a group of proper seed pixel in the desired area and precise guidelines for merging adjacent pixels. When seed pixel growth and appropriate guidelines are selected, region growing method has obvious advantages in computation time and segmentation accuracy. Therefore, we use region growing method to divide image into sky region and nonsky region.
(1) Seed Selection. The seed selection directly influences the effect of region growing. In the image, the brightness of sky region is higher than that of nonsky region. So the maximum brightness pixel in dark channel image is used as seed pixel. Seed pixel is selected as formula
(2) Growth Rules Based on Region Gray Scale Difference. According to the analysis of normal gray images, sky region has relatively higher brightness than nonsky region. The gray scale difference is set as growth criterion. When the gray scale difference between seed pixel and its neighbor is less than a predetermined threshold value, the latter will be merged into sky region. Then, the merged pixel is selected as a new starting point, comparison and merging will continue pixel by pixel, until the region can no longer expand. The grayscale values of sky region are greater than that of nonsky region and the grayscale changes slowly. When it meets the following constraints, the region is sky; otherwise, it is the nonsky region:where is grayscale threshold in the gray image and is grayscale threshold in the dark channel image. Due to the fact that white cloud regions do not meet the growth rules of sky and have no dark channels, is used as threshold to distinguish these regions. Therefore, the pixels of these regions meet formula (14) and will be merged into sky region:
According to literatures [26–28], both ranges of and are from 0 to 0.1. The range of is from 0.5 to 1. In this paper, based on a large number of video image sample tests, there are and . By using the method described above, original image and regional segmented image are shown in Figure 2.
(a)
(b)
Next, two correction coefficients and are introduced to realize accurate transmittance calculation in sky region, where and is the correction factor for atmospheric brightness. In nonsky region, considering the existence of dark channels, we directly use a correction parameter to revise transmittance. Transmittance corrections are calculated as follows:
2.3.2. Transmittance Optimization Based on Multimode Detection
When light transmits in the atmosphere, various factors cause energy attenuation with different degrees. In the same scene, different fog concentrations correspond to different atmospheric brightness, if the same correction factor is selected under different conditions that will result in an inaccuracy in transmittance calculation [29]. Therefore, based on the visibility level cited from the standard of “Monitoring of Visibility and Warning of Heavy Fog on Highway’’ [30], the visibility detection is divided into five modes as shown in Table 1.
For each mode, the difference between estimated visibility and actual visibility is considered as the objective function. By using the least square method, different correction coefficients of transmittance calculation are derived for sky region and nonsky region.
In sky region, the objective function and transmittance correction function in mode are established as formula in which is actual visibility. is estimated visibility when the transmittance is . and stand for correction parameters of sky region in mode , respectively.
In nonsky region, the objective function model and transmittance correction function in mode are set up as formula where is the correction parameter of nonsky region in model .
2.3.3. Transmittance Optimization Based on Guided Filtering
When the sizes of local region are different, the dark channel images are different. The larger the patch is, the greater the potential dark channel is, and the more obvious the block effect will be. However, significant block effect influences the accurate calculation of transmittance. As shown in Figure 3, in the dark channel image, block effect of 9 × 9 patches is more obvious than that of 3 × 3 patches.
(a)
(b)
(c)
Guided filtering theory was proposed by He et al. [31] and applied to eliminate block effect of transmittance calculation in DCP method. This theory is based on local linear model. It assumes that image is a two-dimensional function, and the output and input in a two-dimensional window have a linear relation as formula where is the value of output pixel, is the value of guidance image, and and are pixel indexes. is a window which includes pixel and is the center of . and are two coefficients of linear function in .
In order to obtain the linear function coefficients, the following cost function is used:where is a regularization parameter. Linear regression is used to minimize the difference between output value and actual value . The linear function coefficients are obtained as follows:where and are the mean and variance of in , is the number of pixels in , and is the mean of in .
A pixel may be contained in many windows; that is, each pixel can be described by a number of linear functions. Therefore, when calculating the linear coefficients of each window, the output value of a certain pixel is the average of all the linear functions which contains this point. There is
Therefore, when Figure 3(a) is processed according to the transmittance correction functions (16) and (17), Figure 4(a) can be obtained. After Figure 4(a) is processed with guided filtering optimization method, Figure 4(b) can be obtained. Comparisons of results of estimated transmittance are shown in Figure 4. After optimization, the block effect caused by the size of window can be eliminated, and the edge feature information is well-preserved.
(a)
(b)
2.4. Visibility Detection Processes Based on Dark Channel Prior
In summary, images captured by highway video surveillance systems are colored; the first step is to convert these images to dark channel images, which will be segmented into sky region and nonsky region based on region growing method. Secondly, in accordance with the appropriate detection mode, different factors are selected for correction. In order to improve the accuracy in transmittance calculation, guided filtering is used to eliminate block effect. Then, atmospheric brightness and target region transmittance are extracted. Finally, the Lambert law is utilized to calculate extinction coefficient and visibility value.
3. Precision Analysis of Visibility Detection Based on Dark Channel Prior
3.1. Contrast Test Analysis of Visibility Detection
To verify the accuracy of the aforementioned detection algorithms, as shown in Figure 5, a comparative test system is built based on the traffic test site in Beijing, China [32]. This test system includes a visibility detector, a video camera, and a traffic sign of which distance is 105 meters away from camera. The visibility detector model is VAISALA PWD 12 of which range is from 10 to 2000 meters. And this detector can not only measure accurate visibility but also display the reason for visibility decreasing. Due to the fact that traffic sign regions contain dark channel, this region is selected as the target. In addition, due to lack of light, the current highway camera images cannot be used to detect visibility at night. A light source is installed near by the traffic sign.
Based on the video image database captured from 01/09/2016 to 05/30/2016, an image was intercepted for analysis every ten minutes, and all the parameters of DCP algorithm were carefully verified and validated.
By using the daytime image captured on 03/18/2016, the visibility detection process of DCP is shown as Figure 6. The test system image is displayed in Figure 6(a). When the local patch size was set to 3 × 3, the dark channel image is shown in Figure 6(b). Estimated transmittance map before optimization is shown in Figure 6(c). Estimated transmittance map after optimization is shown in Figure 6(d). The visibility detection value is 956 meters. Simultaneously, the visibility detector value is 974 meters. The relative error is 1.85%.
(a)
(b)
(c)
(d)
By using the nighttime image captured on 03/10/2016, the visibility detection process of DCP is shown as Figure 7. The test system image is displayed in Figure 7(a). When the local patch size was set to 3 × 3, the dark channel image is shown in Figure 7(b). Estimated transmittance map before optimization is shown in Figure 7(c). Estimated transmittance map after optimization is shown in Figure 7(d). The visibility detection value is 732 meters. Simultaneously, the visibility detector value is 765 meters. The relative error is 4.31%.
(a)
(b)
(c)
(d)
One-day video image sequence was selected on 01/10/2016 as a validation. The visibility comparison results and relative errors of test system are shown in Figures 8 and 9. From two figures, the visibility results of optimization algorithm are consistent with that of the visibility detector in both trend and the numerical value. The relative error range is from 0% to 12.3% which meets the demands of highway visibility detection.
3.2. Verification of Dark Channel Prior Algorithm under Five Models
To further verify the effectiveness of the DCP algorithm, a large number of video images, captured by the camera in different locations, were selected to verify the validation of dark channel optimization algorithm under five detection models.
(1) The Verification Results under the First Model. The original image came from Jinliwen Highway of Zhejiang province, China. The time is 5:00 p.m. on 12/05/2013. The traffic sign was selected as target which is 72 meters away from the camera. The visibility detection process of the first model is shown in Figure 10. The visibility detection value is 769.05 meters which are consistent with the day weather record of traffic monitoring center.
(a)
(b)
(c)
(d)
(2) The Verification Results under the Second Model. The original image came from Jinggangao Highway of Hunan province, China. The time is 9:36 a.m. on 01/17/2015. The traffic sign was selected as target which is 31.6 meters away from the camera. The visibility detection process of the second model is shown in Figure 11. The visibility detection value is 236.82 meters which is consistent with the day weather record of traffic monitoring center.
(a)
(b)
(c)
(d)
(3) The Verification Results under the Third Model. The original image came from Rongwu Highway of Shandong province, China. The time is 10:15 a.m. on 02/01/2014. The bridge was selected as target which is 109 meters away from the camera. The visibility detection process of the third model is shown in Figure 12. The visibility detection value is 149.96 meters which are consistent with the day weather record of traffic monitoring center.
(a)
(b)
(c)
(d)
(4) The Verification Results under the Fourth Model. The original image came from Chengyu Highway of Sichuan province, China. The time is 9:40 a.m. on 10/21/2015. The bridge was selected as target which is 61 meters away from the camera. The visibility detection process of the fourth model is shown in Figure 13. The visibility detection value is 81.51 meters which are consistent with the day weather record of traffic monitoring center.
(a)
(b)
(c)
(d)
(5) The Verification Results under the Fifth Model. The original image came from Tuwu Highway of Jilin province, China. The time is 8:00 a.m. on 10/21/2013. The traffic sign was selected as target which is 29.8 meters away from the camera. The visibility detection process of the fifth model is shown in Figure 14. The visibility detection value is 46.17 meters which is consistent with the day weather record of traffic monitoring center.
(a)
(b)
(c)
(d)
3.3. Verification of Dark Channel Prior Algorithm under the Typical Tunnel
The original images were selected from Wangwu Tunnel of Henan province, China. Images were sampled every minute from continuous video streams of three days.
As shown in Figure 15, the sample times of six images are 9:15, 9:16, 9:17, 9:18, 9:19, and 9:20 on 01/23/2014. And the traffic sign was selected as target which was 88 meters away from the camera. As shown in Figure 16, the sample time of six images is 8:47, 8:48, 8:49, 8:50, 8:51, and 8:52 on 01/31/2014. As shown in Figure 17, the sample time of six images is 8:58, 8:59, 9:00, 9:01, 9:02, and 9:03 on 02/03/2014.
When the local patch size was set to 3 × 3, the dark channel images in three conditions are shown in Figures 18, 19, and 20.
Before optimization, the estimated transmittance maps of the tunnel are shown as Figures 21, 22, and 23.
The estimated transmittance maps of tunnel after optimization of three days are shown in Figures 24, 25, and 26.
By calculating the transmittance, the visibility values in three days are obtained, as shown in Figure 27. According to these figures, low visibility condition occurred on 01/31/2014; middle visibility condition occurred on 02/03/2014; high visibility condition occurred on 01/23/2014.
In a certain visibility condition, the actual visibility varied slowly in a short time interval. From Figure 27, visibility is relatively stable in six minutes which is consistent with visibility change rule. Corresponding to the original images, visual visibilities of three cases are from low to high sequentially. Detection visibility by DCP optimization algorithm has the same trend. The test results show that this detecting method and system were feasible.
3.4. Verification of Dark Channel Prior Algorithm under a Mountain Highway
The video image sequences were selected from Xuanda Highway in Hebei province, China. This is a mountain highway. As shown in Figure 28, the times of images are 13:07:17, 13:07:23, 13:26:14, and 13:57:45 on 01/25/2015. Because of the low visibility and snowy weather, an accident occurred during this time. The cabin area of the red truck is selected as target area which is 42 meters away from the camera. The dark channel images are shown in Figure 29. The estimated transmittance maps before optimization and after optimization are separately shown in Figures 30 and 31.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
The eliminating degree of block effect in Figure 31 is better than the one in Figure 30. From the original images of Figure 28, fog gradually diminished over time, and the visual visibility changes from low to high. As shown in Figure 32, the detection visibility values based on DCP optimization algorithm are gradually increasing and fully agree with the actual situation.
4. Conclusions
Low visibility is a major factor of causing of traffic accidents, which often result in a heavy loss of human life and financial properties. The accurate detection of visibility and timely warning play important roles in assuring the highway operation smoothness. In order to make full use of the camera image resources on highways, after summarizing the advantages and disadvantages of template matching method, camera calibration method, and dual differential luminance algorithm, a new method is elucidated to detect highway visibility by using the method of DCP.
The region growing method is used to divide the sky and the nonsky region. Due to the fact that sky regions do not meet the assumptions of DCP method, different correction factors are adopted to decrease the deviations derived from transmittance calculation. According to different visibility conditions, the multimode division method is designed, which can reduce the errors from extracting atmospheric brightness values. Guided filtering method is used to eliminate the block effect and to obtain the accurate transmittance of pixels in those images. Thus, based on the theoretical relationship between transmittance and extinction coefficient in Lambert-Beer law, the accurate visibility values can be calculated.
For verifying the feasibility, a comparative system was developed to test the visibility detection accuracy of optimization algorithm. Also, a large number of highway video images were used to verify the validation of this proposed optimization algorithm under five models. The verification results of video image sequences of tunnel and mountain highway indicate that the detection values are reliable and consistent with the actual situation. Based on a number of experiments, these optimization algorithms can well satisfy the general requirements of low visibility detection on the highway. Meanwhile, we found from those researches that it still needs further analysis to select the target area precisely and automatically in complex highway environments.
Competing Interests
The authors declare that there are no competing interests regarding the publication of this paper.
Authors’ Contributions
Jiandong Zhao and Mingmin Han presented the algorithms, analyzed the data, and cowrote the paper; Changcheng Li and Xin Xin installed the experiment system and performed the experiments.
Acknowledgments
This work is supported by “the Fundamental Research Funds for the Central Universities (2016JBM053).”