Abstract
Candidate smoke region segmentation is the key link of smoke video detection; an effective and prompt method of candidate smoke region segmentation plays a significant role in a smoke recognition system. However, the interference of heavy fog and smoke-color moving objects greatly degrades the recognition accuracy. In this paper, a novel method of candidate smoke region segmentation based on rough set theory is presented. First, Kalman filtering is used to update video background in order to exclude the interference of static smoke-color objects, such as blue sky. Second, in RGB color space smoke regions are segmented by defining the upper approximation, lower approximation, and roughness of smoke-color distribution. Finally, in HSV color space small smoke regions are merged by the definition of equivalence relation so as to distinguish smoke images from heavy fog images in terms of component value variety from center to edge of smoke region. The experimental results on smoke region segmentation demonstrated the effectiveness and usefulness of the proposed scheme.
1. Introduction
During one early fire, smoke is the main visual phenomenon of video surveillance image. Therefore, the research of smoke video recognition is particularly important to detect early fire. Most of smoke detection algorithms proceed as in the following steps: candidate smoke region segmentation, feature extraction, and smoke recognition. In this paper, we study the first step (i.e., candidate smoke region segmentation). At present, candidate smoke region segmentation methods generally can be divided into two categories: one is based on color features, such as RGB color space [1], HSV color space [2], and the combination of RGB color space and HSV color space [3–5]. Another is based on color features and moving features. Kim et al. [6] got the counter of smoke region with two-dimensional wavelet transform. Qin and Ma [7] extracted smoke regions of interest by static saliency image and moving saliency image. Wu et al. [8] established a background model using mixed Gaussian model in RGB space and then suspected that smoke regions are detected by comparing the current frame with this reference model. However, the first kind of method cannot detect smoke of small area and low concentration because moving characteristics are not considered. And the methods cannot distinguish smoke from heavy fog and blue sky.
Rough set can realize the representation and inference of the uncertain knowledge [9]. The theories of approximation representation and equivalence relation were widely used for image processing and pattern recognition [10–12]. This paper presents a novel candidate smoke region segmentation method based on rough set theory. Figure 1 shows the basic process of the proposed scheme. Above all, video backgrounds are automatically updated based on three-frame difference approach to avoid the interference of static smoke-color objects such as blue sky. Afterwards, the upper approximation, lower approximation, and roughness of smoke-color distribution are represented using rough set theory in order to exclude moving regions of non-smoke-color objects, such as leaves and flowers. Finally, smoke regions are initially segmented by gray wave brand of , , basic color features. However, superfine partition can lead to small regions since excessive wave peaks are generated by initial segmentation. Thus these similar regions are merged based on equivalence relation of rough set theory.
2. Moving Pixels Detection
Kalman filtering is used to update video background [13]. Let and denote the value of the pixel of background image and background image, respectively, and let denote the value of the pixel of frame; then the recursive formula is as follows:where denotes gain factor and is computed using the following formula:where the parameters and are the moving factor and the background factor, respectively. The pixel of the frame is marked as moving pixel if , and is the threshold. The method based on Kalman filtering is a common method for moving pixel detection. The selection method of the parameters , , and is described in [13]. In this paper, the moving factor is set to , and the background factor is set to [13].
3. Candidate Smoke Regions Segmentation
There are a mass of non-smoke-color moving objects such as trees and flowers in a surveillance video. In this section roughness measure approach is used to distinguish smoke-color moving regions from non-smoke-color moving regions.
3.1. Suspected Smoke Pixels Detection
Because fire smoke color meets in RGB color space and in HSV color space, the intensity value meets . Therefore, suspected smoke pixels are detected by the following two rules. Rule 1: . Rule 2: ,where the maximum value and the minimum value . The threshold is set to since fire smoke color meets . Thus the pixels that meet Rules 1 and 2 will be detected as suspected smoke pixels.
3.2. Approximation Representation of Color Distribution
In this section, we deal with pixels generated in suspected smoke pixels detection. That is, only suspected smoke pixels are checked in terms of rough set theory. Let the image denote an RGB image of the size and denote gray level; then approximation representations of the image can be defined based on rough set theory. The lower approximation of color distribution is defined as [11, 14]where and . is the gray value of the pixel on , , component and is impulse function.
The pixels that meet smoke color feature will be used to compute . According to the definitions, the lower approximation represents the number of smoke-color pixels whose gray level is ; thus is the certainty measure of gray distribution on , , component.
Let be the neighborhood of the pixel and the pixel ; then the similarity of the pixels and is defined as [10, 14]where is the Euclidean distance of and , . The homogeneity degree function is defined as follows:The upper approximation of color distribution is defined aswhere and , and represents the uncertainty measure of gray distribution of smoke-color pixels whose gray level is on , , component. Obviously, the approximations and satisfy the properties of rough set theory. Therefore, the roughness of smoke-color distribution is defined as [11, 12]where and . If the roughness is larger, the pixels of the gray level locate in the regions of less pixel variety, that is, central region. Otherwise, the pixels of the gray level locate in the edge of region. Figure 2 shows the roughness representation of one smoke video frame.
(a) Original image
(b) Roughness representation
3.3. Candidate Smoke Region Segmentation of Fire Video
The smoke region can be initially segmented by roughness of smoke-color distribution defined in Section 2. Then similarity of two small regions is defined on the basis of equivalence relation theory in order to merge similar regions.
3.3.1. Initial Segmentation of Candidate Smoke Region
Smoke regions can be segmented by gray wave brands that are formed by wave peaks and valleys. Therefore, the selection of peak and valley is important to increase the performance of smoke region extraction. Let be one peak of gray level ; then the peak will be reserved if it meets the condition , where and are mean value and variance, respectively. Thus one new peak sequence is generated, where denotes gray level. Then peak values of gray wave brand are selected as follows [12, 14]:where is the threshold of peak width. The peak location can be gained on the basis of . Wave valley location can be easily gained by computing the maximum between two adjacent peak values. Wave peaks and wave valleys come into being gray wave brands that are used to segment smoke region. The results of initial segmentation are shown in Figure 3. As seen from Figure 3, one smoke region is divided into some small clusters of smoke pixels. Smoke pixels of each cluster have similar roughness of smoke-color distribution.
(a) Original image
(b) Results of initial segmentation
3.3.2. Merging Small Smoke Region
Although smoke region can be segmented using the proposed method, the method often results in superfine partition. Furthermore, the method cannot distinguish smoke images from heavy fog images. Fortunately, smoke has one characteristic that is different from heavy fog. Smoke concentration gradually decreases from centre to edge of smoke region, which results in the fact that component value of smoke region gradually reduces in HSV color space. Therefore, in this section small smoke regions are merged in HSV color space. Initial equivalence relation of one small smoke region is defined aswhere is discourse domain and , . is the threshold of smoke region similarity. The threshold is set to . is computed as follows:Let and be two initial equivalence relations, and then generalized equivalence relation is defined aswhere , . is the threshold of equivalence relation similarity. The selection of the threshold is discussed in Section 4.2.1. is computed as follows:where denotes the number of the objects in the intersection , and denotes the number of the objects in the union set . Namely, the similarity of two smoke regions generated in Section 3.3.1 can be computed by the formula (12). If the similarity , two small smoke regions and are merged into one larger smoke region. Figure 4 shows the results after merging small smoke region. As seen from Figure 4, some small clusters of smoke pixels are merged. Five small regions of the image at the top right of Figure 3(b) are merged into three clusters. Three small regions of the image at the bottom right of Figure 3(b) are merged into two clusters. We can exclude many smoke-color objects according to the characteristic that component value of smoke pixels in HSV color space reduces from center to edge of smoke region. If one candidate smoke region is divided into 2~3 clusters and the component value of the smoke region decreases from center to edge, it is decided as smoke region. Otherwise, it is detected as non-smoke regions.
(a)
(b)
4. Experimental Result and Analysis
The proposed method of candidate smoke region segmentation is performed with an image size of . In the experiments, a total of 2,304 video frames from 16 fire video clips are used to verify the performance of the proposed method, including 326 positive video frames and 129 negative video frames. This section compares our method with other two methods: smoke regions detected based on RGB and HSV color space proposed in [3] (RGB-HSV for short) and smoke regions obtained using regions of interest extraction method based on vision saliency proposed in [7] (ROI for short).
4.1. Initial Smoke Region Segmentation
4.1.1. Low Concentration Smoke Region Detection
For one smoke image of low concentration, smoke region is difficultly segmented not only since it is very small, but also since the color difference between it and background is not obvious. For example, RGB-HSV method cannot detect the small smoke region of low concentration. Figure 5 shows the results of candidate smoke region extraction using our method and ROI method. As shown in Figure 5, our method and ROI method can effectively segment smoke region of low concentration.
4.1.2. Detecting Smoke Region of Images including Interference Objects
Color features of some interference objects in smoke images are similar to smoke region; thus these objects may be falsely detected as smoke and result in the fact that smoke regions cannot be segmented. Figure 6 shows experimental results of candidate smoke region extraction using our method, RGB-HSV method, and ROI method. As shown in Figure 6, our method and ROI method can truly segment smoke region of images including interference objects such as blue sky. RGB-HSV method falsely regards sky region as smoke.
4.2. Small Smoke Region Merging
4.2.1. Selection of the Threshold
The threshold is directly related to the performance of candidate smoke region detection. 60 positive images and 50 negative images are randomly selected from experimental dataset in order to select the threshold . Figure 7 shows average precision and recall with variety of the threshold . Precision and recall rates are computed as follows:As shown in Figure 7, when , both precision and recall are the highest. Small regions cannot be merged when the threshold is too large. All the regions can be merged into one smoke cluster when the threshold is too small. Therefore, too low threshold or too high threshold does not facilitate the segmentation of smoke regions using the characteristic that smoke concentration gradually decreases from centre to edge in HSV color space.
4.2.2. Excluding Heavy Fog Region
Color features of dense fog are very similar to those of smoke, and fog pixels are moving pixels. Therefore, it is a difficult problem distinguishing smoke from fog. Figure 8 shows component value variety of smoke region and heavy fog. As shown in Figure 8, the intensity of smoke region decreases from center to edge, and the intensity of fog pixels in one image is nearly the same. Therefore, we distinguish smoke from heavy fog using component variety of smoke region.
4.3. Performance of Smoke Region Extraction
Figure 9 shows average precision and recall of smoke region extraction using the three methods. It can be clearly seen from this figure that the proposed smoke region extraction using rough set theory outperforms the other two approaches in both precision and recall. Precision and recall of RGB-HSV method are the lowest since RGB-HSV method cannot detect some smoke images of low concentration including interference objects. Recall of ROI method is lower since ROI method falsely considers fog as smoke.
5. Conclusions
This paper presents a novel candidate smoke region extraction scheme of fire video, which adaptively updates video background and detects smoke regions in terms of roughness definition and equivalence relation definition of small smoke region using rough set theory. Experimental results show that the interference of static and moving smoke-color objects is excluded, and the proposed scheme outperforms the other methods in both precision and recall.
It is worth noting that although the performance of the proposed methods has been significantly improved, there are several potential works for future development. One is to amend the roughness of color distribution to effectively eliminate noise. Another is to adaptively select the threshold which contributes to merging small smoke region.
Conflict of Interests
The author declares that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This work was supported by the National Natural Science Fund (Grant no. 31200496).