Abstract
Smoke detection is a very key part of fire recognition in a forest fire surveillance video since the smoke produced by forest fires is visible much before the flames. The performance of smoke video detection algorithm is often influenced by some smoke-like objects such as heavy fog. This paper presents a novel forest fire smoke video detection based on spatiotemporal features and dynamic texture features. At first, Kalman filtering is used to segment candidate smoke regions. Then, candidate smoke region is divided into small blocks. Spatiotemporal energy feature of each block is extracted by computing the energy features of its 8-neighboring blocks in the current frame and its two adjacent frames. Flutter direction angle is computed by analyzing the centroid motion of the segmented regions in one candidate smoke video clip. Local Binary Motion Pattern (LBMP) is used to define dynamic texture features of smoke videos. Finally, smoke video is recognized by Adaboost algorithm. The experimental results show that the proposed method can effectively detect smoke image recorded from different scenes.
1. Introduction
Fires are a constant threat to forest ecological systems and human safety; moreover, forest fires are an important problem in regions which present hot climate. With the development of computer vision techniques, forest fire video surveillance has been one of research focuses in the field of forest fire prevention. Generally, methods for detecting fire video can be categorized as flame detection and smoke detection. However, most of the fire video systems are mainly designed for smoke detection, since the appearance of smoke is in most cases more visible than the fire itself. Smoke detection algorithms are divided into systems based on single and based on multiple frames. In the first case, smoke images are recognized by color [1, 2], texture [3, 4], and energy [5]. In the second case, smoke features are extracted by video sequence. For example, Favorskaya and Levtin [6] extracted spatiotemporal features of smoke video by grouping moving regions with a turbulence parameter connecting with fractal properties of smoke in order to track effectively a smoke propagation. In [7], wavelet transform is used to detect high frequency information of moving pixels so that smoke flickering features are analyzed. Interesting work was presented by Wang et al. [8] who firstly detected motion regions from video frames, and then four flutter features of the motion regions are extracted over a sliding time window, including the flutter direction and three types of flutter intensities.
Smoke detection systems have made some achievements but cannot be used as a self-sufficient solution. They often have a high false rate and thus need an additional human confirmation for final decision. For bringing the performance of the detection systems closer to the results that could currently be obtained by human observers, this paper presents a novel forest fire smoke video detection based on spatiotemporal energy and dynamic texture features. First of all, candidate smoke regions are segmented and then are divided into small blocks. Afterwards, for each block, three features including two spatiotemporal features and one dynamic texture feature are extracted. Finally, Adaboost algorithm is used to recognize smoke video clips. Figure 1 shows the basic process of the proposed scheme.
2. Spatiotemporal Feature Extraction
Kalman filtering is firstly used to update video background [8] in order to detect motion regions. And then three features of early smoke of forest fire are extracted in terms of flutter analysis, energy analysis, and color analysis. Motion regions in terms of Section 2.1 are divided into blocks of the size 8 × 8. Then smoke features are extracted from the candidate smoke blocks.
2.1. Spatiotemporal Consistency Feature
When smoke diffuses in the scene, it covers part of the scene. The edges of one smoke region are blurred and high frequency information slowly changes. Letting and , respectively, denote the high frequency energy of corresponding background and current frame with the block , then square difference of high frequency energy is defined asA smoke block is detected as candidate smoke block according to the following condition:where represents the mean of the square difference of the previous frames.
For a smoke block, its spatially and temporally neighboring blocks have greater possibility of being smoke blocks. We count the number of neighboring smoke blocks of the block of the video frame at the time :where denote the number of 8-neighboring smoke blocks at the corresponding position of the video frame at the time . Figure 2 shows the calculation process of the spatiotemporal consistency energy.
2.2. Flutter Feature
As we know, there is massive heavy fog in the forest, so heavy fog can easily cause false alarm. Fortunately, forest fire smoke has one characteristic that is different from heavy fog. Smoke moves from bottom to top because heat smoke has a lower density than air, which is significantly different from heavy fog. The motion direction of one block is mapped as a direction code by computing the centroid motion of one candidate smoke block. The order moment of one candidate smoke block is computed by the following formula:where and denote starting and ending point of one smoke region width, respectively, and and denote starting and ending point of the region height. The centroid coordinates of one candidate smoke block are defined aswhere , denotes one frame rate. and denote horizontal ordinate and vertical ordinate of the centroid of the th candidate smoke block in th time window. The moving direction angle of the th candidate smoke blockFlutter direction angle is defined as
2.3. Dynamic Texture Feature
The Local Binary Pattern (LBP) is a powerful means of texture description. The operator labels the pixels of an image region by thresholding the neighborhood of each pixel with the center value and considering the result as a binary number (binary pattern). The classical definition of LBP can be represented as follows:where corresponds to the gray value of the pixel at position and to gray values of equally spaced pixels on a circle of radius with the center at position .
LBMP uses the basic Local Binary Pattern (LBP) to extract both dynamic and appearance features of dynamic texture of candidate smoke region. We choose one search window of the size 5 × 5. represents the search window in the current frame , and is its central pixel. Let denote the LBP descriptor of the pixel computed by (8); denote the LBP descriptor of one arbitrary pixel in the search window of adjacent frame . The matching point can be found by the formulawhere ranges from 1 to 25 due to one search window of the size 5 × 5. For one block of the size 8 × 8, the dynamic texture is defined aswhere is Kronecker’s delta that is defined as
3. An Adaboost Approach for Classification
3.1. Block-Based Statistic Characteristics
One image is divided into regular blocks of the fixed size . Block-based statistic characteristics are computed as [9]where is the number of blocks and and represents moving direction or energy lowering ratio or color feature of the th block.
3.2. Adaboost Algorithm
Adaboost is superior to tradition neural networks on learning abilities and is applied in many fields of image processing, such as car license plate detection and face recognition. Adaboost algorithm is a training procedure for a collection of weak classifiers [10]. If the weak classifiers have the success rate about 0.5, they are boosted by suitable voting process to obtain a strong classifier. Because the collection of fire smoke videos is relatively difficult, the number of negative samples is much more than positive [11].
Let the set of training samples be , where denotes training sample and denotes positive sample and negative sample, respectively. Suppose and denote the number of positive samples and the number of negative samples, respectively. The implementation details are as follows.
Step 1. The weight of the th sample is initialized by formula (14):
Step 2. Then, the weight of weak predictor is computed as follows:where denotes the sum of predictive errors.
Step 3. Supposing , , denote the predictive sequence, the iterative formula of the weight of training sample is computed bywhere denote the normalization factor.
Step 4. Let denote the number of weak predictors. All the weak classifiers form a strong classifier using the following formula:
4. Experimental Results
The proposed method is tested on 23 video clips of varying length and scene, including 13 positive samples and 10 negative samples, which are generally processed around 20 fps. 12 video clips (7 positive videos and 5 negative videos) are randomly chosen for training; the remaining 11 clips are used for testing. One part of the dataset is publicly available at website http://imagelab.ing.unimore.it/visor/ and another part of dataset is recorded by ourselves.
4.1. Segmentation of Smoke Regions
Smoke region segmentation is key part for extracting effective features of smoke region. Figure 3 shows the results of smoke region segmentation based on Kalman filter. In Figure 3, red points denote the centroids of smoke regions.
(a) Original images
(b) Segmented smoke regions
4.2. Centroid Motion of Smoke Regions
Figure 4 shows centroid motion direction of smoke regions in one video clip. As we see, smoke often moves from bottom to top, which is significantly different from heavy fog.
4.3. The Results of Smoke Detection
Figure 5 shows the results of several frames in three smoke videos from different scenes. Figures 5(a) and 5(b) show the results of the videos that are recorded by the camera from close distance and far distance, respectively. Figure 5(c) shows the results of the video that includes obviously moving objects.
(a) The video recorded by the camera from close distance
(b) The video recorded by the camera from far distance
(c) The video that includes obviously moving objects
4.4. Performance of Smoke Detection
A smoke strong classifier of four-layer cascaded architecture is established, in which the numbers of weak classifiers in each layer are 1, 26, 69, and 126. Cost factors of each layer are 60, 6, 4, and 2, respectively. To validate the performance of our method, this section compares the proposed smoke detection method with other two methods: smoke detection using image energy and color information in [5] (EN-CI for short) and smoke detection method based on mixed Gaussian model and wavelet transformation in [12] (MGM-WT for short). We evaluate the performance of smoke detection method by computing true positive rate () and true negative rate which are, respectively, defined byFigure 6 shows the performance of the three methods, including the proposed one, in terms of TPR and TNR, where different datasets were used. Both TPR and TNR of our method are higher than the other two methods, as shown in Figure 6. It indicates that dynamic features extracted by the proposed method are effective in discriminating between smoke and nonsmoke video.
(a) TPR of positive testing videos
(b) TNR of negative testing videos
5. Conclusions
In this paper, a novel smoke detection scheme using spatiotemporal and dynamic texture features is proposed. Three dynamic features, spatiotemporal energy, flutter features, and dynamic texture feature, are extracted for recognizing forest fire smoke video effectively and efficiently. The experimental results show the proposed approach provides higher accuracy of wildfire flame detection with comparable computational time. It is worth noting that there are several potential works for future development. One is to improve background model to effectively detect candidate smoke region in smoke-like scene; another is to extract more effective features for completely excluding the interference of heavy fog.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This work was supported by the National Natural Science Fund (Grant no. 31200496).