Abstract
Currently, video surveillance-based early fire smoke detection is crucial to the prevention of large fires and the protection of life and goods. To overcome the nighttime limitations of video smoke detection methods, a laser light can be projected into the monitored field of view, and the returning projected light section image can be analyzed to detect fire and/or smoke. If smoke appears within the monitoring zone created from the diffusion or scattering of light in the projected path, the camera sensor receives a corresponding signal. The successive processing steps of the proposed real-time algorithm use the spectral, diffusing, and scattering characteristics of the smoke-filled regions in the image sequences to register the position of possible smoke in a video. Characterization of smoke is carried out by a nonlinear classification method using a support vector machine, and this is applied to identify the potential fire/smoke location. Experimental results in a variety of nighttime conditions demonstrate that the proposed fire/smoke detection method can successfully and reliably detect fires by identifying the location of smoke.
1. Introduction
Video surveillance can be used to monitor fires and fire-related disasters. However, live surveillance systems require numerous people to monitor video screens continuously, leading to the possibility of human error; these systems also have other disadvantages including the need to store large amounts of data and the related high cost. Recently, more attention has been given to the detection of fire using surveillance cameras with machine vision which can detect smoke. Various researchers have proposed video fire/smoke detection methods based on signal processing and the related methods [1–8]. A video smoke detection technique comprising background subtraction, flickering extraction, contour initialization, and contour classification using both heuristic and empirical knowledge about smoke has been proposed [1]. The image processing approach involves the extraction of the smoke plume from the background using frame difference technologies. Image features such as motion, flickering, edge-blurring of moving segments, and edge-blurring of regions were extracted from the video to detect smoke. These features have been extracted using background subtraction, temporal wavelet transformation, and spatial wavelet transformation. Smoke pixels were analyzed with a chromaticity-based static decision rule and a diffusion-based dynamic characteristic decision rule. A video smoke detection technique comprises background subtraction, flickering extraction, contour initialization, and contour classification using both heuristic and empirical knowledge related to the characteristics of smoke [2]. To avoid false flame detection which can result from interference caused by background illumination, neon colors, or traffic lights, the time-varying property of flame geometry is taken into account. When segmenting smoke features, color processing has advantages over grayscale processing. Color processing can avoid the generation of false alarms caused by variations in lighting conditions, for example, natural background illumination, better than grayscale processing can. Further, a video camera is a volume sensor, and it can potentially monitor a larger area. However, many current video-based fire surveillance systems have a major weakness regarding the detection of smoke at night. Video-based camera systems can only sense variations in the indoor environment when sufficient lighting is illuminating the monitored region. Illumination at night is usually insufficient for smoke detection, and consequently smoke from a smoldering fire cannot be detected promptly.
To avoid the false flame detection caused by interference from background illumination and colored artificial light such as neon or traffic lights, the property of variation over time of flame geometry is taken into account [3]. In our previous work to detect fire flame and smoke using fuzzy logic and a continuously adaptive mean shift (CAMSHIFT) vision tracking algorithm was employed to provide feedback of the real-time position of fire flames and smoke using a high frame rate [2, 4]. To overcome the limitations related to illumination and the shortcomings of conventional smoke sensors, multisensor systems [5] and infrared cameras [6–8] have been used instead of regular visible-range cameras. Ruser and Magori [5] presented a fire detection method that used a combination of ultrasonic and microwave sensors to avoid false alarms. Liu et al. [6] presented a smoke recognition method based on active infrared charge-coupled device (CCD) video imagery. However, smoke detection using an active infrared camera to capture video at night is still difficult because of the high levels of background noise in the video. A false alarm for a fire may be caused by a variety of stimuli, such as a heater, sunlight, or a spotlight. Yuan et al. [7] proposed a fire detection method based on a plurality of infrared radiation arrays mounted in an area to detect smoke. Fang et al. [8] proposed the use of light section image detection to overcome the shortcomings of conventional beam-type smoke sensors designed to achieve early detection of fire/smoke. However, the drawback of these methods is that the detection system requires a setup involving many elements, such as infrared radiation arrays, infrared cameras, and signal processing units. Moreover, such systems are limited to monitoring only the flow of fire and/or smoke.
A fuzzy logic thresholding approach employed to provide the detection of nighttime fire/smoke in the earliest stages of a fire with a real-time alarm system was presented in our previous study [9]. However, the detection of fire signals will cause an alarm if the amount of smoke increases to the point that incident light scattering reaches a predetermined threshold, thus decreasing the reliability of the method in complex environments.
To address these problems, a smoke recognition method based on a color camera with active laser imagery that is able to sense smoke in a nighttime indoor environment under zero lux illumination is proposed. This paper introduces and analyzes current fire/smoke detection methods in Section 2. Section 3 then gives some experimental results and discussion. Finally, Section 4 provides the conclusions.
2. Nighttime Fire/Smoke Detection Using a Support Vector Machine (SVM)
To distinguish and classify candidate regions as fire/smoke or aliases, an SVM is used as a more sophisticated classifier than the fuzzy logic thresholding approach used in our previous research studies [9]. Support vector machines are data-driven and nonlinear approaches used for pattern classification problems which do not incorporate problem domain knowledge [10–14]. Support vector machines are supervised learning techniques, which are a relatively new class of learning machines. Furthermore, an SVM not only effectively learns complex relationships without heuristic feature parameters but is also suited to the use of a limit trained data set.
The main goal of SVM theory is to use kernel functions to map the training data into high-dimensional feature space. A hyperplane is then found in this feature space that maximizes the margin between categories. The constructed hyperplane can then be used as a basis for classifying vectors of unknown classification with the largest margin between classes. Given the training sample () that are input patterns and () where are the corresponding desired response. The equation of a decision surface in the form of a hyperplane that does the separation is where is an input vector, is a weight vector, and is a bias. The general form of the discriminant function is and gives an algebraic measurement of the distance from to the optimal hyperplane. To map the input vector to a high-dimensional feature space, the radial basis function (RBF) kernel is chosen as follows: where is the input feature vector and are the training samples. The symbol is the parameter that controls the width of RBF. In the proposed method, the feature vectors chosen were diffused laser light signal and scattered laser light signal from the extracted fire smoke candidate regions. These two features are then used as the input to the classifier.
To realize the production of an effective and efficient nighttime fire/smoke detection system, the proposed system is composed of a line-strip laser and a color CCD camera. The line-stripe laser emits light in the monitoring region of the camera. A general digital color video camera is used to capture several laser-projected smoke sample image sequences with a pixel resolution of 1,280 × 720. The fire/smoke detection process consists of four steps. First, the input color image is transformed into a YCbCr color space. YCbCr is a color format commonly used in digital video surveillance systems requiring data compression, where Y is the luminance, and Cb and Cr contain blue and red chrominance components. The YCbCr color space is applied instead of other color spaces because it can separate luminance from chrominance information. Experimentation has shown that the Y component apparently outperforms the other components as the diffusing and scattering laser signal received by the camera.
Image capturing and processing are the two major challenges involved in the construction of a laser light intensity measuring system. In the proposed fire/smoke detection system, the video signal capturing process is conducted using a low-cost CCD camera with a resolution of 1280 × 720 pixels. The captured synchronized video frames are then transmitted via USB to a personal computer. Then, the diffusing light intensity of the projected line-stripe laser is obtained by the camera. Next, the scattering light signal from the small and invisible-to-the-human-eye particles of fire/smoke is obtained using a color CCD camera. In the final step, the statistical distribution of the diffusing and scattering probability density is predicted by the SVM algorithm to determine the potential of actual fire/smoke as the result of the analysis.
3. Results and Discussion
Most fires in buildings start with a strong emission of fire and/or smoke. The particles and the air turbulence produced yield partial obstructions in the projecting laser section image, so the fire/smoke can be sensed. Smoke visibility refers to the attenuation of light or opacity along the line of sight, and the saturation will decrease when opacity increases. By relating the reduction in laser light intensity directly to smoke obscuration, the obscuring coefficient of the diffused laser light is defined as . The is represented as average pixel intensity values from a region of interest (ROI) window. A decrease in the light intensity of the sensed line-strip laser indicates that there is smoke in the region that is obscuring light from the line-strip laser, and the resulting light gradually becomes invisible to the camera. Smoke between the laser light source and the projected wall surface will decrease the quantity of incident light on the sensor. Light signal obscuration detection will cause an alarm if smoke particles block part of a laser light beam transmitted to a camera so that the laser light intensity decreases.
When particles of smoke from a fire interact with the laser beam, the light is scattered to the camera, whose light intensity response is a direct count of the number of particles [15]. The laser light scattered by the particles can be imaged to allow detection. The amount of smoke can be characterized by the scattered signal intensity received by the camera. Detection of scattered light signals will cause an alarm if the amount of smoke increases to the point where incident light scattering increases. The principle behind the operation of the proposed scattering light signal detection is to transform the camera surveillance area into a light scattering detection zone by directing a line-strip laser beam across it (Figure 1). Hence, the light scattered by particles of smoke from a fire in the beam can be used to calculate the smoke density detected by the camera. In order to have robust and continuous scattering feature values, scattering signals are accumulated in every frame. The particle sensing coefficient of the accumulative scattered laser light is defined as . The is represented as average pixel intensity values with corresponding masks having all nonzero values.
To create the proposed nighttime fire/smoke detection system, a general digital color video camera (Microsoft LifeCam HD-5000) was used to capture several fire/smoke sample image sequences, along with a harmless low-power laser light emitter (LT-B65100-GLD, 45 mW). The computer used in our system is a 2.5 GHz Pentium E5200 PC with 2 GB of RAM running Windows 7. The laser source emits a spatially modulated line-strip laser, and the laser beam is projected on a wall surface 8 meters from the camera. The camera receives radiant energy from the diffusing and scattering line-strip laser beam and processes it to detect the existence of a fire and/or smoke conditions. Ten additional classified positive fire/smoke detections are also required prior to a decision to separate smoke and smoke-like aliases. The proposed system can readily be made to work in complete darkness. The laser and camera are mounted on a tripod at a distance of 7 m from the burning smoke box.
To verify that the measuring system effectively collects smoke information, a smoke sensor (i.e., HS-135) of a semiconductor type based on tin dioxide (SnO2) was employed [16]. SnO2 sensors are widely used as a base material for commercial semiconductor gas sensors for detection of smoke. Smoke signals received by the HS-135 sensor are processed by the microcontroller, and then the microcontroller sends back numerical data to the computer via an RS232 serial link. The data acquired is recorded and synchronized with every video frame. The HS-135 sensor was installed above and near a smoke/burning test box at a height of 0.4 m above ground level. The fire/smoke was accurately classified by performing 10-fold cross-validation sets of training data (Tests 1–5).
The feasibility of the proposed approach was investigated with a series of eight test fires using different fuels. Figure 2 and Table 1 show the results of implementing the proposed fire smoke detection system under zero lux illumination (Tests 1–8). The zero lux intensity was confirmed using a digital lux meter (Testron Model TES-1332).
Figure 3 shows the results of evaluating the light obscuring and scattering characteristics of the tested images with no fire/smoke inside the experimental room. Since the line-strip laser is not obscured, the value of diffused laser light coefficient is close to a constant value. The employed off-the-shelf commercial laser diodes operate at a single wavelength. However, the wavelength is unstable because of fluctuations in the power supply and temperature [17]. Peak intensities vary as the wavelength varies and consequently the consecutive value fluctuates (Figure 3). Also, no scattered laser is sensed, and so the value of the particle sensing coefficient is zero. Fire/smoke information is verified by the reading of smoke concentration from the HS-135 sensor output (Figure 4).
In the second experiment, clothes are used as the burning material (Figure 5); the line-strip laser is obscured by smoke, and so the value of diffused laser light coefficient decreases. Also, particles of smoke in the view region are scattering light from the laser beam, appearing as a visible blob to the camera; hence, the value of the particle sensing coefficient increases. Since fire/smoke is turbulent and will fluctuate with periodic oscillations, the particle sensing coefficient also fluctuates periodically. The potential fire/smoke was detected in the 381th frame with the summed-score level crossing the threshold, and the smoke concentration is verified by the HS-135 sensor output (Figure 6).
Experiments 3, 4, 5, 6, and 7 were conducted with various burning materials (Figures 7, 8, 9, 10, 11, 12, 13, and 14); clearly, the fire/smoke can be correctly extracted and an appropriate alarm sounded. In experiments 6 and 7, the HS-135 sensor failed to sense the smoke produced by plastic bags and diesel oils. However, the proposed approach did successfully detect the fire/smoke produced by diesel oils and plastic bags in the 10th frame and 118th frame (Figures 13, and 14).
In the last experiment, water mist spray was selected for a false fire alarm test with fire/smoke being produced in the burn box. The line-strip laser is not obscured by small particles, and hence the value of the diffused laser light coefficient is close to a constant value. However, scattered laser light is sensed, and so the value of the particle sensing coefficient is not zero (Figure 15). Then, the values of and are classified by the SVM system, and no alarm was sounded during Test 8 for a nuisance fire scenario.
4. Conclusions
This paper presents an approach which can be used to detect fire and/or smoke during the earliest stages of a fire with a real-time alarm system. Diffused laser light, scattered laser light signals, and an SVM classifier are employed to extract and detect real fire smoke images from the background image and are adopted to help validate the accuracy of the fire/smoke detection. The results show that fire and/or smoke can be successfully detected and the proposed algorithm can be integrated into existing surveillance systems to achieve the detection of fire/smoke recorded in video databases, as well as the real-time detection of fire. Future work is planned to fuse multisensors to improve the ability of the system to minimize false alarms and to accurately classify burning material.
Acknowledgment
This research was partially supported by the National Science Council in Taiwan under Grant NSC 101-2221-E-224-008.