Abstract
Infrared images are fuzzy and noisy by nature; thus the segmentation of human targets in infrared images is a challenging task. In this paper, a fast thresholding method of infrared human images based on two-dimensional fuzzy Tsallis entropy is introduced. First, to address the fuzziness of infrared image, the fuzzy Tsallis entropy of objects and that of background are defined, respectively, according to probability partition principle. Next, this newly defined entropy is extended to two dimensions to make good use of spatial information to deal with the noise in infrared images, and correspondingly a fast computation method of two-dimensional fuzzy Tsallis entropy is put forward to reduce its computation complexity from to . Finally, the optimal parameters of fuzzy membership function are searched by shuffled frog-leaping algorithm following maximum entropy principle, and then the best threshold of an infrared human image is computed from the optimal parameters. Compared with typical entropy-based thresholding methods by experiments, the method presented in this paper is proved to be more efficient and robust.
1. Introduction
Image segmentation is an important topic in the field of digital image process. It intends to extract objects from background based on some pertinent characteristics in an image such as gray level, color, texture, and location [1]. Thresholding is one of the most popular segmentation approaches because of its simplicity [2, 3]. It serves a variety of applications such as biomedical image analysis, character identification, and change detection [3]. Compared with visible images, the intensity of human targets in infrared images is obviously different from that of background. Therefore, segmenting infrared human images by threshold selection is feasible [4]. Moreover, the intensity of human targets in infrared image is mainly determined by its temperature and radiated heat and is independent of the current light conditions, so the detection system can be applied indiscriminately in both day and night [4]. However, infrared images are not perfect either. Due to the limitations in camera technology, most infrared images have lower spatial resolution and less sensitivity than visible images, which often leads to poor image quality, such as blurring, low target-to-background contrast, and great noise. Therefore, it is a complex challenge to make precise segmentation of human targets in infrared images by threshold selection [5].
2. Related Work
Excellent reviews on early thresholding methods can be found in [6]. Among all the thresholding methods, entropy-based method is widely studied and is considered effectively [3]. Recently, Tsallis entropy was applied to the field of image thresholding for its excellent performance in the description of nonadditive information existing in images [7]. Later, this work in one-dimensional histogram was extended to two-dimensional one with much better effects [8]. Fuzzy sets play a significant role in many systems because of their capability to model nonstatistical imprecision. The notion of entropy, in the theory of fuzzy sets, was first introduced by De Luca and Termini [9]. There are numerous applications of fuzzy entropies in image segmentation. Cheng et al. proposed fuzzy homogeneity vectors to perform multilevel thresholding [10]. Tao et al. presented an approach to segmenting objects in infrared images based on ant colony optimization algorithm and fuzzy entropy [11]. Cheng et al. presented a thresholding approach by performing fuzzy partition of a two-dimensional histogram based on fuzzy relation [12]. Zhao et al. presented an entropy function based on fuzzy c-partition (FP) and probability partition (PP) [13] to perform image thresholding.
Considering the fuzzy nature of infrared image and the excellent performance of Tsallis entropy in the description of nonadditive information, in this paper, we present a fast infrared human image thresholding method based on two-dimensional fuzzy Tsallis entropy. We first transform Tsallis entropy into fuzzy domain and then extend it to two-dimensional histogram, and thus 2-D FTE (two-dimensional fuzzy Tsallis entropy) is defined and applied to the field of threshold selection. To handle the exponential increase in computation time brought about by extending one-dimensional method to two-dimensional one, we provide a fast algorithm to reduce the computation complexity of 2-D FTE from to for an image with gray levels. And then SHLA (shuffled frog-leaping algorithm) [14] is employed to speed up the search of optimal threshold. The comparison between the proposed method and other typical entropy-based methods demonstrates that the former is more effective and robust.
3. The Fuzzy Entropy of Images
3.1. The Fuzzy Set of Images
Let , , and ; then for an image of size with gray levels, its domain can be expressed as . Let and denote as a threshold by which an image is segmented into object and background . Then is an unknown probability partition of , and its probability distribution is described as , , respectively. For each , let then the following equations hold [11]: where and denote the conditional probability of a pixel that is classified into the classes and , respectively, obviously .
To threshold an image in fuzzy domain, we first transform the image into fuzzy domain by fuzzy membership function. Suppose the fuzzy memberships of a pixel with grey level belonging to classes and are and , respectively; then . We can decide the class of the pixel by the value of and . According to [11], let and by substituting (3) into (2), we get
We select -function and -function as the fuzzy membership functions because they are smoother than other fuzzy membership functions, such as Triangle function and -function. The standard -function and -function used in this paper are shown in the following:
3.2. The Fuzzy Tsallis Entropy of Images
In the domain of an image, let , in which denotes the cardinality of a set. Then the histogram of an image can be expressed as , and is transformed into fuzzy domain; that is, . Consequently, the fuzzy distribution of object and background can be obtained as where , and are defined in (4).
According to the definition of Tsallis entropy, we define the fuzzy Tsallis entropy of object and background , respectively, as follows: where and .
It can be proved that the limiting case of the fuzzy Tsallis entropy defined in this paper is equal to the fuzzy Shannon entropy defined by Tao et al. [11]. Take , for example, when ; then
Similarly, we can prove that , where and are the fuzzy Shannon entropy of object and background defined by Tao. Therefore, we can say that the fuzzy entropy defined by Tao is a special case of the fuzzy Tsallis entropy defined in this paper.
4. Fast Threshold Selection Based on 2D FTE
4.1. The Definition of 2-D FTE (Two-Dimensional Fuzzy Tsallis Entropy)
To define the 2D FTE (fuzzy Tsallis entropy) for an image of size with gray levels, we first calculate the 2-D histogram of the image by where denotes the number of pixels whose gray level equals and local average gray level equals [8]. The 2-D histogram can be viewed as a full Cartesian product of two sets and , where represents the gray levels and represents the local average gray levels; that is, . Each axis of the 2-D histogram can be transformed into fuzzy domain using -function and -function, and then four fuzzy sets are defined as follows [12]:
The fuzzy relation (object) and (background) are subsets of the full Cartesian product space , respectively; that is, , . The 2-D fuzzy membership functions and can be obtained as follows [12]:
Then mapping the 2-D histogram into fuzzy domain employing , obtained above, we get two fuzzy distributions of classes and as follows: By the similar way used in the deducing of (4), and can be computed by
According to the definition of two-dimensional Tsallis entropy [8], we extend the fuzzy Tsallis entropy defined in (7) to two-dimensional ones and define the 2-D FTE of classes and as follows:
According to the nonadditivity rule of Tsallis entropy, the total two-dimensional fuzzy Tsallis entropy of an image can be obtained by
We follow maximum entropy principle [8] to search for the optimal threshold by
4.2. Fast Computation of Two-Dimensional Fuzzy Tsallis Entropy
The proposed segmentation method involves the optimization of (16), and thus the 2-D FTE needs to be computed once for each pair of parameters according to (14)~(15). For an image with gray levels, the time complexity of this computation is , which is too much even for a heuristic search algorithm like SHLA. Therefore, we present a fast algorithm to speed up the calculation. We take the derivation of the fast computation of as an example here; the derivation of is similar. We divide the 2-D histogram of image into two parts by the diagonal line from to as illustrated in Figure 1.
For convenience, is denoted as hereafter. As -function is a monotonically nondecreasing function, from (11), we get In Figure 1, and ; then, we get
When the two-dimensional histogram is obtained, we first compute the four accumulation arrays, which just need to be done once for each input image: where means accumulating all the elements in to the upper side of the 2-D histogram along vertical lines like in Figure 1. In the similar manner, means accumulating all the elements in to the right side along horizontal lines like .
For and , we first compute for each element of the 2-D histogram and then accumulate them to the upper and right side in the same way as and .
For each pair of fuzzy parameters , we first calculate and for each . Then from (14) and (18), we get Substituting (19) into (21), we get In a similar manner, we get Substituting (22) and (23) into (7), we get
Therefore, the computation of for each pair of fuzzy parameters is sped up from to .
4.3. Thresholding Based on 2-D FTE and SHLA
To obtain the optimal threshold it is required to find the optimal combination of the fuzzy parameters. Therefore, the segmentation problem can be formulated as an optimization problem. SHLA [14] is an excellent global optimization algorithm proposed recently. It combines the benefits of the genetic-based memetic algorithm and the social behavior-based particle swarm optimization algorithms. In SFLA, the population consists of a set of frogs (solutions) that is partitioned into subsets referred to as memeplexes. The different memeplexes are considered as different cultures of frogs, each performing a local search. Within each memeplex, the individual frogs hold ideas that can be influenced by the ideas of other frogs and evolve through a process of memetic evolution. After a defined number of memetic evolution steps, ideas are passed among memeplexes in a shuffling process. The local search and the shuffling processes continue until defined convergence criteria are satisfied. Due to its rather fast convergence speed compared with other algorithm [15], SHLA is used to select the optimal fuzzy parameters in this paper. The major steps consist of the following.
Step 1. Calculate the 2-D histogram of the input image by (9) and produce four accumulating arrays by (19) and (20).
Step 2. Search the optimal fuzzy parameters by SHLA, in which the fitness function is calculated according to (15).
Step 3. Set the threshold to be the crossover point of the membership function which has membership 0.5 implying the largest fuzziness [12], and segment the input image.
5. Experiments and Discussions
In order to verify the validity of the presented method, it is tested on a variety of synthetic and real images. To make purposeful yet effective comparisons, the selected thresholding methods for comparison are all entropy-based thresholding methods. They are Tao’s 1D fuzzy Shannon entropy method [11], Sahoo’s 2-D nonfuzzy Tsallis entropy method [8], and Cheng’s 2-D fuzzy Shannon entropy method [12]. Additionally, the parameter in (15) is set to be 0.7 as Sahoo did in [8], and the parameters of SHLA are set to be the values recommended in [14].
In order to evaluate the performance of the competing methods objectively, we conduct quantitative analysis on the results. There is no universally accepted objective criterion to evaluate the performance of segmentation methods. In this paper, our primary interest is the integrality and accuracy of object segmentation. Therefore, as Tao et al. did in [11], we also use the absolute error ratio as the comparison criterion. The absolute error ratio is defined as the ratio between the absolute error and the total number of pixels, , of an image; that is, where the absolute error is defined as the absolute difference in the number of object pixels between the ground truth image and the thresholding image obtained by each method [11].
5.1. Experiments on Synthetic Image
The image in Figure 2(a) is a simple synthetic image with pixels and shows an English character “H” (with gray level 80) on grey background (with gray level 160). Ideal segmentation is shown in Figure 2(b). Figure 2(c) is the image into which Gaussian noises with zero mean and 0.005 variance are added, and the histogram of Figure 2(c) is shown in Figure 2(d).
(a)
(b)
(c)
(d)
The results obtained by the competing methods corresponding to Figure 2(c) are shown in Figure 3. It can be found that, as a 1-D method, Tao’s method obtains the worst result; however the proposed method obtains the best one. Table 1 shows the thresholds and absolute errors obtained by competing methods. From Table 1, we can find that the absolute error ratio of the proposed method is lower than those of other methods.
(a)
(b)
(c)
(d)
5.2. Experiments on Real Images
In order to further verify the validity of the presented method, we tested it on a variety of real images including those selected from IEEE OT CBSV WS Series Bench [16] and those captured by ourselves using a Guide IR928 thermal camera. According to the method used in [11, 17], the average of the segmentation results got by five people familiar with thermal imagery is used as ground truth images here. The original images are shown in Figure 4, where (a)~(d) are selected from the public dataset and (e), (f) are captured by ourselves. Those images are captured in different seasons under different weather conditions and light strengths. It can be observed that the human targets in (a)~(c) are smaller while those in (d)~(f) are bigger. The corresponding ground truth images are shown in Figure 5. The results of the four competing methods are shown in Figures 6–9. The thresholds obtained by different methods are shown in Table 2, and absolute error ratios of the four competing methods are shown in Table 3.
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
From Figures 6–9, it can be found out that for image (a) all the four competing methods get rather good segmentation results by both visual inspection and quantitative analysis. However, for image (b), both Sahoo’s method and Cheng’s method fail. For image (c), Sahoo’s method fails again, and Tao’s method classifies nearly all the pixels as background. For image (d), only the proposed method segments out the infrared human targets successfully. For images (e) and (f), the results of Tao’s method express some kind of over-segmentation. For all the six images, the results of the proposed method are rather reasonable.
As for the analysis of time complexity, Sahoo’s method used a purely exhaustive algorithm whose time complexity is . Cheng’s method calculated the 2-D fuzzy entropy by exhaustive algorithm and then selected the fuzzy parameter sets by genetic algorithm, so the time complexity of Cheng’s method is , in which denotes the number of iterative generation and denotes the size of population. Tao’s method is a 1-D method in which ant colony optimization is adopted, and thus its time complexity is . By reducing the time complexity of the computation of 2D FTE from to and employing SHLA to search for the optimal parameters of fuzzy membership function, the time complexity of our method is also .
When the experiments are carried out in Matlab (R2013a) on a computer with Intel Core i7-2600 [email protected] GHz and 16 GB RAM, the running times (second) of different methods are listed in Table 4. It can be found from Table 4 that Sahoo’s method takes the most time among all the methods, and the running time of our method is nearly equal to that of Tao’s 1-D method.
6. Conclusion
In this paper, a thresholding method of infrared human image is developed based on 2-D Tsallis entropy and fuzzy sets theory. In the presented method, both the fuzziness of image and the spatial information of 2-D histogram are considered. To relieve the huge computation burden brought about by extending 1-D method to 2-D one, a fast algorithm is designed to reduce the time complexity of the computation of 2-D FTE from to . And then SHLA is employed to speed up the search of optimal threshold further. The experiments on both synthetic and real images show that the presented method is robust and effective.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work is supported by National Natural Science Foundation of China (no. 61302178), Natural Science Foundation of Guangxi Zhuang Autonomous Region (no. 2013GXNSFAA019347 and no. 2013GXNSFAA019336), Research Foundation of Education Bureau of Guangxi Zhuang Autonomous Region (no. 2012LX659 and No. 2013LX088) and Natural Science Foundation of Guangxi University of Science and Technology (no. 1307111).