Abstract
This paper presents a variation of the fuzzy local information c-means clustering (FLICM) algorithm that provides color texture image clustering. The proposed algorithm incorporates region-level spatial, spectral, and structural information in a novel fuzzy way. The new algorithm, called RFLICM, combines FLICM and region-level Markov random field model (RMRF) together to make use of large scale interactions between image patches instead of pixels. RFLICM can overcome the weakness of FLICM when dealing with textured images and at the same time enhances the clustering performance. The major characteristic of RFLICM is the use of a region-level fuzzy factor, aiming to guarantee texture homogeneity and preserve region boundaries. Experiments performed on synthetic and remote sensing images show that RFLICM is effective in providing accuracy to color texture images.
1. Introduction
Image segmentation is one of the most important tasks in computer vision, and many other fields of application are closely related to it, including pattern recognition, remote sensing, and medical diagnostics. The purpose of segmentation is to partition an image into homogeneous regions. Although many methods have been proposed to solve this problem, the fuzzy c-means algorithm (FCM) [1] has been widely applied to image segmentation, because its fuzzy nature allows more original information being considered. FCM has some obvious advantages such as the straightforward implementation, the fairly robust behavior, the applicability to multichannel data, and the ability to model uncertainty within the data [2]. However, the traditional FCM failed to consider the spatial information, which may cause poor results when dealing with images corrupted by noise, outliers, and other image artifacts.
To compensate this drawback of FCM, many attempts have been proposed, including adding a preprocessing image smoothing step before clustering [3] or incorporating spatial context in different ways, for example, the fuzzy membership function refinement [4, 5], the dissimilarity function improvement [6], the objective function regularization [2, 7], and the fuzzy treatment of hidden Markov random field (MRF) model-based image segmentation (HMRF-FCM) [8]. More recently, following the method of objective function regularization, Krinidis and Chatzis [9] presented a robust image clustering method called fuzzy local information c-means (FLICM). In FLICM, the clustering is dependent on both the spectral and local spatial information which cooperate by using a fuzzy factor. However, this algorithm assumes that the label of one pixel is only related to the labels of its neighboring pixels. Therefore, only interactions between neighboring pixels can be used, which makes the algorithm defective in dealing with color texture images due to the lack of information of large scale interactions between image patches instead of pixels.
In recent years, there is an increasing trend to analyze images based on image regions in order to make use of different kinds of large scale local information (e.g., spectral, spatial, and structural information). The MRF model is one of the most popular methods to integrate these kinds of information together, because of its powerful capability of describing the continuity of image characteristics. Based on region-level MRF (RMRF), Yang and Jiang [10, 11] proposed a gray image segmentation method and showed good performance; Clausi et al.’s research group also presented algorithms to deal with gray images [12], multivariate images [13], and polarimetric SAR images [14]. All of these methods have shown the superiority to pixel-based ones.
Inspired by the success of RMRF-based image segmentation methods in image segmentation, this paper improves FLICM into a region-based version, named RFLICM. In RFLICM, a novel fuzzy factor is defined by the basic idea of RMRF and incorporated into the objective function of FCM. The fuzzy factor can simultaneously incorporate the region-level spatial, structural, and spectral information in a fuzzy way and helps to guarantee the texture homogeneity as well as preserving region boundaries. All these characteristics make RFLICM more general and suitable for color texture image segmentation.
The remainder of the paper is organized as follows. Section 2 briefly describes the fuzzy local information c-means algorithm, followed by the basic theory of RMRF-based image segmentation. The RFLICM algorithm is introduced in Section 3. Experimental results are presented in Section 4 and conclusions are drawn in Section 5.
2. Related Work
2.1. Fuzzy Local Information c-Means (FLICM) Clustering Algorithm
In [9], a fuzzy factor is used to incorporate local spatial and gray level information into the objective function of FCM:where denotes an -pixel image defined on a rectangular lattice set in the -dimensional vector space, is the number of image pixels, is the number of clusters with , is the degree of membership of in the th cluster, is the weighting exponent on each fuzzy membership, is the prototype of the center of cluster , is a distance measure between object and cluster center , and is the fuzzy factor which is defined aswhere th pixel is the center of the local window (e.g., 3 × 3), is the reference cluster, and pixel belongs to the set of neighboring pixels falling into the window around the th pixel (denoted as ). is the spatial Euclidean distance between pixel and and is the membership of the th pixel in the th cluster.
Local minimum extreme of is obtained iteratively as follows.
Step 1. Set clusters number , fuzzification parameter , and the stopping condition .
Step 2. Initialize randomly the fuzzy partition matrix.
Step 3. Set the loop counter .
Step 4. Calculate the cluster prototype using
Step 5. Compute membership values using
Step 6. If , then stop; otherwise, set and go to Step 4.
2.2. RMRF-Based Image Segmentation
Assume the image has been over-segmented previously into disjoint regions. In region-level methods [11–14], all pixels in the same region are assumed to have the same label. Let be the label image. In the Bayesian image segmentation framework, the segmented image is:where and are the class conditional probability and prior probability, respectively. With the assumption that the noise in the image is independent Gaussian white noise, can be written as follows:where is the Gaussian probability density function with mean value and covariance matrix . And if RMRF is used to model the prior probability, can be described as:where is a normalizing constant and is an energy function defined as:where is the set of neighboring regions of , and is defined as [10]:where is the length of the common boundaries between region and , is the boundary length of region , is the area of region , and are the mean value of region and , respectively, and is the Kronecker function.
Then the configuration of can be determined bywhere is the potential function. Equation (10) can be solved by simulated annealing (SA) [16] or iterative conditional mode (ICM) [17]. For the sake of efficiency, ICM is a good choice. In this paper, we denote the ICM algorithm based on the RMRF as RICM. By using a RMRF model-based prior probability, RICM can make use of the large scale local spectral, spatial, and structural information during the segmentation process.
3. Region-Level Fuzzy Local Information c-Means (FLICM) Clustering Algorithm
Motivated by the successful application of RMRF to image segmentation, we propose, in this paper, a novel framework for image clustering by extending FLICM to a region-based version, called RFLICM.
For each region, some region-level features are extracted firstly. For example, and are the mean value and covariance matrix of , respectively, and are the area and boundary length of , respectively, and is the common boundary length between and , satisfying the constraint of .
3.1. The Region-Level Fuzzy Factor
In order to overcome the defect of FLICM on using large scale information, the new fuzzy factor should take into account the region-level spatial, spectral, and structural information. Following the basic idea of RMRF, the fuzzy factor for each region is only determined by its direct neighboring region set . So, in this paper, the novel region-level fuzzy factor for region is defined aswhere is the membership value of belonging to th cluster.
It is easy to see that the factor makes the influence of regions within the direct neighborhood , to change flexibly according to their areas, boundary lengths, and mean values. Therefore, more region-level information can be used to deal with color texture segmentation. In (11), acts like a normalizing coefficient preventing to have a too large value. Similar to , (2), also reflects the damping extent of the neighboring regions to be assigned to different labels. So the designed fuzzy factor has the ability to force neighboring regions to be equally labeled.
3.2. General Framework of RFLICM
We assume that all pixels in the same region have the same membership value belonging to different clusters. So the objective function RFLICM is defined aswhere is a constant weighting parameter, it has the similar function as the potential function in RMRF-based image segmentation, (10). The first term in the right part of (12) depicts the fidelity of clustering result to image data, and the second term is a smooth term that forces neighboring regions to have the same label. The parameter controls the contribution of these two terms.
In this paper, we assume that pixels in each cluster follow a Gaussian distribution. So the distance measure is defined aswhere and are the mean value and covariance matrix of the th cluster, respectively, and is the spectral number of the image being processed.
Similar to FLICM, a solution of the objective function of RFLICM, (12), can be obtained through an iterative process, which is carried as follows.
Step 1. Get the initial partition of the input image.
Step 2. Set the number of the cluster prototypes, fuzzification parameter , weighting parameter , and the stopping condition .
Step 3. Set the loop number , perform FCM clustering on image pixels, and get the initial region level fuzzy partition matrix based on the minimizing membership the rule:where is the membership value of pixel belonging to the th cluster.
Step 4. Calculate the cluster prototypes using
Step 5. Calculate the membership functions using
Step 6. If , then stop; otherwise, set and go to Step 4.
When the algorithm has converged, a defuzzification process of maximum membership is employed to convert the partition matrix to a segmentation result:
A constraint is imposed on the membership value in (15) to avoid the underfitting of Gaussian parameters:
Another issue that is worthy to be pointed out is the determination of the weighting parameter . It controls the smooth strength of the region-level fuzzy factor. In this paper, we employ an increasing schedule for to guarantee the accuracy of parameter estimation and the homogeneity of segmentation result. It increases with the iterative procedure:At the first stage of the iteration, takes a small value to depress the influence of the region-level fuzzy factor , which forces the estimated Gaussian parameter to fit well the image data. As increases, more large scale local spectral, spatial, and structural information is introduced into the clustering procedure. Finally, reaches its limitation, the iteration procedure arrives at a balance state, and begins to help to refine the final result.
4. Experimental Results
In this section, we show the performance of the proposed method by presenting results on synthetic images and remote sensing images. Furthermore, we compare the efficiency of RFLICM with HMRF-FCM [8], FLICM [9], and RICM described in Section 2.2. For FLICM, we set the window size as 3 × 3. For HMRF-FCM, we use the 2nd neighborhood. For both RICM and our proposed RFLICM, mean shift algorithm (MS) [18] is chosen to get the initial partition. In this experiment, we set for our algorithm and set different for RICM using the trial-and-error method (we select the parameter in the range of with the interval of 0.5, and the one with the highest segmentation accuracy is recorded and is listed in Table 1). For MS, it is difficult to choose the optimal parameter set. How to choose good parameters is out of the scope of this paper. Intuitively, if parameters are set too large, there will be a risk of losing useful information in the initial oversegmentation; otherwise, large scale interactions cannot be obtained and employed properly. In this paper, we just experimentally choose both the spatial resolution parameter and the range resolution parameter to be 6 and the size of the smallest segment to be 10.
In order to quantitatively evaluate the proposed method, both accuracy and Kappa coefficient are used as indicators [19]. If is the proportion of subjects that were assigned to the th class by the first image and the th class by the second image, denote and , the values of and are defined as
First of all, we apply these methods to a synthetic noisy image shown in Figure 1(a), which was obtained by adding Gaussian noise (zero mean noise with 0.06 variance) to a four-color checkerboard image shown in Figure 1(b). The size of the noisy image is 128 × 128, and the gray levels of four desired regions are 0, 0.3333, 0.6667, and 1.0, respectively. The segmentation results are shown in Figure 1. From Figures 1(c) and 1(d), one can find that both FLICM and HMRF-FCM can separate the image into proper regions but still have some obvious misclassifications and seriously smoothed boundaries, while Figures 1(e) and 1(f) show that RICM and RFLICM can obtain more accurate boundaries and less misclassifications. The most interesting comparison can be found between the result of RFLICM (Figure 1(f)) and those of referenced methods (Figures 1(c)–1(d)); it is obvious that RFLICM shows the best performance to resist image noise. The corresponding and Kappa are listed in Table 2. The quantitative indicators also show that the proposed method is more robust than the referenced ones.
(a)
(b)
(c)
(d)
(e)
(f)
Secondly, we apply these four algorithms to a group of synthetic color texture image, which are shown in the first row of Figure 2. The first three images are obtained from the Prague Texture Segmentation Data generator [15], and the last one is synthetized from four different kinds of remote sensing textures. The 2nd row shows the corresponding ground truth images. The segmentation results of FLICM, HMRF-FCM, RICM, and RFLICM are shown in the 3rd, the 4th, the 5th, and the last row in Figure 2, respectively.
(a)
(b)
(c)
(d)
(e)
(f)
From the segmentation results, it is easy to be seen that both FLICM and HMRF-FCM cannot get acceptable results: most texture areas are wrongly labeled. The main reason is that they only employ the interactions between image pixels. Therefore, there is not enough information to be used for describing texture models in images. The RICM algorithm can get rather better results than FLICM and HMRF-FCM. But it is very easy to trap into local minima, which results in inconsistent segmentation results, while, for all of these textured images, our proposed method can get rather satisfactory results. As shown in the last row in Figure 2, the segmentation results exhibit homogeneous texture regions and also pretty good boundaries. Such results have verified the basic idea of this paper to improve FLICM into a region-based version. Table 3 also gives the quantitative comparison results of these four algorithms on these synthetic color texture images. The highest values are highlighted in the table. It is clearly illustrated that the proposed RFLICM algorithm gives rise to better texture segmentation performance than the referenced ones.
Besides, we also applied the same four algorithms on the multispectral high resolution remote sensing images, which are shown in Figures 3(a) and 4(a), respectively. Figure 3(a) is a piece of 256 × 256 Quickbird image of an outskirt in Wuhan, and Figure 4(a) is a 438 × 438 Spot5 image of Pingshuo area, China. Their segmentation results are shown in Figures 3 and 4, respectively. And the corresponding quantitative measurements are recorded in Table 4.
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
In Figure 3(a), there are mainly three kinds of land cover types: the farmland, trees, and buildings. All of these types have rich texture information. The segmentation results of FLICM and HMRF-FCM have a lot of misclassifications. This is caused by the fact that using pixel-level interactions only cannot effectively model the complex texture pattern over the image. Furthermore, both and are very low. The RICM algorithm can get better results than these pixel-based methods but still has obvious misclassification. The RFLICM algorithm performs much better than the others since large scale (spectral, spatial, and structural) information can be employed in a fuzzy way in the clustering procedure. Therefore, it leads to a consistent result, with and being the highest, that is, more than 88% and 81%, respectively.
In Figure 4(a), four different kinds of land cover types are obviously distributed, with highly staggered appearance. The rich texture characteristics of this image make the results of FLICM and HMRF-FCM with serious misclassification. The result of RICM also contains much nonhomogeneous regions. It is mainly because that RICM cannot use the smooth term in a fuzzy way. Compared with the referenced methods, our proposed method shows a much better performance, with and more than 87% and 81%, respectively.
We also evaluate the computational time for all methods in the previous experiment. All of these algorithms are executed on a computer with an Intel Pentium Dual-Core 2.2 GHz central processing unit (CPU) and 4 G random access memory. The computation time is recorded in Table 5. It can be shown that the proposed method is a little slower than FLICM. It is slower mainly because of the time consumed by creating the initial oversegmentation and the use of more complicated distance function in our proposed method. But considering the accuracy obtained, it is worthy for us to pay such a price.
5. Conclusion
In this paper, a novel region-level fuzzy local information c-means (RFLICM) algorithm for image segmentation was presented. The proposed method can detect the clusters of color texture images. It overcomes the disadvantages of the FLICM algorithm by incorporating region-level spectral, spatial, and structural information. The region-level fuzzy factor used in RFLICM aims to model in the fuzzy manner the large scale interactions between image regions instead of pixels. In terms of accuracy, experimental results of both synthetic and real datasets have illustrated that our proposed algorithm has better performance than that of FLICM, HMRF-FCM, and RICM. The color texture image experiments exhibited several unacceptable failures for all referenced methods, but the proposed RFLICM always gave accurate segmentation results.
However, compared with FLICM, RFLICM need a prepartition step which should provide a deliberate oversegmentation to preserve as much details as possible. Besides, it also needs to provide an initial value for the weighting parameter, while FLICM is completely free of any parameter determination. Although the segmentation results are not too sensitive to this value if it is in a proper range, we have to use the trial-and-error method to find a suitable value for the segmented images (in our experiments 10 is good enough).
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work is supported in part by the National Natural Science Foundation of China under Grant no. 41001251, the China Scholarship Council (CSC) during a visit of Guoying Liu to the University of New Brunswick, the Canada Research Chairs Program and National Key Discipline of Forest Management support from Southwest Forestry University under Grant no. XKZ200901, the Key Technology Projects of Henan province of China (no. 132102210212), the Key Technology Projects of the Educational Department of Henan Province of China (no. 13A520011), and the Excellent Youth Teacher of Henan Educational Department of China (2011).