Abstract
A narrow band active contour model for color image segmentation is proposed, which applies local statistics to improve the robustness against noise. The crux of our approach is to use intensity mean of local region to define the force function within a level set framework, within which a narrow band is implemented to further improve the computational efficiency. In addition, the image is segmented from channel-to-channel, which shows superior performance when the intensities of the object and background are similar. Furthermore, a multichannel segmentation combination method is used to integrate the information of multiple level sets. The proposed model has been applied to both synthetic and real images with expected results, and the comparison with the state-of-the-art approaches demonstrates the accuracy and superiority of our approach.
1. Introduction
Image segmentation is a process that divides the image into meaningful parts. It plays a vital role in the field of image analysis and pattern recognition. Up to now, there have been kinds of algorithms for image segmentation, such as thresholding [1], clustering [2], and active contour models [3–9].
Active contour models (ACMs) have been used widely and successfully in image segmentation [3–16]. The basic idea is to make a contour deform so as to minimize a given energy function and generate desired segmentation [10]. ACMs can be categorized into two classes: edge-based [3–5] and region-based ACMs [6–16].
The region-based models [6–16] utilize some region descriptors such as color and intensity, showing better performance over the edge-based models for images with noise and weak boundaries in most cases. One of the most famous region-based models is CV model [6], which assumes that each region of the image is statistically homogeneous. Then, it is extendable to a two-phase model [7] for vectorial images and the multiphase model [8, 9] which partitions image into arbitrary parts. The energy function of CV model is measured by the difference between each pixel and the region intensity means. However, the CV model, as well as its extension, fails to detect the object boundaries accurately in the presence of intensity nonuniformity and noise. Aimed at the problems caused by intensity inhomogeneities, some local region-based models are proposed [10–12] in the literature, which draw upon intensity information in spatially varying local regions determined by a scale parameter [11, 12]. By sliding the Gaussian kernel to each region found by the contour, the local region model approximates intensity averages at a certain scale. According to the level set function, the local region-based models are classified into signed distance function (SDF) ACMs [10] and local approximation SDF ACMs [12, 13]. For SDF ACMs, the level set function is defined as a SDF. And thus the reinitialization is required in this evolution. In order to improve the efficiency, only values of the level set function on a narrow band around the zero level set are calculated [10].
Usually, color images are processed by converting them into scalar ones. The models in [7, 17, 18] are similar to the approaches which transform color images into scalar ones. When the objects have similar intensities, they are invalid after the application of averaging all the channels [14]. In [14], the classical CV model is extended for color images, which is based on the idea of segmenting an image from channel-to-channel and then combining the segmentation results. The extended scheme provides better capacity of color discrimination and segmentation accuracy. However, when the image is corrupted by noise, the extended scheme starts to suffer from difficulties.
In this paper, a region-based ACM to segment noisy color images in a much more efficient manner is proposed. The idea of incorporating localized statistics into a variational framework, proposed firstly by Lankton and Tannenbaum [10], makes contribution to the robustness to noise [15]. The proposed model measures both the global and local statistical properties of the object to adjust the level set function and fit the object with the zero level set, which is found to improve the robustness to strong noise and hold global dependence greatly. Meanwhile, drawing inspiration from the works in [14], we segment color images from channel-to-channel. Furthermore, a combination method is used to integrate the segmentation results in each channel. Thus, the proposed approach possesses the ability of handling with close intensity images. We define the energies in terms of a signed distance function. In order to improve efficiency, a narrow band [10, 16] is used around the contour, and the deformations of local intensity means in the local window are confined within this band.
The rest of the paper is organized as follows. Some related models for both gray and color images are briefly reviewed in Section 2. Then, Section 3 describes how the proposed model is established. Afterwards, the implementation and experiment results are presented and analyzed in Section 4. And then the conclusion is conducted in the last section.
2. Related Works
Let be a given vector valued image, and let be the image domain. is a vector dimension of the vectorial image . For gray images, , and for color images, . of is a pixel in . The goal of image segmentation is to divide the image into disjoint subregions .
2.1. Local Region-Based Framework [10]
Lankton and Tannenbaum propose a natural framework that allows any region-based energy to be localized in a fully variational way [10]. Let denote a closed contour, which is represented as the zero level set of the level set function ; that is, . The level set function (LSF) is defined as a signed distance function. We employ the following approximation of the smoothed Heaviside function to specify the interior of : Similarly, the exterior of is defined as . To specify the area just around the curve, a smoothed version of the Dirac function is defined as The parameter is usually set to 1.5 as in [4, 5]. Then is used to mask local regions: This function will be 1 when the point is within a ball of radius centered on , and 0 otherwise. In order to keep the curve smooth, we penalize the length of the curve weighted by a parameter . Using , the energy functional based on a generic force function is written as where refers to a generic internal energy measure, which can be chosen mainly according to the images to be segmented, such as uniform modeling (UM) energy, mean separation (MS) energy, or histogram separation (HS) energy [10, 16].
Taking the first variation of the energy function with respect to , we obtain the following gradient flow:
2.2. Vectorial Multiphase Chan-Vese Model [7]
Reference [7] provides an extended multiple model of the Chan-Vese model [6] for gray images to the vector valued case. In level set form, for -level sets, there are phases that partition the image into regions. is the th channel of the color image . Each channel contains the same image with different colors. The model is defined as where , , and are the average intensities of the regions in the th channel.
The CV-like models [6–9, 14] are based on the differences between the intensity of each pixel and the global intensity means. Benefited from the global dependence, the CV-like models are adequate in the presence of noise to some extent [6, 7]. However, when the noise level is high, they begin to suffer from difficulties. In addition, since the vectorial CV model uses the weighted average method to average all the channels, it is difficult to distinguish the objects.
2.3. Extended Scheme of Chan-Vese Model for Color Image Segmentation [14]
In [14], the classical CV model [6] is generalized for color images by segmenting an image from channel-to-channel (STP-CV model for short). The energy function for every channel is defined as where is the curve in the th channel. and are the means inside and outside of , respectively. and are constants. and are weighting parameters. The first and second terms of the right side are the length of the curve and the area of the regions found by contours, respectively. In the level set method, is represented by the zero level set of , such that The equation for updating level sets is Evolve the curve in each channel. Afterwards, a multichannel segmentation combination (MSC) method is used to integrate the information of multiple level sets.
3. The Proposed Model
This section develops a robust region-based active contours model for noisy color images segmentation via the narrow band implementation, which segments an image from channel-to-channel. We refer to the proposed model as NBRACM model.
3.1. Active Contour Model for Image Segmentation
In order to improve the robustness to strong noise, the proposed model applies local statistics substituting each pixel in the level set functional in the th channel. We define to mask a local region as (3). Therefore, local means in terms of are expressed as
Different from the classical CV-like models, the force function is measured by the differences between the local and global intensity means. When a given pixel in the image domain is corrupted by noise, the energy , which is based on the difference between each pixel and global region means, would reach the minimum even though the curve is not exactly on the boundaries. Hence, some pots in this case are taken wrongly as the object which we are detecting. Problems that appear with the classical CV model derive from the ignorance of local statistics. However, in order to reduce the effect of noise, we replace each pixel with the average of points in its neighborhood. That is, the local information helps to make the proposed model free from noise. Actually, the operation, which replaces each pixel with local region mean, is deemed to approximate the original noisy image with a fitting image which is obtained through neighborhood average filter. In fact, the fitting image would be oversmoothed by the average filter with a lager radius of neighborhood. The lager the radius size is, the fuzzier the image would become. However, when the radius size is small enough, the fitting image is still affected by noise. Similarly, with a smaller neighborhood size, the proposed model using local intensity average will be sensitive to noise. At a larger size, it would be less sensitive to noise and reduce the accuracy of segmentation.
In this paper, we calculate the force function in each channel; thus, the level set evolution is conducted in each channel separately, that is, the R, G, B channels. As a result, different objects of color image with similar intensities are distinguished easily. Flowchart of the proposed model is shown in Figure 1.
We apply the local region-based active contour model in each channel (R, G, B) of the input image, respectively. After the application of multichannel segmentation combination (MSC) method, we obtain the segmentation results. With the aid of the global intensities means, the internal force in each channel is formed by where and are global means inside and outside of the curve , respectively, which are calculated as
The can be substituted directly into (4) to form a completely localized energy within full image domain. To obtain the level set evolution equation for , we take the derivative, expressed as
By substituting the derivative of into (5), we obtain the following evolution equation in the full domain implementation:
All the partial derivations and in (14) are approximated by the central finite differences. The temporal derivation is discretized as a forward difference. The time dependence is given in discretized form with spatial index and temporal index . Then, an iteration scheme is obtained by discretization of the partial differential (14): where is the approximation of the right hand side in the evolution (14).
After the application of the segmenting model in each channel, a combination method should be used to obtain the final results. It is worth noting that the multiple-phase CV [9] model employs two or more level sets. Thus, different sign sequences of the level sets represent different regions. In [14], the level sets are assigned in a different way, namely, one level set per channel. Likewise, in this paper, a set of -size sign sequence can be represented as follows: where the functional is defined as
Figure 2 employs a simple example to illustrate the MSC method. ( in this paper) are conducted in R, G, B channels, respectively. Each region in the image has a sign array which is composed of signs. In particular, we employ two regions per channel, and hence the proposed model is restricted to be valid only in the case of color images with regions. In this paper, the first sign in the array illustrates whether the corresponding region contains R component. If it is true, it is denoted by a symbol of “−”, and vice versa. The other signs in the array are determined in the same way. For example, the region in Figure 2 is denoted by “− − −.” Assigning different signs to each region, we can get regions at most. Thus, we obtain the final segmentation results. However, when the detected regions are more than eight, more level sets for one channel could be used.
3.2. Narrow Band Implementation
Throughout the evolution processing, each point in the full image domain with local statistics needs to be computed, which indicates a quite huge computation. Instead of dealing with the entire domain, we only consider an inner and outer band, both sides of the curve, that is, a narrow band around the zero level set. The width of the band should be slightly lager at least than twice as much of the space steps; thus, it ensures a point per side at least. The narrow band implementation is simple, in which the iteration process consists of updating local interior and exterior statistics in the narrow band and performing evolution according to (15).
Firstly, we locate the initial curve with an arbitrary shape and initialize it to be a binary LSF. In the image domain , LSF only takes two values 1 and −1 as is the subset of . We should initialize every pixel in the narrow band with the local interior and exterior statistics as well. In addition, when the narrow band moves to include an uninitialized pixel, its local statistics should be initialized as well [10].
Then, the internal force in (13) can be calculated via the narrow band approach:
Calculating the curvature along the narrow band, we evolve the LSF for every point in the narrow band according to (15) in the th channel until to convergence.
4. Implementation and Experimental Results
4.1. Implementation
In order to improve efficiency, we only compute values of in a narrow band around the zero level set [10]. Consequently, we reinitialize every few iterations using a fast marching scheme [19]. Then the steps of the proposed model are as follows.(1)Locate the initial curve with an arbitrary shape, and initialize it to be a binary LSF. Establish a narrow band around the zero level set in each channel.(2)Initialize every pixel in the narrow band with the local interior and exterior statistics in each channel.(3)Calculate and in the global region, and in the narrow band, and then update the force function according to (19).(4)Get forces from curvature penalty along SDF.(5)For every point in the narrow band, evolve the level set function according to (15).(6)Smooth the SDF using the method in [20].(7)Return to step (3) if the evolution has not converged; otherwise, stop the evolution.(8)Combine the segmentation results of each channel to obtain the final results.
4.2. Experimental Results
The performance of the proposed model for noisy color images segmentation has been validated through two groups of experiments carried out on both synthetic and real images. The first group of experiments is conducted to demonstrate the color discrimination and robustness to initial contours. In particular, as for the color discrimination, we apply the proposed model without using the idea from channel-to-channel, which would pick and put the objects with different colors in the same class. The robustness to initial contours of the proposed model is compared with the FMLSM model in [18]. For the second group of experiments, color images added with different noises with different variances are used to show the performance of the proposed model. At the same time, the effect of the radius of local region is also detected. All the experiments are conducted in MATLAB R2010a, on a personal computer with an Intel(R) Core(TM) Duo CPU and 2.00 GB memory.
4.2.1. Color Discrimination
For color images segmentation, the traditional method is to integrate multichannel information, which transforms a color image into a gray one. Actually, our approach can also conduct color image segmentation in the manner mentioned above, called the vectorial NBRACM model. To demonstrate the greater color distinguishing ability of our approach, we conduct experiment on synthetic color image by comparing the vectorial NBRACM with the proposed model, namely, the NBRACM model.
Figure 3, used in [18], shows a synthetic image with three circles filled with different colors. We choose two circles to be initial contours. The regions, detected by the final contours, are labeled with different colors for easy observation. We use multiple-phase vectorial NBRACM to segment the multiple objects. Figures 3(c) and 3(e) are the corresponding final contours of the vectorial NBRACM and the proposed model, respectively. As we can see, the vectorial NBRACM model can only detect the top yellow ball, taking the bottom objects as the entire one. On the contrary, the proposed model is capable of making a distinction among the three balls.
(a)
(b)
(c)
(d)
(e)
(f)
The results for three synthetic images from [18] applied with different initial contours are shown in Figure 4. We compare the FMLSM model in [18] with the proposed model. We choose the initial contours as in [18]. For the first image, when we set two disjoint initial contours as in the first row of Figure 4(a), the two models obtain perfect image segmentation results. When using joint initial contours, for the FMLSM model, the final contours are exactly on the boundaries of the objects. However, the fitting image, which is labeled with different colors, is inaccurate, assigning the bottom two objects with the same color. That is, the bottom objects could not be differentiated by the FMLSM model. However, the proposed model, set with the same initial contours, obtains satisfactory final contours and results. The second synthetic image has three objects with different shapes. When we choose two disjoint circles as initial contours, the same problem happens again. Namely, the dark-blue rectangle and sky-blue star are put in the same class. When the initial contours are set as many small circles, the objects are distinguished well by the FMLSM. On the contrary, our model can differentiate the three objects under these initial conditions. For the third image, the FMLSM obtains accurate segmentation when the initial contours are small circles. The experiments in Figure 4 also demonstrate that the FMLSM obtains satisfactory results when small circles are chosen. However, our model segments the images successfully, showing more robustness to initial contours than the FMLSM model.
(a)
(b)
(c)
(d)
(e)
Actually, for color images segmentation, both the vectorial NBRACM and the FMLSM [18] models work in the same way. They are similar to the methods which transform the color images into scalar ones [14]. For their multiphase models, there are two active contours evolving in the regions. Both the movements of the contours are determined by the force which averages all the channels using weighted average method. Namely, the intensities of each channel work jointly on the evolving contours during the whole process. However, as can be seen from Figure 1, we assign an active contour to each channel for our model. Thus, the active contours work in each channel independently. The final segmentation results are obtained through MSC method.
To demonstrate the efficiency of our model, we compare the computation CPU time for the above three synthetic images with the vectorial CV model [7], STP-CV model [14], FMLSM [18], and our model (marked as “Ours #2”) in Table 1. In particular, to demonstrate the advantage of narrow band implementation, we also perform the proposed model in the traditional means of computing local statistics in the entire domain (marked as “Ours #1”). As can be seen from Table 1, FMLSM model, which applies split Bregman method, requires the shortest CPU time. On the other hand, our model (#2) is much more efficient than the one using traditional method (#1), which demonstrates the superiority of narrow band implementation. Besides, we can also see that our model is much more efficient than the vectorial CV model, STP-CV model. However, the direction of our future work lies in the incorporation of globally convex image segmentation and the split Bregman method into the proposed model.
The experiments in Figure 5 are conducted on a group of real images. We compare the proposed model with the FMLSM. The image in the first row is a synthetic image with many shapes. The FMLSM model confuses the rose red objects with the blue ones, while our model obtains successful results. The same situation exists in the fourth image. For the flower images, the FMLSM possesses the ability of detecting objects, providing good segmentation. However, benefited from the color discrimination, our model can distinguish more colors, making the segmentation closer to the reality.
(a)
(b)
(c)
(d)
4.2.2. Noise Robustness
In this subsection, firstly, we demonstrate that the proposed model possesses more robustness to noise, compared with the multiphase vectorial CV model in [7, 9], the STP-CV model in [14], and FMLSM model in [18]. We choose small circles as initial contours to improve the efficiency.
Figure 6 shows the segmentation results on a synthetic image added with Gaussian noise with 0 mean and variance 0.05. There are quite many different shapes and four colors in the image. To be fair, we set the same parameters to validate the better performance of the proposed model than that of the multiphase vectorial CV model. As we can see from Figures 6(b)-6(c), the multiphase vectorial CV model detects the boundaries of the objects, dividing them into correct classes. However, when the pixels are polluted by noise, the multiphase vectorial CV model takes some noise pots as the objects. It is due to the fact that the CV-like models are based on the differences between each pixel and the global region means. By contrast, the proposed model is less affected by the noise, providing perfect results as in Figures 6(d)-6(e).
(a)
(b)
(c)
(d)
(e)
Figures 7 and 8 present the segmentation results of the proposed model and the STP-CV model on a synthetic image. In Figure 7, we add Gaussian noise with mean 0 and variances 0.01, 0.03, and 0.08 to this image, respectively. Figure 7(a) shows the noise-corrupted images. Figures 7(b)-7(c) show the corresponding final contours and results of the STP-CV model, while Figures 7(d)-7(e) show the corresponding final contours and results of the proposed model. It can be observed from Figure 7(b) that the STP-CV model fails to segment the objects due to the fact that it takes no local statistics into consideration. In this case, the STP-CV model finds not only the objects boundaries but also some noise pots. Also, we set the same parameters for both the STP-CV model and the proposed model for comparison. However, as can be seen from Figures 7(d)-7(e), the proposed model generates more accurate segmentation results. Not only because our model takes local statistics into consideration, but because it possesses the global dependence.
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
Figure 8 compares the proposed model to the STP-CV model on the synthetic image polluted by salt and pepper noise. We add the salt and pepper noise with variances 0.01, 0.05, and 0.1, respectively, to the synthetic image. Again, a clear improvement is shown. Meanwhile, the STP-CV model shows better performance when dealing with images contaminated by salt and pepper noise than Gaussian noise. Our model, exploiting local intensity information, possesses the ability of segmenting noisy images.
The experiments in Figures 9 and 10 compare the FMLSM with the proposed model in regard to the robustness to various severe noises. A synthetic color image with many objects is employed. In Figure 9, we add severe Gaussian noise with mean 0 and variances 0.1 and 0.5 to the image, respectively. The FMLSM model has the capacity to segment noisy image [18]. Thus, the objects in the noisy image are detected correctly when the variance is 0.1, which can be observed from Figures 9(b)-9(c). However, the FMLSM begins to fail when the variance of noise increases to 0.5. In Figure 10, a similar phenomenon has occurred in the presence of multiplicative noise. Conversely, the proposed model, which employs the local intensities, obtains satisfactory segmentation. The experiments in Figures 9 and 10 show that the proposed model outperforms the FMLSM in the presence of severe noises.
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
For the proposed model, the radius of is a vital parameter. If the radius is very small, then the model is sensitive to noise. On the other hand, if the radius of is quite large, the proposed model would have trouble in shape detection. In particular, if the radius grows such that includes the entire image, the local statistics are exactly the global regions statistics; therefore, the differences between them would always be zero. Thus, the proposed model would be invalid. We show the effect of radius size on a synthetic image added with Gaussian noise with variance 0.1 in Figure 11. The radius is set to 1, 2, 5, and 9 pixels, respectively. As can be seen from Figures 11(b)–11(e), with the smallest radius, the objects along with some noise pots are detected. When the radius grows to 9, the detected boundaries are distorted. For quantitative evaluation, four region overlap metrics are used to compare the performances of the models quantitatively. They are the Jaccard similarity (JS) [21], the dice similarity coefficient (Dice) [21, 22], the false positive ratio (rfp), and the false negative ratio (rfn). represents the foreground of the ground truth image while stands for the foreground obtained by the models. Since the tested image is a synthetic image, its ground truth has been already known to us. is the common part of and . indicates the pixel numbers of the region. These metrics are defined as The closer the JS and DSC values to 1, and the RFP and RFN values to 0, the better the segmentation results. The radius-error curve is shown in Figure 12.
(a)
(b)
(c)
(d)
(e)
As seen from Figure 12, the accuracy of the model with the smallest radius is satisfactory. This is due to the fact that though some noise pots are taken wrongly as the objects, their pixel numbers are too little to affect the value of the error. When the radius grows larger, the accuracy becomes lower. Usually, we set radius to be 2 in all experiments in this paper.
Finally, we apply our model to a group of real images. The results are shown in Figure 13. Again, we can observe that our model can segment these color images well.
(a)
(b)
(c)
(d)
5. Conclusion
In this paper, a method to segment color images with local region-based active contour model is proposed. The model segments color images from channel-to-channel so that it is able to distinguish close colors. Integrating the merits of the local statistics and the idea of channel-to-channel segmentation, our model possesses the ability of segmenting noisy images, color discrimination, and robustness to noise. After the application of the level set evolution, a MSR method is used to integrate segmentations of each channel. In addition, the proposed model is robust to initial contours. In order to improve efficiency, the level set evolution is calculated in a narrow band frame. Experimental results have demonstrated superior performance of our method.
However, the model cannot cope with images, suffering from intensity inhomogeneity, which is the direction of future work.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The research is supported by the Open Project Program of Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information (Grant no. 30920130122005) and Research Fund for Doctoral Program of Higher Education of China (no. 20133219110027).