Abstract
The level set methods have provided powerful frameworks for image segmentation. However, to obtain accurate boundaries of the objects, especially when they have weak edges or inhomogeneous intensities, is still a very challenging task. Actually, we have studied the popular existing level set approaches and discovered that they failed to segment the images with weak edges or inhomogeneous intensities in many cases. The weak/blurry edges and inhomogeneous intensities cause uncertainty and fuzziness for segmentation. In this paper, a novel fuzzy level set approach is proposed. At first, -function based on the maximum fuzzy entropy principle (MEP) is used to map the image from space domain to fuzzy domain. Then, an energy function is formulated according to the differences between the actual and estimated probability densities of the intensities in different regions. A partial differential equation is derived for finding the minimum of the energy function. The proposed approach has been tested on both synthetic images and real images and evaluated by several popular metrics. The experimental results demonstrate that the proposed approach can locate the true object boundaries, even for objects with blurry boundaries, low contrast, and inhomogeneous intensities.
1. Introduction
Image segmentation is an important component of image analysis and computer vision. The results of segmentation are not always satisfactory because of low contrast, blurry boundaries, noise, and inhomogeneous intensities. Hence, image segmentation is still a quite difficult task [1].
Recently, active contour models have attracted great attention from researchers [2]. There are two kinds of active contour models. One is the snake model which defines a parametric curve. All snake properties and its behavior are specified through an energy function. A partial differential equation controlling the snake makes it evolve to reduce the energy [3]. The physical analog can be extended, and the motion of the snake can be viewed as the simulated force acts on it. The other one is the geometric model [4, 5]. The main difference between these two kinds of approaches is that the geometric active contour introduces the level set function representing the evolving curve into the energy function. The implicit boundary representation does not depend on a specific parameterization. During the propagating, no control point mechanisms need to be employed. Level set-based active contour models have many advantages, and among them the most important one is the ability to track the topological variations of the boundary curves [6, 7].
Level set-based segmentation models can be divided into four categories: “region-based” [5, 8–11], “edge-based” [4, 12–14], “shape-prior” [15, 16], and “multifeature-based” [17]. Region-based approach utilizes global information such as region statistics, region mean, and weighted mean of the constrained or scalable neighboring regions and can produce more semantically meaningful results [18]. The major limitation of region-based approach is that it is very difficult to define a suitable region descriptor for image with inhomogeneous intensity [19]. Some methods are based on a general piecewise matter [20, 21]. These methods do not assume the homogeneity of the intensities; therefore, they are able to segment images with inhomogeneous intensities, and they have been actively studied recently [5, 8]. However, these methods are quite sensitive to the noise and cannot work well with blurry and weak edges. It will be discussed in detail in the Experimental Results.
Shape-prior approach incorporates the shape information. The method introduces a representation for deformable shapes and defines a probability distribution over the variances of a set of training shapes. Obviously, such method cannot work well if the prior shapes of the objects are unknown.
Edge-based approaches do not need to assume the homogeneity of image intensities; hence, they can even be applied to images with inhomogeneous intensities [12] and has been applied to many segmentation tasks. An edge representation is used to find the boundary curve with strong edge response [13]. However, these approaches suffer from serious boundary leakage problems, especially when the objects have inhomogeneous intensities; and they cannot converge on the real boundaries of the objects in many cases either [22]. This will be discussed in detail in the Experimental Results as well.
Multifeature-based approaches use the surround inhibition weights of individual features, including orientation, luminance, and luminance contrast. Features are combined according to a scale-guided strategy, and the combined weights are then used to modulate the final surround inhibition of the objects [17].
The blurry and weak edges, the noise, and the inhomogeneous intensities can cause uncertainty and fuzziness which will result in poor outcomes of segmentation. In fact, images have uncertainty and fuzziness due to the following. (1) When performing 3D to 2D projection, some information was lost. (2) Some definitions of images such as edges and contrast are uncertain and fuzzy. For instance, an edge is defined as there is intensity difference between a pixel and its neighbor pixels; however, how large the difference should be is not precisely specified and defined, and eventually it is task-dependent. Therefore, we should use fuzzy logic to handle the uncertainty and fuzziness of the images [23, 24].
Many methods try to detect contour by calculating high gradients of color or gray levels. As a result, they are very sensitive to noise and textures. Entropies can be used as a measure of dissimilarity or inverse cohesion between two (or more) probability distributions [25]. For example, in [26], a thresholding scheme is proposed to minimize the Tsallis cross-entropy between the original image and the thresholded image. Then, the contours of the objects are obtained. Average Entropy (AE) is defined as a new information measurement of regions in image [27].
In this paper, a novel fuzzy level set active contour model is proposed. It will combine the advantages of fuzzy logic and level set theory and can generate much better results in segmentation. In experiments, a series of images are employed for evaluating the proposed method and comparing with existing segmentation algorithms. The experimental results demonstrate that the proposed approach can segment both synthetic and real images satisfactorily. Furthermore, it makes the evolving function converge to the real boundaries even with low contrast, inhomogeneous intensities, and blurry edges.
The rest of the paper is organized as follows. In Section 2, the proposed fuzzy level set approach for segmentation is described. A variety of images have been tested and validated the proposed approach (however, due to page limit, only a few of them are shown here), and the performance is evaluated in Section 3. Finally, the conclusions are summarized in Section 4.
2. Proposed Fuzzy Level Set Approach
The proposed fuzzy edge-based level set approach consists of the following major components: fuzzification, fuzzy energy function, and evolution equation.
2.1. Image Fuzzification
Assume that the size of image is , and is the gray level of the pixel at coordinates (). In order to apply fuzzy logic to deal with the fuzziness and uncertainty of the image, a suitable membership function is necessary. The most commonly used membership function is the standard -function [24]:
The value of represents the membership of , which is simplified as , and the fuzzified image is denoted by . Parameters , , and determine the shape of the -function. Based on information theory, the maximum entropy corresponds to the maximum information. We use the maximum fuzzy entropy principle to determine parameters , , and . We will find the combination of the parameters corresponding to the maximum entropy:where is the entropy of the image and is the Shannon function:There are many ways to find the maximum value of (2) such as simulated annealing, neural networks, and genetic algorithm. In this paper, we use simulated annealing algorithm [24] to find the optimum values of and to avoid sticking at local minimum: where and are the minimum and maximum intensity values of the image, respectively.
Using above fuzzification process, we can have the maximum information when transforming image from space domain to fuzzy domain according to information theory. In addition, the -function can enhance the images [23]; that is, it can improve the weak edges and prevent leakage further.
Then, the original image is transformed to a fuzzified image according to (1).
2.2. Fuzzy Energy Function and Evolution Equation
Considering a fuzzified image as a real positive function defined in domain , the boundaries are defined as the fuzzy zero level set of , which will be simplified as . Given the fuzzy level set function , the basic form of the energy function in the ordinary space [10] can be adapted and transformed into the fuzzy domain: where is a parameter controlling the effect of penalizing the deviation of from a signed distance function; and are positive constants; and is a penalizing term defined as the integral below: characterizes how close function is to a signed distance function in domain .
An external energy for function is defined aswhere is the univariate Dirac function, is the Heaviside function, is the length of the zero level curve, is used to speed up curve evolution which is the weighted area of the subregion, and is the fuzzy edge indicator function:where is the Gaussian kernel with standard deviation and is the convolution operator. The energy function drives the fuzzy zero level curve towards the boundaries and stops evolving with the strongest boundary response.
The fuzzy zero level curve evolves to the gradient flow correspondingly and drives the evolution equation for finding the minimum of the energy function. With total variation method, the associated gradient flow is derived:where is the initial condition defined in fuzzy domain and the last equation in (9) is the boundary condition. The initial condition can be formulated aswhere is a predetermined constant larger than , which is set to 4 for all experiments, and is the initial boundary.
For numerical calculation, the Dirac function is smoothed aswhere is used in all experiments.
Equation (9) is discretized by the central difference, and the approximation iswhere is the central difference operator.
The flowchart of the proposed method is described in Figure 1. The steps of the proposed method are summarized as follows:(1)Initialize the fuzzy level set function using (1).(2)Calculate fuzzy edge indicator function using (8).(3)Initialize the fuzzy level set function .(4)Calculate from by (12).(5)Check whether the convergence of is satisfied; if it is not steady or has not reached the predetermined number of iterations, go to step 4.
3. Experimental Results
We conduct five groups of experiments using synthetic and real images. The same images were also tested by IGAC (improved geometric active contours) model [4] and LIF (local image fitting) model [5]. Due to page limit, we only use a few of the images to demonstrate the effectiveness and usefulness of the proposed approach here. The parameters used here are as follows. The step time can be chosen from 0.1 to 100, and here it is set to 10. The time step and the coefficient must satisfy . The coefficient determines the smoothness of the zero level curve, and it can be chosen from 1 to 30. The coefficient of the weighted area term should be a positive value, so that the contours can shrink faster. Also , , and . All parameters are determined by experiments.
In experiment 1, both the object and background are homogenous. We can see that the LIF method does not work well as shown in Figure 2(b); the IGAC model can perform relatively well with minor errors as shown in Figure 2(c); however, the proposed method can detect the boundaries even better as shown in Figure 2(d).
(a)
(b)
(c)
(d)
In experiment 2, a more complex image with inhomogeneous intensities is tested. The background is homogeneous and the object is inhomogeneous with stepwise gray values. Experiment 2 demonstrates that the proposed method performs better than IGAC model on inhomogeneous images. The LIF method completely failed and cannot converge as shown in Figure 3(b). In Figure 3(c), four regions of the object are wrongly segmented. This is due to the fact that IGAC model tends to drive the zero level curve towards the boundaries corresponding to the gradients and to stop evolving with the strongest boundary response. However, in many cases, the real boundary may not have the strongest response; and IGAC model cannot have sufficient global knowledge to capture the real boundary.
(a)
(b)
(c)
(d)
In experiment 3, the proposed approach, LIF method, and IGAC method are applied to a real image from Amsterdam Library of Object Images (ALOI) [28]. The result of LIF is also very poor as shown in Figure 4(b). After applying the IGAC method, the ill-defined border of the box is not connected well due to the leakages occurring in the weak edges. The result of the proposed approach is shown in Figure 4(d), where the border is well connected and correctly detected as shown in Figure 4(d).
(a)
(b)
(c)
(d)
We have also tested many images with low contrast and nonuniform illuminations selected from Amsterdam Library of Object Images (ALOI). We can observe from Figures 5 and 6 that the proposed method produces good results, and the shapes and edges of the objects can be extracted much better. The IGAC method tends to converge to the interior of the objects and obtains wrong boundaries. The leakages occurred in the week edges. The LIF method performs the poorest among these methods as shown in corresponding Figures 5(b) and 6(b).
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
In experiment 4, methods are applied to the real images from other resources. LIF method generates too many segments as shown in Figures 7(b) and 8(b). More background regions are wrongly covered when using the IGAC method as shown in Figure 7(c). The proposed method can capture the complex boundaries more accurately and achieve better performance than both the IGAC and LIF methods.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
In experiment 5, we use real breast ultrasound (BUS) images [29] to evaluate IGAC, LIF, and the proposed methods. The images are very noisy, with low contrast, and inhomogeneous. Due to high level of inherent speckle noise, LIF produces oversegments as shown in Figures 9(b) and 10(b). In Figure 9(c), IGAC converges to a false boundary and because of that the image is noisy and has blurry boundary of the tumor. In Figure 10(c), although the tumor boundary is quite clear, IGAC still achieves wrong segmentation due to leakage. The proposed method can obtain accurate results as shown in Figures 9(d) and 10(d).
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
For evaluating segmentation results, three area error metrics were used: the true positive (TP) ratio, the false positive (FP) ratio, and the similarity (SI) [30, 31]. They are popularly used for evaluating the performance of segmentation. Let be the object region selected by the algorithm and let be the corresponding real object region; the three error metrics are
The object regions obtained by the algorithms, , are compared with manual delineations which are considered as the grand truths. When the TP ratio is higher, it means that more real object region is covered by ; and when the FP ratio is lower, it means that less background region is covered by . Meanwhile, the higher SI ratio implies that is more similar to ; that is, the overall performance is better. Since LIF oversegments all the images and cannot find the major regions in the background and objects, the following discussion will not utilize the results of LIF. The performances of the IGAC model and proposed method are listed in Table 1.
The TP ratios of the proposed method are much higher than that of the IGAC model (especially in the second and third rows in Table 1), and they indicate that the real object regions in all images were segmented by the proposed method more accurately. Because of low contrast of the edges, there are many local minima and the IGAC model may converge to some local minima, and its TP ratios could be extremely low (Table 1); and the FP ratios of the IGAC model are much higher than that of the proposed method. It means that many background regions are included in the object regions generated by the IGAC model. In addition, the unsuitable regions cannot be cut off easily and the results directly influence the subsequent analysis. The proposed method can handle the blurry and weak boundaries well and the segmentation results are more accurate and reliable. In the last row of Table 1, the FP ratio of the proposed method is a little higher than that of the IGAC model. This is due to the weak edges and blurry boundaries, and the evolving function of IGAC method will tend to converge to the interior of the object; therefore, even if it has lower FP ratio, it achieves severally wrong segmentation. Nevertheless, the proposed method has much higher SI ratios than those of the IGAC model that demonstrate that the overall performance of the proposed method is much better.
4. Conclusions
In this paper, we have developed a novel level set active contour method based on fuzzy logic and variation theory. The proposed approach is more efficient than the level set methods in performing image segmentation due to its capability in handling fuzziness and uncertainty. Three popular area error metrics are used for evaluating segmentation performance. The proposed method and other popular methods (IGAC model and LIF method) are applied to the same images for comparison. The experimental results demonstrate that the proposed method is more accurate and robust even with weak boundaries, noise, and inhomogeneous intensities. This is because the proposed approach takes the advantages of both level set theory and fuzzy logic. It may find wide applications in the related areas.
Competing Interests
The authors declare that they have no competing interests.
Acknowledgments
This work is supported, in part, by National Natural Science Foundation of China and the Civil Aviation Administration of China (Grant no. U1433103).