Abstract
We propose a new variational model for segmenting objects of interest from color images. This model is inspired by the geodesic active contour model, the region-scalable fitting model, the weighted bounded variation model and the active contour models based on the Mumford-Shah model. In order to segment desired objects in color images, the energy functional in our model includes a discrimination function that determines whether an image pixel belongs to the desired objects or not. Compared with other active contour models, our new model cannot only avoid the usual drawback in the level set approach but also detect the objects of interest accurately. Moreover, we investigate the new model mathematically and establish the existence of the minimum to the new energy functional. Finally, numerical results show the effectiveness of our proposed model.
1. Introduction
Image segmentation is one of the fundamental problems in image processing and computer vision. Recently, variational methods have been extensively studied for image segmentation because of their flexibility in modeling and advantages in numerical implementation. In generally these methods can be categorized into two classes: edge-based models [1–3] and region-based models [4–6].
Edge-based models utilize the image gradient to stop evolving contours on object boundaries. The geodesic active contour (GAC) model is one of the most well-known edge-based models. It was proposed in [2] and widely used in practice [7]. However, this model has one major disadvantage: given an initial curve, during the evolution, the energy may evolve to its bad local minimizers. Region-based models have some advantages over the edge-based models. Firstly, region-based models do not utilize the image gradient, so they have better performance for images with weak object boundaries. Secondly, these models are significantly less sensitive to the location of initial contours. The Chan-Vese (CV) model [5] is one of the most popular region-based models. It is successful for an image with two regions, each of which has a distinct mean of pixel intensities. In order to handle images with multiple regions, Vese and Chan proposed the piecewise constant (PC) models [8], in which multiple regions can be represented by multiple level set functions. And yet, these PC models are not very successful for images with intensity inhomogeneity. In order to segment images with intensity inhomogeneity efficiently, local intensity information was incorporated into the active contours models [9–11]. For example, Li et al. [9] proposed the region-scalable fitting model which draws upon local intensity means. This model is able to segment object boundaries accurately. In papers [12, 13], both local intensity means and variances are used to characterize the local intensity distribution in their proposed active contour models. However, these local intensity means and variances must be defined empirically. In addition, similar forms of local intensity means and variances were also introduced in [14] for a statistical interpretation of Mumford-Shah functional [15]. In a word, these methods have a certain capability of handling intensity inhomogeneity. Moreover, there are some models that make full use of advantages of the edge-based and the region-based models, such as the model [7] that unifies the GAC model and the CV model. This model can obtain good results when the contrast between meaningful objects and the background is low.
When the aim is to detect all the objects, the above models are very efficient. However, if only the objects of interest (OOI) are concerned, the segmentation of OOI using active contour methods becomes difficult. In order to detect OOI, the properties of OOI must be known mathematically. However, it is not an easy task to find out a suitable mathematical description of these characters. Many segmentation methods have used shape or texture characters of OOI. For example, the active shape models (ASM) [16], the geodesic active contour method incorporated with shape information [17], and the supervised texture segmentation method [18].
In this paper, inspired by the weighted bounded variation model [19], the geodesic active contour model, the region-scalable fitting model and the active contour models based on the Mumford-Shah model [7], we propose a new model which can be applied to segmenting objects of interest from color images. In order to segment desired objects in color images, the energy functional in our model includes a discrimination function [20] which determines whether an image pixel belongs to the desired objects or not. Moreover, our new model uses not only the edge detector which contains information concerning boundaries of desired objects but also the spatially varying fitting functions which are used to approximate the image intensities. Compared with other active contour models, our method cannot only have the advantages of both edge-based and region-based models, but also avoid the usual drawback in the level set approach (e.g., initiation and reinitiation). In particular, our new model can detect the desired objects accurately. Finally, we investigate this new model mathematically: we establish the existence of the minimum to the energy functional, analyze the property of it and implement the numerical algorithm efficiently.
The remainder of the paper is organized as follows: in Section 2, we show some background. In Section 3, our new model is proposed. Theoretical results, iterating schemes and experimental results are also given in this section. Finally, we conclude our paper in Section 4.
2. Background
2.1. The Region-Scalable Fitting Model
Let be the image domain and be a given gray-scale image. The region-scalable fitting model is defined by minimizing the following energy functional: where is a Gaussian kernel and are two functions that fit image intensities near the point . Moreover, is the level set function embedding the evolving active contour and is the Heaviside function.
This model does not need to reinitialize periodically during the evolution because of the second term of (2.1). If , (2.1) is equivalent to The steady state solution of this gradient flow is And it is known that (2.3) is the gradient descent flow of the following energy:
2.2. Some Related Models
In [2], the geodesic active contour model (GAC) is defined by the following minimization problem: where is the length of the curve and the function is an edge indicator function that vanishes at object boundaries. For a color image , a stopping function was proposed in [21] where is the largest eigenvalue of the structure tensor metric in the spatial-spectral space, and where and , , represent the pixel values of Red, Green, and Blue after Gaussian convolution, respectively, that is, , and .
In order to segment the desired objects in color images, a novel stopping function depending on both the discrimination function of OOI and the image gradient was proposed in [20]: where is a Gaussian kernel and the discrimination function can be defined by the following steps in [20].(1) sample pixels are chosen from the OOI. The color information of the th sample pixel is denoted by . Therefore, is a matrix.(2)Compute the correlation coefficient matrix of as .(3)Compute the eigenvalues of and the corresponding eigenvectors . And these eigenvalues satisfy .(4)Compute .(5)The confidence interval is constructed as with the degree of confidence , where , and is the -value of -distribution with degrees of freedom.(6)Compute . If belongs to the above interval, . If does not belong to the above interval, .
3. Our Proposed Model
Let be a given color image where is the image domain (a bounded open set). Inspired by the GAC model, the energy functional (2.4), and the active contour models based on the Mumford-Shah model [7], our new model is constructed. This model is to minimize the following energy functional w.r.t. : where are positive constants, , is a diffusion coefficient defined as the formula (2.8), are the Gaussian kernels, and is a discrimination function. The purpose of constructing such a discrimination function is to derive the characteristics of desired objects so that these characteristics can be shown in the energy functional. This is done by analyzing sample pixels chosen from the desired objects. The Principal Components Analysis (PCA) and interval estimation [22] are used in the analysis. By using PCA, a new set of variables which is called principal components, is obtained. By using the interval estimation, we can define an interval that covers many samples for each principal component. In our problem, at most the first two principal components are used. Without loss of generality, we only assume the first principal component is used. An interval for this principal component can be constructed by the interval estimation. When every pixel of the color image is projected from its RGB values to the first principal component axis, we get a new value . If the value is within the interval, the pixel is probabilistically regarded as a pixel in the desired objects. Thus the discrimination function based on the color information is constructed as
In order to understand more details of constructing this discrimination function, please see [23].
In the following, we analyze the above model from two aspects. Firstly, for any given , according to the necessary condition of the minimization problem, it is known that the functions must satisfy the following equations: where takes larger values at the points near the center point , and decreases to as goes away from . Therefore, are allowed to vary in space.
Furthermore, for any given , the model (3.1) can be converted into a simpler form. That is
If is limited to a characteristic function , the energy functional (3.4) can be changed into the following form: where is a constant.
In this case, the model (3.4) is equal to the following constrained minimization problem: when approximating of desired objects with spatially varying fitting functions .
The above analysis shows that our new model uses both the edge detector which contains information concerning boundaries of desired objects, and the spatially varying fitting functions () which are used to approximate the image intensities of desired objects. Just because of this, our model can segment the desired objects very accurately.
3.1. Mathematical Results
In [7, 24], the proof of the existence of models has not been given. In the following, we will state the existence of the minimizer to the energy functional (3.4) and analyze the property of it.
Theorem 3.1. For any given (), there exists a function minimizing the energy functional in (3.4).
Proof. Let
By , we get ). Since and , we have
Assume and is the minimizing sequence of (3.4) in , that is, . So there is a positive constant such that
The structure tensor metric is symmetric positive and is the largest eigenvalue of . Thus
where .
Since , and , where is a positive constant. That is, . Then
So is bounded.
Because , the BV-norm of is bounded. Thus, there is a subsequence, also denoted by , and such that strongly in .
Moreover, according to the formula in , we know that there is a subsequence, also denoted by , satisfying almost everywhere for . Since for any , almost everywhere for .
Assume , s.t. and , for any . Let
Then
That is, strongly in and for every .
Furthermore, by the lower semicontinuity for the space, we get
So . According to the dominated convergence theorem, we know
The formulas (3.14)-(3.15) imply the weak lower semicontinuity of the energy functional
Therefore, the infimum is attained by and it is a minimum of the energy functional .
Similar to [7, 25], we can obtain a property of minimizers to the energy functional (3.4).
Theorem 3.2. Let . For any given (), if is any minimum of the energy functional defined in (3.4), then for almost every , the characteristic function is also a global minimum of the functional where is the boundary of the set .
In addition, according to the above theorem, we know is a minimizer of the following minimization problem
3.2. Numerical Implementation
In the numerical algorithm, we do not deal with the new variational model (3.1) directly since too many equations need to be computed. In order to improve computational efficiency, we use the algorithm framework of the paper [24] to deal with our model. That is, we minimize the energy functional by alternating the following steps.(1)Considering fixed, compute and by using the formula (3.3).(2)Considering and fixed, update by using the iterative schemes of the minimization problem (3.4). When a steady state is found, the final segmentation is obtained by thresholding at any level in (in our experiments, we choose ).
In the following, we will give the iterative schemes of the minimization problem (3.4). To solve this minimization problem, we firstly change it into the following unconstrained minimization problem: where is an exact penalty function provided that the constant is chosen large enough. According to the paper [7], it is known that this unconstrained minimization problem has the same set of minimizers as the minimization problem (3.4). The energy functional is convex, so it doesn't possess local minimizers. Hence, any minimizer of is global.
Based on [26–28], a convex regularization is used where is chosen to be small enough so that we almost have . Thus, we consider the iterative schemes of the above weak approximation as the iterative schemes of step (2). Since this functional is convex, its minimum can be computed by minimizing this functional with respect to and separately. That is(1) being fixed, we search for as a solution of (2) being fixed, we search for as a solution of
According to [29], we know the solution of (3.20) can be given by where is given by The previous equation can be solved by the fixed point method Moreover, the solution of (3.21) is given by where
In this paper, our model is solved numerically by alternative minimization schemes and not by using the classical Euler-Lagrange equations method. When using Euler-Lagrange equations, one must consider instead of the weighted TV-norm, where the small parameter is necessary to prevent numerical instabilities. The direct result of this regularization parameter is the obligation to use a small temporal step to ensure a correct minimization process. Thus, a large number of iterations to reach the steady state solution is necessary. In other words, although it is correct, the segmentation process remains slow. In the algorithm of our paper, the iteration process (3.22)–(3.24) is very similar to the standard Chambolle's projection algorithm. The only difference is the appearance of the factor . This iteration process has two important advantages: firstly, it can be implemented very fast since it uses many useful convex optimization tools. Secondly, it is more faithful to the continuous formulation of the energy since it does not use the additional artificial parameter . Therefore, our algorithm is fast and doesn't need so many iterations to reach the steady state. Moreover, the algorithm of our paper is numerically more efficient than classical level set methods since the level set methods need to initialize the active contour in a distance function and reinitialize it periodically during the evolution. In fact, the algorithm of our paper belongs to the algorithm framework mentioned in [24]. And there are many papers [7, 28] that have analyzed the complexity of this kind of algorithms. So we just analyze it simply in this paper. If more details are wanted, please see [7, 24, 28].
3.3. Experimental Results
Our method can be implemented on many synthetic and natural images. First, we consider two different cases Figures 1(a) and 2(a). Figure 1(a) is an image containing three different objects. The blue rectangle is our desired object. Figure 2(a) is not a simple image where the bird is our interested object. Figures 1(b), 2(b), 1(c), and 2(c) display the final active contours and got by using our new model, respectively. From these experimental results, we find our model can detect OOI, regardless of other objects.
(a)
(b)
(c)
(a)
(b)
(c)
In the following, we compare our new model with the model [23] in Figures 3, 4, and 5. Figures 3(a), 4(a), and 5(a) are the original images. Our purpose is to segment the plane in Figure 3(a), the yellow leaves in Figure 4(a) and the green leaves of oranges in Figure 5(a). Figures 3(b), 4(b), 5(b), 3(c), 4(c), and 5(c) denote the final active contours and obtained by using our new model, respectively. According to these results, we see that our model can detect the boundaries of the desired objects very accurately. And these results are hard to achieve by using the model in [23].
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
4. Conclusion
This paper describes a new variational model for segmenting desired objects in color images. This model is inspired by the GAC model, the region-scalable fitting model, the weighted bounded variation model and the active contour models based on the Mumford-Shah model. In order to segment objects of interest from color images, the energy functional in our model includes a discrimination function which determines whether an image pixel belongs to the desired objects or not. Compared with other active contour models, our new model cannot only avoid the usual drawback in the level set approach but also detect the desired objects accurately. Our numerical results confirm the effectiveness of our algorithm. Moreover, we establish the existence of the minimum to the new energy functional and analyze the property of it.
Acknowledgments
The authors wish to thank the referees for their valuable corrections and suggestions to the original manuscript. This work is supported by “the Fundamental Research Funds for the Central Universities” (Grant no. HIT. NSRIF. 2009049) and also supported by Natural Sciences Foundation of Heilongjiang Province (A200909).