Abstract
We describe a new method for simultaneous image denoising and level set-based active contour segmentation using multidimensional features. We consider an image to be a surface embedded in a Riemannian manifold. By defining a metric in the embedded space, which in our case includes multidimensional image features as well as a level set-based active contour model, a minimization problem in the image space can be obtained through the Polyakov action framework. The resulting minimization problem is solved with a dual algorithm for efficiency. Benefits of this new method include the fact that it is independent of any artificial “running” parameters, and experiments using both synthetic and real images show that the method is robust with respect to noise and blurry object boundaries.
1. Introduction
Unsupervised image segmentation is an important problem with many applications in science, including medical imaging. Image segmentation is a postprocessing problem in many computer vision tasks; its aim is to divide an image into finite number of subregions. The features of different subregions are utilized as the segmentation criteria. The statistical methods, such as expectation-maximization (EM) algorithm [1] and fuzzy C-means clustering (FCM) algorithm [2], are applied in classifying the pixels based on some particular image features segmentation criteria. In general, the statistical methods achieve the classification based on only one segmentation criterion. However, there is various kinds of features in an image and the features may vary spatially. Therefore it will be not precise to use one kind of these methods. How to extract the features of an image and how to utilize these features as the segmentation criterion are significant for segmentation.
Many works utilize the difference between invariable pixel intensities, as well as their spatial connectivity, in assessing whether two pixels belong to the same object. These active contour models based on the level set method [3] classify the pixels by only one image feature, that is, the image intensity based on uniform distribution [4–6]. Nevertheless, the image intensity varies spatially; thus the image intensity is not necessarily described by one kind of specific distribution. For improving the precision, the works of [7, 8] extract the multifeature to deal with more complex information content. Simultaneously, the additional artificial parameters are introduced; thus it needs the experience to set the parameters.
The Polyakov action was introduced in image processing by Sochen et al. in [9]. This segmentation model is different from the other segmentation methods in two ways. First, images are represented as Riemannian manifolds embedded in a higher dimensional spatial-feature manifold. Second, the Polyakov action provides an efficient mathematical framework to embed the multifeature of images in higher-dimensional Riemannian manifolds by harmonic maps. Bresson et al. [10] propose active contour models based on the Polyakov action. These models map several kinds of features, for example, color and texture, into higher dimensional space. Because these models choose a metric with artificial parameters on the feature space, it requires careful manual parameter-tuning.
In this paper, the proposed active contour model is formulated in the framework of the Polyakov action [9]. Unlike the other related works [7–9], a metric on the feature space manifold is defined by the invariant geometry of images. Consequently, the proposed method is purely based on the geometrical features of images without any artificial parameters. We implement the segmentation through two steps. First, an approximated image, removing the noise while preserving the main structures, is found in the feature space built on geometrical features of the original image. Second, the active contour is embedded into the feature space built on both the statistical and geometrical features of the approximated image. For efficiency, we solve the proposed model via the improved Chambolle dual formulation [10] of the minimization problem.
The paper is organized as follows. In Section 2, we introduce the mathematical framework based on the Polyakov action. In Section 3, we introduce the proposed model and the numerical algorithm of the proposed method is also summarized. In Section 4, we validate our model by some experiments on medical images. In Section 5, we end the paper by a brief conclusion.
2. Geometrical Framework Based on Weighted Polyakov Action
Sochen et al. introduce a general geometrical framework for low-level vision, based on the Polyakov action [9]. In this framework, images are represented as the surfaces on a Riemannian manifold. The Polyakov action is a functional that measures the weight of a mapping between an -dimensional embedded manifold (e.g., the image manifold) with coordinates and the -dimensional manifold with the coordinates , . A Riemannian structure metric can be introduced to measure the local distances on the embedded manifold , whereas we use the metric to measure the distance on the manifold . To measure the weight of the mapping , the Polyakov action is used as a generalization of the -norm on the embedded image to space feature manifold :where is the determinant of the image metric tensor and is its inverse. The metric is chosen as the induced metric, obtained by the pullback relation: ; the Polyakov energy is shortened to
In the relevant works [7, 8], the authors get the denoised image and the segmentation results by minimizing the energy functional (2) with respect to denoising and segmentation, respectively. In seminal work [9], they embed grey images in the feature , where is the grey intensity value for pixel . They choose a metric ; is a constant. Based on this metric on feature space and the Polyakov energy, the regularization term on the intensity values is given by . Although it allows setting the scale of the feature dimension independently of the spatial dimensions, the accuracy of the scale is subject to the artificial parameter .
3. The Active Contour Model in Multifeature Space
In this work, we utilize an improved geometrical framework based on the weighted Polyakov action without any artificial parameter. First, we get an approximated image by embedding it into the feature space constituted by the features of the original image. Second, given the approximated image, active contour is driven by embedding the level set function into the higher dimensional feature space composed of the geometrical and statistical features of the approximated image.
3.1. Approximating Image under an Improved Geometrical Framework
The original image is defined on the image manifold with coordinates . The approximated image is defined on the image manifold and denoted by . To preserve the main edges of the original image, we extract the geometrical features of edges, , derived from the anisotropic diffusion equation [2]. Considering the intensity value as another feature, we build the feature space, , denoted by for the sake of simplicity. To avoid the influence of the artificial parameter, we choose a metric tensor on the feature space , which is defined by the invariant geometry of the original image . Consider . The pullback relation yields the determinant of metric tensor on manifold :Analogizing based on the Polyakov energy (2), we get the approximated image by minimizing the energy functional as follows:where the weight coefficient of second term, corresponding to the third element of the metric , denotes the coefficients of first fundamental form in differential geometry. When this weight coefficient is larger, the edge structure is enhanced in the vicinity of the edges; otherwise, smoothing the image is strengthened. The weight coefficient of the third term, corresponding to the last element of the metric , denotes the coefficients of second fundamental form in differential geometry. Approximating the intensity is strengthened when this coefficient is larger, whereas smoothing the image is strengthened when the weight is smaller.
3.2. Active Contour Evolution under the Improved Geometrical Framework
The active contour is represented as the zero level set function on the image manifold . For avoiding the effects of the intensity nonuniformity, we extract the statistical features on the local region of size , where , denote the mean intensity in the local region inside and outside the zero level set. The feature space is , denoted by . The metric tensor defined on this feature space is . The pullback relation yields the determinant of metric tensor on manifold :According to the Polyakov energy (2), we drive the curve evolution by minimizing the energy functional as follows:where the weight of first term is actually an edge detector. The curve evolution tends to stop when it decreases to zero, whereas the evolution goes on.
3.3. Dual Algorithm
To apply the dual gradient algorithm, we introduce the dual variable, . The total variation term in (4) and (6) can be formulated as follows:The approximation formulation of the energy of our model can be rewritten aswhere . We then apply the split Chambolle dual algorithm [10] to solve the optimization problem.
Introducing the auxiliary variables , , solving the energy functional (8) is equivalent to minimizing the problem as follows:where the parameter is chosen to be small for avoiding smearing the edges (in this paper, we choose .), is an exact penalty function provided that the constant is chosen large enough compared to such as , , and . The minimization problem (9) can be divided into four subproblems as follows and can be solved alternatively.
(a) Given image , , update . we search for as the solution of The solution of (10) is given bywhere can be updated by fixed point method: initializing and updating In this paper, we choose to ensure convergence.
(b) Given , , we search for by solving the minimization problem as follows:The solution of (13) is given byEquation (14) is solved by a fixed point method:
(c) Given the solution of , we search by solvingThe solution of (16) is given by
(d) Given , , we search for as the solution ofThe solution of (18) is given byAfter is solved, it is utilized in (12) for next iteration.
The algorithm of minimizing our model is described in the following.
Step 1. Initialize .
Step 2. Given the fixed threshold of iterations , if , then stop; else go to Step 3.
Step 3. Do the iteration for solving subproblem.
Step 3.1. While do
Step 3.2. Update the dual variable by (14) and then update by (13); go to Step 3.3.
Step 3.3. Given , update the dual variable according to (15) and then update by (16); go to Step 3.4.
Step 3.4. Given , , compute and update by (17). Go to Step 3.5.
Step 3.5. Given , , update by (19); then go to Step 3.1; otherwise, go to Step 4.
Step 4. End while.
4. Experimental Results
All the experiments are run with Matlab code on the PC of CPU 3.2 GHz, RAM 728 M. we show the experiments results for medical image segmentation of Chan-Vese model (CV) [11], the structure-based level set method (SLM) [12], and the region-scale fitting model (RSF) [4]. Figure 1 shows the experiments on the synthesized noisy images. This image is of size with 10% white Gaussian noise.
(a) Original image
(b) CV model 200th
(c) SLM 50th
(d) The proposed model 50th
(e) The approximated image
As shown in Figures 1(b) and 1(c), the CV model and the SLM models are sensitive to noise. As shown in Figure 1(d), the proposed model is robust to noise and obtain the correct boundary. The CV model generates the unwanted contours because of the strong noise. Based on the edge detector function, the SLM model is more robust to the noise than the CV model. The result of the proposed model shows that it is able to extract the real object boundary even when the noise is strong.
Figure 2 is brain magnetic resonance image (MRI) of size with 2% noise and 10% level intensity nonuniformity. The brain MRI mainly consists of three parts: the cerebrospinal fluid, the gray matter, and the white matter. The cerebrospinal fluid is the dark matter which exists in two places: the middle of the brain surrounded by the gray matter and the gap between the cranium and the brain. The task of segmenting the brain MRI is to extract the contour profile between the white matter and the gray matter. Since the contrast of boundary between the white matter and the gray matter is lower than the boundary between the cerebrospinal fluid and the gray matter, the latter is always extracted wrongly as the object boundary. As shown in Figures 2(b) and 2(c), the CV model and the SLM model cannot extract the object boundary with low-contrast. Although the RSF model can extract the boundaries with low-contrast, it also extracts some unwanted objects. The segmentation results of the proposed model show that it is robust to noise and can extract more boundaries with low-contrast.
(a)
(b)
(c)
(d)
(e)
(f)
Figure 3 is a brain MRI of size with 5% noise and 40% intensity nonuniformity. As shown in Figures 3(b) and 3(c), the drawbacks of the RSF model and the SLM model still exist. Compared with the other methods, the proposed method clearly extracts more object boundaries with low-contrast between the gray matter and the white matter.
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4 show the segmentation results of the active contour methods based on multilayer level set functions. By the multilayer level set functions, the cerebrospinal fluid, the gray matter, and the white matter can be extracted simultaneously. In Figures 4(b) and 4(f), the results of CV model show that the cerebrospinal fluid is not extracted completely. As shown in Figures 4(c) and 4(g), the boundaries between the white and gray matters are not extracted completely by SLM model. We can observe that in Figures 4(d) and 4(h), the results of the proposed method based on multilayer level set functions show that the completed boundaries of the cerebrospinal fluid, the gray matter, and the white matter are extracted simultaneously. To show the convergence speed of the compared methods, Table 1 shows the iteration numbers and processing time in each iteration for the CV model, the SLM model based on the steepest descent method, and the proposed model with both single- and multilayer level set functions. Table 2 shows the segmentation accuracy of the compared active contour models based on multilayer level set functions.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
The data in Figure 5 is download from the website [13]. We also show the segmentation accuracy in Table 3 by the DICE metric [14], compared with the ground truth given in this website. We can see from Figures 5(c) and 5(d) that the CV model and the SLM model only extract the boundaries with high contrast, while the object boundaries between the gray matter and white matter are not extracted. And some CSF of the image is not extracted. The proposed method can extract the more completed CSF and can preserve the cerebral cortex more efficiently.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
5. Conclusion
In this paper, we propose a new variational model for image segmentation and image denoising simultaneously. We obtain the approximated image by embedding the approximating criteria into a specific multifeature space. And then the segmentation result is obtained by embedding the active contour into another multifeature space which is composed by the segmentation criteria depending on the approximated image. The segmentation and the denoising problems are solved by the split Chambolle dual algorithm alternately. The comparisons of the other popular segmentation models demonstrate the accuracy and efficiency of the proposed model.
Additional Points
The following are the research highlights of this paper. The proposed variational model incorporates segmentation and denoising together. Segmentation and denoising processing are achieved alternately by the Polyakov action framework. An improved Polyakov action framework is purely based on the geometric features of the image without any manual parameters. Minimizing the variational model is achieved by the improved Chambolle algorithm.
Competing Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported in part by the Natural Science Foundation Science Foundation of China under Grant nos. 61502244, 61402239, and 71301081, the Science Foundation of Jiangsu Province under Grant nos. BK20150859, BK20130868, and BK20130877, the Science Foundation of Jiangsu Province University (15KJB520028), NJUPT Talent Introduction Foundation (NY213007), NJUPT Advanced Institute Open Foundation (XJKY14012), China Postdoctoral Science Foundation (2015M580433, 2014M551637), and Postdoctoral Science Foundation of Jiangsu Province (1401046C).