Abstract
We propose a multilabel segmentation that aims to partition a texture image into multiple regions based on a homogeneity condition using local entropy measured at varying scales. For multi-label segmentation, a bipartitioning segmentation scheme is recursively applied to confined regions obtained by previous segmentation steps. The empirical entropy is measured in the local neighbourhoods at varying scales, which is used as a characteristic feature in determining the spatial regularity of elementary texture structures. The experimental results on a variety of texture images demonstrate the efficiency and robustness of the proposed algorithm.
1. Introduction
Image segmentation plays a significant role in understanding visual scenes. This critical task involves partitioning an image domain into subdomains based on a statistical homogeneity criterion. There have been a large number of features developed for measuring the stationarity of intrinsic texture elements within textures. The most common approach is to employ a class of filter banks [1], but this does not incorporate the notion of scale in the segmentation procedure. Alternatively, an image decomposition approach has been proposed in [2] based on the total variation denoising scheme [3]. In the characterisation of texture features, entropy is a potentially effective measure in computing homogeneity of constituent texture regions. In terms of information theory, entropy measures the amount of the uncertainty associated with a random variation. An image segmentation procedure that mainly consists of region determination and parameter estimation steps and entropy measure can be naturally considered for estimating parameters associated with regions to determine. In the estimation procedure based on local entropy at each location, it is necessary to compute the probability density function within a certain neighbourhood. The size of local neighbourhood is related to the scale of texture, and the elementary size of the texture is often called texton. There have been some relevant works that deal with segmentation evaluation methods based on entropy measures [4–6]. The objective of these works is to quantitatively evaluate the performance of a segmentation algorithm based on the homogeneity assumption developed using entropy measure. In their methods, it is assumed that a set of segmented regions are given and the uniformity of intensity within each region is maximised which is equivalent to the minimum of the normalised empirical entropy. In this work, we first introduce a texture scale and propose a local entropy measure at varying scales as a characteristic feature in order to take a range of scales into consideration. A convex bipartitioning segmentation method is then recursively applied to obtain multiple partitions based on the local entropy signature. To the best of our knowledge, most previous approaches for texture segmentation only deal with a single scale in the characterisation of intrinsic texture properties based on entropy measures in a global way, whereas our method characterises texture features at a range of scales using local entropy measures so that a variety of texture properties at multiple scales are considered in determination of partitioning regions. In addition, the majority of existing algorithms for texture algorithms are based on the Mumford-Shah functional [7] that is designed to partition an image domain into two distinctive regions. In contrast, we propose a recursive approach for a hierarchical multilabel segmentation that yields multiple segmenting regions in our method. As opposed to prior approaches that are based on nonconvex energy formulations, our energy functional is developed in a convex form that allows a global solution, independent of initial condition.
2. Texture Model
Let be an image, where is the image domain. A texture , is a subregion that exhibits spatial regularity under a certain statistical criterion that is designed to effectively characterise texture properties. The elementary structure of a texture , called a texton, is regularly repeated within a texture with a space of permissible variations in shape or appearance, which ranges over the preattentive human texture perception. A statistical discrepancy , is employed for measuring the statistical homogeneity within a texture region , where a sufficient statistic is established to satisfy the stationarity condition for all and for some with a symmetry boundary condition. An essential property associated with the statistical measure regarding the homogeneity of a texture is an invariance that satisfies where is an element of a certain transformation group. The notion of scale related to characterising texture is introduced by the cardinality of the texton denoted by such that provides a minimal sufficient statistic for describing a texture. The scale of a given texture can be obtained by the optimisation problem in a variational framework: where is a control parameter on the regularity. The texture scale is then determined by . It is assumed that an image generally consists of various textures that are characterised at multiple scales, which leads to propose a scale-entropy signature as a characteristic feature at varying scales as described in the following section.
3. Local Entropy Signature
We propose a texture descriptor that is developed in an information theoretical approach. The local statistical measure is associated with a range of scales, which form a signature at each point in image . Let be the local neighbourhood centred at with scale . We consider a rectangular neighbourhood with a size of using the -norm () for simplicity, but any -norm () can be used for a different shape of the neighbourhood. Let be the probability density function given by where is the local image patch with scale and denotes the Dirac delta function. The empirical entropy is a measure of the uncertainty in image intensity within the local neighbourhood and is proposed as a feature in texture characterisation. A desirable translation-invariant property can be achieved using the entropy measure within the same texture if the scale of the local neighbourhood is larger than the scale of the texture. For a given image , we compute the empirical entropy at with a scale based on the probability density function in the neighbourhood as defined by
For each point , provides the uncertainty of the image structure in the local neighbourhood and enables hierarchical texture analysis at varying scales. The spatial regularity of a local image structure needs to be estimated when determining the boundary of a texture, which leads to introducing a discrepancy measure, and we use the optimal transport distance, commonly called the Wasserstein distance [8], defined by where and are two different local entropy signatures and denotes the range of scales at which the local entropy signatures are computed.
4. Multilabel Texture Segmentation
We propose a multilabel segmentation algorithm based on local entropy signatures for partitioning an image into different texture regions. A multilabel segmentation is to divide the image domain into mutually disjoint subdomains such that , , and if . We consider the following energy functional for a multilabel segmentation problem: where is the number of partitions, is the local entropy signature within a region , is a control parameter, and denotes the arc length parameter. The first term represents the data fidelity, and the second term represents the regularization. The regions are desired to be determined based on the statistical homogeneity measured by an affinity function in terms of the local entropy signature . The affinity function is defined by where is the local entropy signature of an image and is a constant entropy signature. The desired multilabel texture segmentation is achieved by optimising the segmentation energy in regions and the approximation parameters with the number of regions fixed. An alternative optimisation procedure follows an expectation-maximization (EM) algorithm where we optimise in given and then in given . Given the region , the approximation can be solved in the closed form by solving the equation: which results in in . Given an estimate for , a convex segmentation scheme [9] based on the piecewise constant model [10] is applied to optimise for that is represented by a smooth function as follows: where . This energy functional is convex, and it can be efficiently minimised by the dual formulation of the total variation norm as described in [9]. The optimisation algorithm for the energy functional in (9) is achieved using the primal-dual algorithm [11] that is efficient to alternatively minimise the energy by introducing a dual variable. The energy in (9) is designed for a bipartitioning segmentation, and we apply a bipartitioning segmentation on the segmented regions in a recursive way, similarly to the method described in [12] to achieve multilabel segmentation as follows: where is a binary mask used for confining the domain to one of the segmented regions obtained by the previous segmentation steps. This procedure is recursively applied to obtain partitions.
5. Experimental Results
The performance of the proposed algorithm is evaluated on the Berkeley database [13], which includes a variety of texture images. For each image, we compute the local entropy at a range of scales from 1 to 30. The selection of the scale range of local entropy may vary depending on the texture property of the image to segment. A segmentation algorithm is then applied based on the local entropy signature, and a recursive approach is used to obtain a different number of partitions. Figures 1 and 2 show the original images on the leftmost and the computed local entropy at different sampled scales on the middle block of images. The images consist of regions of different texture scales, and it is shown that different texture regions are effectively characterised at their corresponding entropy scales. Thus, it is necessary to consider the local entropy signature computed at a range of scales in the segmentation procedure. On the rightmost columns in Figures 1 and 2, the segmentation results based on the local entropy signatures are presented. In the optimisation of the segmentation energy, the initialisation is given by the random labels, and the regularisation parameter and the number of labels are selected based on the best visual match between the image and the labels that are represented by different colours. As shown in the results, our segmentation algorithm accurately delineates the texture objects, since the local entropy computed at multiple scales effectively characterises different texture regions at various scales. The texture scale is not determined a priori and the experimental results indicate that multiscale entropy signatures are robust to delineate the boundary of texture objects with different scales.
(a)
(b)
(c)
(a)
(b)
(c)
Figure 3 shows segmentation results based on a number of images with textured objects in the Berkeley database [13] using the proposed model and other methods for the purpose of comparative analysis. In Figure 3, the first row shows the original images, and the second row shows the segmentation results using the fast global minimisation of active contour (GAC) [9] that is developed based on the assumption that an image is formed by a piecewise constant function and therefore yields poor results on images with texture patterns. The third and fourth rows show the segmentation results by the local histogram based segmentation method using the Wasserstein distance (LHSWD) [8] that is based on the homogeneity of local probability density function obtained within varying neighbourhood sizes. In the computation of local histogram for the probability density function, the binning size is set to be 100, and the sizes of neighbourhood are set to be 10 for the third row and 30 for the fourth row, respectively. The associated parameters with LHSWD algorithm are and . The fifth row presents the segmentation results obtained by the proposed method with the same values for the common parameters, and . The segmentation results with the proposed algorithm are far better than other methods, GAC and LHSWD, as one can see that the patterns of tiger, cheetah, and fish are more accurately segmented. This indicates that rich statistical information across multiple scales is beneficial to characterise texture objects in segmentation, rather than using a fixed scale. Our proposed algorithm is built on a convex relaxation of region-based segmentation algorithm [13] that is computationally more efficient than a conventional region-based segmentation algorithm in level set framework [10]. The computational advantage is due to the convex form of energy, which yields a global solution independent of initial condition and an efficient optimisation scheme based on the primal dual projection scheme [11].
(a)
(b)
(c)
(d)
(e)
6. Conclusion
We have proposed a multilabel texture segmentation algorithm based on local entropy signature that is designed to characterise local texture features at varying scales. It has been shown that our local entropy signature is in particular effective to account for texture images that consist of textures with different scales. It would be potentially beneficial if the texture scale is computed and incorporated into the segmentation energy for further studies.
Acknowledgment
This research was supported by the Chung-Ang University Research Scholarship Grants in 2010.