Abstract
According to the characteristics of high-resolution remote sensing (RS) images, a new multifeature segmentation method of high-resolution remote sensing images combining the spectrum, shape, and texture features based on graph theory is presented in the paper. Firstly, the quadtree segmentation method is used to partition the original image. Secondly, the spectrum, shape, and texture weight components are calculated all based on the constructed graph. The matching degree between pixels and the texture is computed similarity. Finally, the ratio cut standards combination of the spectrum, shape, and texture weight components is used for the final segmentation. The experimental results show that this method can obtain more ideal results and higher segmentation accuracy applied to RS image than those traditional methods.
1. Introduction
In the past decade, many scholars have done the research about graph-based approaches to image segmentation. Graph theory has got a lot of attention because of its representational power and flexibility properties. Bai et al. [1] apply graph cut (GC) theory to the classification of hyperspectral remote sensing images. Fuzzy SVM classifier and the GC-based classification were used in two-step classification strategy in this paper. Felzenszwalb and Huttenlocher [2] define a predicate for measuring the evidence for a boundary between two regions using a graph-based representation. The time complexity of the method runs in time for graph edges. The work of Wang and Siskind [3] presents cut ratio as a new cost function of graph methods for segmenting image. This method is useful for some image segmentation applications. Cui and Zhang [4] use Minimum Span Tree optimal theory to realize object based on high-resolution image segmentation. The result proved that this method can obtain high quality segmentation. Kato et al. [5, 6] propose a Markov random field (MRF) image segmentation model based on the integration of colour and texture descriptors. This method can use both synthetic and natural color images. Another early approach to image segmentation based on graph cut has been proposed [7–15].
The objectives of this paper are to obtain better image segmentation results and relative high segmentation accuracy for high-resolution RS images. For RS image, the algorithm’s time complexity and space complexity rate will be high if only graph theory for image segmentation is used. Quarter-tree segmentation method is a fast image segmentation algorithm, but it cannot divide meaningful target area for RS images. If the threshold is set too low, oversegmentation phenomenon will be very serious. If the threshold is set too high, it cannot form a more accurate target edge [1, 13]. Therefore, this paper designs a new method by combining the merits of quarter-tree segmentation and ratio cut algorithm, and the method can be used in high-resolution RS images. This method is effective to reduce the size of the graph vertices, improving the accuracy of image segmentation. It first establishes the mapping relation of RS image and graph and then sets an energy function of the graph according to the remarkable weights. We can solve the energy function to get the minimum which will lead to the result of graph segmentation. At last we mapped the graph segmentation result back to image. Because the construction of graph and the extraction of remarkable weights can be based on both pixel and image blocks, the methods based on graph theory will be good in image segmentation.
The organization of this paper is as follows. In the second section, quadtree segmentation method and -cut theory for multifeature segmentation of RS image are described. Results and discussion are given in Section 3. The final section is the conclusions.
2. The -Cut Theory for Multifeature Segmentation of RS Image
2.1. Quadtree Segmentation Method
A quadtree is a tree data structure in which each internal node has exactly four children. Quadtrees are most often used to zone a two-dimensional space by recursively subdividing it into four quadrants or regions [16]. Quadtree decomposition is currently a valuable method in image processing and computer graphics. The procedure of quadtree segmentation is as follows.
Step 1. The original image (typically ) is divided into four same size regions.
Step 2. It is to detect the constant gray level of each region segmentation image.
Step 3. If it cannot meet the request of constant gray level of the image, then each district will be divided into four areas of the same size and go to Step 2.
Step 4. If it meets the request, then stop the iterative process.
2.2. -Cut Standards
The graph partition problem is defined on data represented in the form of a graph , with vertices and edges. Where in form, corresponds to the image, the vertices correspond to regions, and the edges correspond to adjacent relations between the regions. Ratio cut represents the ratio of the corresponding sums of two different weights of edges along the cut boundary. A minimum ratio cut refers to the smallest cut ratio [3].
The energy function of -cut standards is as follows:where and represent two different image blocks, respectively, and and calculated the energy of cut sets by using two methods of weight calculation, respectively:where denotes the vertex of , denotes the vertex of , and denote the weights associated with each edge , respectively, is the first edge weight, and is the second edge weight.
2.3. -Cut Reduction Algorithm
The -cut reduction algorithm steps are as follows.
Step 1 (calculate minimum ratio cut). In order to simplify the calculation, we calculate the minimum ratio ring instead of the minimum ratio. The dual graph of the graph is constructed and shown in Figure 1. There is a one-to-one correspondence between the minimum ratio cut of graph and the minimum ratio ring of the dual graph , therefore, the problem that calculate the minimum ratio cut set of the original can be transformed into the minimum ratio ring of the dual graph .
Step 2 (calculate the minimum ratio ring). In order to simplify the calculation, the negative simple ring is computed instead of the minimum ratio ring. The representations of the dual graph with the linear conversion , where is obtained by doing the conversion: and functions and are the edge weights function of graph . The conversion of weights function will not change the minimum ratio ring loop of the dual graph , so it will not change the minimum ratio cut of the original . Only the minimum loop cost of is nil consideration, and the graph has a minimum ratio of the loop which contains the loop ratio . The graph has a minimum ratio of the loop which contains the loop ratio , if and only if the minimum loop cost of is equal to zero. The relationship between and is as follows.
If has a negative cost loop, then . While if does not have negative cost loop, then . Let and , respectively, be the minimum and maximum loop ratio of , so and . Then, . Let ; if has a negative cost loop, then value is set to ; otherwise the value is set to , continuing the repeated calculation, until we cannot find a negative cost of simple loop concerning the one corresponding to ; and now is the minimum loop ratio and at the same time the negative cost simple loop of is the negative cost simple loop which we want to find.
Step 3 (calculate the minimum cost perfect matching). In order to reduce the calculation, the negative simple ring is computed instead of the minimum cost perfect matching. Construct a new graph from the graph which is obtained by the previous step. Graph contains a negative cost loop, if and only if has the minimum cost perfect matching. From graph to graph , the specific conversion principles are as follows.(i)For each vertex of graph , graph contains two vertices and an edge of which weight value is equal to zero.(ii)For each edge of graph , graph contains two corresponding vertices and and five corresponding edges. Figure 2 shows the weight value of the five edges.
According to which is obtained through the above three steps, we can calculate the and and obtain the minimum ratio cut according to formula (1).
2.4. Weight Calculation
In this paper, a multifeature segmentation method which takes into account the spectrum, shape, and texture features of RS image is applied.
The weight component based on the spectrum is defined as , the weight component based on the shape is defined as , the weight component based on the texture is defined as , and is the combination of the above three aspects of information [8]:The weight component based on the spectrum isIn formula (4), denotes the standard deviation of the pixel color. can be described as follows:where is the number of filter’s types. If and correspond to two pixels, is used to record the spectral similarity between the pixels and . If and are corresponding to the two blocks, is used to record the spectral similarity between the blocks and .
The weight component based on the shape iswhere is used to mark the matching degree between the two pixels or blocks and , which is obtained by calculating the maximum value of all the probability coefficient along the line of the set of pixels after connecting and in a straight line. If this line exactly intersects with a profile, then is large, the weight is small, and and may belong to two classes; on the contrary, if the line is parallel to the profile, then is small, the weight is greater, and and may belong to the same class.
The weight component based on the texture iswhere and are the histogram obtained by doing texture operator transform for and , respectively, and denotes the standard deviation of the texture of object. is used to record the texture similarity between and . If the difference between and is too large, the values of will be large, and is very small. So and do not belong to the same class [1–4, 8].
The algorithm flowchart is shown in Figure 3.
3. Experimental Results and Analysis
3.1. Evaluation Method of Segmentation Results
Evaluation method of image segmentation is divided into qualitative and quantitative analysis.
This paper makes a quantitative analysis on the segmentation results by using the theory proposed in [14, 17–20]. The specific evaluation index is defined as follows.
3.1.1. Homogeneity
From [14], we handle the standard deviation of all the pixels as a measure of the object homogeneity criterion. The standard deviation of the object can be written aswhere is the number of all pixels within the object, represents the pixel gray value of pixel , and represents the gray mean of the object.
3.1.2. Heterogeneity
For each object, we calculate the average difference absolute value of the object with the neighborhood to reflect the degree of difference between the object and the adjacent object [14]. The formula of heterogeneity can be written aswhere is the boundary length of the current object, is the common edge length of the current object with adjacent objects, is the gray mean of the current object, is the gray mean of adjacent objects, and is the number of adjacent objects with the current object.
3.1.3. Segmentation Evaluation Index (SEI)
SEI is inversely proportional with its homogeneity and proportional with its heterogeneity [14]. The SEI of the object is defined as follows:
3.1.4. Probabilistic Rand Index (PRI)
The Probabilistic Rand Index (PRI) counts the similarity of pairs of pixels whose labels are consistent between the computed segmentation and the ground truth. The expression of PRI can be defined asThis measure takes values in when two images have no similarities, and when two images are identical, where is the segmentation that is to be compared with the reference segmentation image and is ground-truth segmentations, where denotes the event of a pair of pixels and having the same label and its probability [17–19].
3.1.5. The Variation of Information (VoI)
The Variation of Information (VoI) metric defines the distance between two segmentations as the average conditional entropy of one segmentation given the other one and thus roughly measures the amount of randomness in one segmentation which can be explained by the other [20]. The formula of VoI can be written aswhere and , respectively, represent the entropies and the mutual information between two clustering of data and data . This measure takes values in .
3.1.6. The Global Consistency Error (GCE)
The Global Consistency Error (GCE) measures the extent to which one segmentation can be viewed as a refinement of the other. GCE can be defined as and are input segmentations images. and are the local refinement error, respectively. is zero precisely when is a refinement of as pixel , but not vice versa. [21].
3.2. The Experiment Results
In this section, we apply the proposed algorithms to real high-resolution data by the ALOS high-resolution RS images of Shi Zuishan Industrial Park, Ningxia, China. Its ground spatial resolution is 2.5 m and the size is pixels. According to the human visual, field surveys and spectral measurement results, we select five samples, lime pile, cinder heap, house, road, and wasteland. To validate the algorithm, the images were segmented from the spectral, shape, texture, and multifeature segmentation based on graph theory, respectively, and then made a comparison among the four segmentation results. Original image and various algorithms segmentation results are shown in Figures 4 and 5, respectively. From Figure 4, the spectral segmentation based on graph theory has certain limitation; it is prone to split too small for textured areas, but it is less likely to split for the areas of relatively close texture. The shape segmentation based on graph theory is prone to split too small. Although the method of the texture segmentation based on graph theory can get a better segmentation of all types of surface features, it is not obvious to the boundary between the surface features. The multifeature segmentation method of remote sensing images based on graph theory not only can make the measurement, the spectra, and texture information of different objects better, reflect the differences between the different types of surface features, and achieve better segmentation, but also can accurately obtain the boundary between different types of objects; in short, it can ensure the accuracy of subsequent analysis.
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
To be more accurate and objective evaluation of segmentation results of the algorithm, this paper uses the above-mentioned evaluation method for quantitative evaluation of the segmentation results. The lime heap and cinder heap are selected as evaluation object because the paper mainly monitors industry solid waste. Specific segmentation scale evaluation results of origin RS image 1 are shown in Table 1.
From Table 1, we can see that the lime heap homogeneity index is 2.1045 with the multifeature segmentation method of remote sensing images based on graph theory, which is smaller than the lime heap homogeneity index of spectral-based and shape-based segmentation, and the cinder heap homogeneity index is 0.9877, which is smaller than the cinder heap homogeneity index of texture-based segmentation and is close to the cinder heap homogeneity index of shape-based segmentation; this comparison and contrast herein prove that, by using the multifeature segmentation method based on graph theory, one can get a better measurement of the spectrum and texture information of surface features. The heterogeneity index of the multifeature segmentation based on graph theory is greater than the other three segmentation methods, and it further shows that the multifeature segmentation method based on graph theory can obtain more precise boundaries between different types of surface features. In summary, the results of the multifeature segmentation method based on graph theory make good internal object homogeneity, and at the same time there is an obvious contrast between adjacent objects.
The results obtained with the other segmentation methods and the proposed algorithm over two high-resolution RS images are shown in Table 2. The parameters of PRI, GCA, and VoI of each segmentation methods are computed. Quadtree, watershed, mean shift, multiresolution, and the proposed method are the region-based segmentation methods. The quadtree method starts at the root of the tree that represents the whole image. If it is found nonuniform (not homogeneous), then it splits into four son squares (the splitting process). If, in contrast, four son squares are homogeneous, they are merged as several connected components (the merging process). This process continues recursively until no further splits or merges are possible. The multiresolution approach partitions the image at different scale, using a pyramid or quadtree structure. The watershed approach considers the gradient magnitude of an image as a topographic surface. Pixels having the highest gradient magnitude intensities correspond to watershed lines, which indicate the region boundaries. Mean shift method is defined as finding modes in a set of data samples, showing an underlying probability density function. Canny operator and sobel operator are boundary-based segmentation method.
From Table 2, taking into account the quality of the results from the evaluation parameters, it will be noticed that the best results are reached by the proposed method. The value of PRI of this method is the highest compared with the other segmentation methods. This is mainly due to the fact that this method combines the spectrum, shape, and texture of image and the segmentation region is close to real region. From the results of Table 2, we can see that the results of region-based segmentation methods are better than the boundary-based segmentation methods.
4. Summary
In this paper, we took into account a number of feature information of the image and used -cut theory for RS images segmentation. Experimental comparison shows that multifeature segmentation method based on graph theory achieved better segmentation results than the methods based on single feature. Overall, the method can be used in high-resolution RS images. Even though the method also has shortcomings, such as the effectiveness and the implementing speed of algorithms which are not very satisfactory, in the future, we will keep on seeking an efficient solving process and the weight calculation formula to apply to RS image segmentation.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This work is supported by National Natural Science Foundation of China (61162013, 61461003).