Multispectral, Hyperspectral, and Polarimetric Imaging TechnologyView this Special Issue
Research Article | Open Access
A New Methodology for Spectral-Spatial Classification of Hyperspectral Images
Recent developments in hyperspectral images have heightened the need for advanced classification methods. To reach this goal, this paper proposed an improved spectral-spatial method for hyperspectral image classification. The proposed method mainly consists of three steps. First, four band selection strategies are proposed to utilize the statistical region merging (SRM) method to segment the hyperspectral image. The segmentation map is subsequently integrated with the pixel-wise classification method to classify the hyperspectral image. Finally, the final classification result is obtained using the decision fusion rule. Validation tests are performed to evaluate the performance of the proposed approach, and the results indicate that the new proposed approach outperforms the state-of-the-art methods.
Hyperspectral images are generally composed of hundreds to thousands of spectral bands. This rich spectral information can effectively distinguish different objects and physical materials and thus cause broad applications in the mineral detection, environment monitoring, and precision agriculture. The classification technology is currently the predominate method for analyzing hyperspectral images and has received much attention. Over the past decades, numerous pixel-wise classification methods, which only use spectral information, have been proposed to classify remote sensing images. In reviewing the literature, pixel-wise classification methods mainly include maximum-likelihood , spectral angle classifier [1, 2], neural networks , genetic algorithms [3, 4], decision tree , and kernel-based methods [5–7]. Particularly, support vector machine (SVM) provides higher classification accuracy in most cases [5, 8, 9]. In this field, Lu and Weng  presented a good review and analysis of classification methods and techniques for remote sensing images.
Although pixel-wise classification methods have been researched for years, the spatial information has still not been sufficiently investigated. Generally, the spatial information is important for classification accuracy that can cause decrease of the classifier performance if neglected, particularly for very high spatial resolution satellite images. Previous studies show that pixel-wise methods will sometimes produce classification maps that look noisy (also known as “salt and pepper” effects) if the image spatial information is not used [10, 11]. An alternative to current pixel-wise methods is to combine spatial information with spectral information in classifier systems to form so-called spectral-spatial classification methods. This approach simultaneously considers the spectral and spatial information and it can generally achieve higher classification accuracy than pixel-wise classifiers [11–16]. Usually, there are two means to implement spectral-spatial classification. One of the spectral-spatial classification methods incorporates spatial information from its neighborhood, using a fixed size window. Mathematical morphology is used to construct morphological profiles (MPs) [11, 12] to define spatial information. Another approach is the integration of contextual/textural information in the kernel methods proposed by Camps-Valls and Bruzzone . However, such spectral-spatial methods have the problem of scale selection, especially when the remote sensing images consist of complicated structures , such as roads, buildings, and other man-made objects.
Another spectral-spatial scheme includes the postclassification spatial information using a segmentation map. An approach for classifying high spatial resolution urban satellite imagery is based on the different segmentation results of various scales . The partitional clustering method  and the watershed transformation  have also been used to measure spatial structures. The advantage of aforementioned methods is that it is easy to perform with low computational complexity. It should be pointed out that, however, the partitional clustering suffers from the following two limitations: the cluster numbers have to be set by users in advance; the clustering result depends on the initialization values and thus the clustering result is unrobust. These drawbacks raise the problem that it is unclear if the partitional clustering can improve the pixel-wise classification accuracy.
Based on the aforementioned analysis, this study presents a new spectral-spatial classification approach for hyperspectral images. The spatial information is obtained from the statistical region merging (SRM) , not the partitional clustering technique. The method is easy to implement and the experimental results are presented in the latter part of this paper showing not only that the method can improve classification accuracy but also that the results are robust.
The main contributions of this paper are two-folder:(1)proposing a strategy for band selection from the hyperspectral image;(2)proposing a method for spectral-spatial classification using SRM based on the designed band selection strategy.
The remainder of this paper is organized as follows. The spectral-spatial classification using grouping clustering is introduced in Section 2, while the proposed method is validated on two experimental images in Section 3. Section 4 includes the conclusions and future work.
The proposed spectral-spatial classification combines advances in SVM classification and SRM segmentation methods. The proposed method has three main steps, as summarized in Figure 1. Details of each step are introduced as follows.
2.1. Hyperspectral Image Segmentation
To segment the hyperspectral image, statistical region merging (SRM)  was selected to achieve the purpose. The advantage of this algorithm is that it can segment an image into regions in a fast and robust manner. In the SRM, let the observed image be denoted as which contains pixels. The image contains color channel values belonging to the set (where ). Let the perfect scene of an observed image be denoted as . The observed color channel is sampled from a family of , taken on values from and distributions at each pixel of . controls the number of regions: the higher is, the greater the number of regions generated is. Two components are essential in defining the SRM algorithm: the merging predicate and the merging order. The merging predicate is defined aswhere and and represent a fixed couple of regions of and ; merge and if . The merging order is to choose a function to sort pixel pairs in . One choice of iswhere and are the pixel channel values.
The original SRM algorithm is used for segmenting color images that contain only three spectral channels and thus cannot directly segment hyperspectral images. Although a minimum heterogeneity rule based SRM method  is proposed for multispectral satellite image segmentation, this study designed a strategy from different viewpoint that fully utilizes the rich spectral information of hyperspectral images. To this end, an improved SRM algorithm is presented in this study. The central idea of the proposed approach is to select three spectral bands from hyperspectral images, which is relying on four different band selection strategies. The advantage of band selection is that it can be repeated and thus will generate multiple information sources and redundant information, which complement each other and improve the robustness. The band selection strategies include a static method which selects three predetermined bands and three dynamic methods which select three bands based on the distribution. Details of the band selection process are described as follows.
(1) The First Three PCs. Firstly, the principle component analysis (PCA) transformation is performed on the original hyperspectral image to select the first three principle components (PCs) as the input of SRM. Although this study utilizes PCA to perform band selection, there are also numerous band selection technologies that can be used, such as linear discrimination analysis (LDA)  and nonparametric weighted feature extraction (NWFE) . The interested reader is referred to . It is obvious that this band selection method is static. By contrast, the following three methods are dynamic methods that select three bands based on the distributions, including uniform distribution, distribution, and distribution.
(2) Uniform Distribution. The weights of hyperspectral bands are supposed to be equivalent and three spectral bands are randomly selected as the input of SRM.
(3) Distribution. The distribution is based on the assumption that the hyperspectral bands weights are unequivalent  and can be measured by LDA. The weight of each band is computed by the following equation:where denotes the discrimination power of the band, and and represent within-class scatter matrix and between-class scatter matrix, respectively. Here, and are defined aswhere is number of classes, is number of samples, represents spectral value, is th class mean, and is the overall mean of spectral values.
(4) Distribution. Unlike the distribution, the distribution uses entropy to compute the band weight. Entropy  is a statistical measure of randomness that can be used to characterize the texture of the input image. The weight of each band is determined by where is the entropy of spectral band.
Once the distributions are obtained, band selection can be performed using the pseudorandom number generation theory [24, 26]. After the band selection process, the hyperspectral image is segmented using Algorithm 1.
2.2. Spectral-Spatial Classification
The spectral-spatial classification is performed to postprocess pixel-wise SVM classification result after segmentation results obtained by SRM. In this study, the scheme [14, 15] (see Figure 2), which combines advances of segmentation and classification results, is selected to implement spectral-spatial classification. Particularly, the majority voting algorithm is selected as the decision fusion rule, as it is easy to implement. The SRM segmentation and spectral-spatial classification will be repeated 200 times, resulting in 200 classification results. Finally, these 200 classification results will be fused using the majority voting algorithm to produce the final classification result.
In order to evaluate the performance of the proposed spectral-spatial classification approach, experiments on two hyperspectral images were carried out. The first experiment used a ROSIS image whereas an AVIRIS image was used in the second experiment. In this study, MATLAB with R2010b version was used as the coding environment on a PC that has Intel Core2Quad processor with 2.83-GHz clock speed.
3.1. First Experiment
The University of Pavia image is of an urban area recorded by the ROSIS-03 optical sensor, with an image size of 610 × 340 pixels. The image has a spatial resolution of 1.3 m per pixel and the number of spectral bands is 115, which ranges from 0.43 to 0.86 μm. It should be noted that the 12 noisiest channels have been removed in the preprocessing step. There are 9 classes in the first experiment, denoted as C1, C2, C3, C4, C5, C6, C7, C8, and C9. The training and testing data sets used in this experiment were provided by Professor Paolo Gamba from University of Pavia. A false color image (bands 50, 27, and 17) and the ground truth data are shown in Figures 3(a) and 3(b), respectively.
The supervised classification was firstly created by the multiclass SVM and without feature selection. Table 1 shows the training and testing sample numbers for each class. This study implemented the SVM classification by LIBSVM library . The Gaussian radial basis function (RBF) kernel was used in this experiment and the optimal parameters and were determined by fivefold cross validation, resulting in , . The pixel-wise SVM classification result is shown in Figure 5(a). After pixel-wise SVM classification, SRM algorithm was then applied to segment the hyperspectral image. Three bands were selected based on distribution as the input of SRM. A Matlab toolbox  was used to perform the SRM algorithm. The value of SRM was defined as 1024 in this study. Figure 4 shows the SRM results using different band selection strategies.
The spectral-spatial classification was then performed after the segmentation maps were obtained. Figures 5(b)–5(e) show the spectral-spatial classification results using different band selection strategies. From the visual analysis, it can be seen that the pixel-wise SVM result looks like more “noisy” than that of spectral-spatial classification methods. By contrast, the latter provide more homogeneous regions than pixel-wise SVM. In order to quantitatively evaluate the performance of the proposed approach, two measures, including overall accuracy (OA, the number of well-classified samples divided by the number of test samples) and kappa coefficient (, the percentage of agreement corrected by the amount of agreement that would be expected by chance alone), are used in this study.
Table 1 reports the classification accuracy for each class. From Table 1, it can be seen that spectral-spatial classification methods can improve the classification accuracy of the pixel-wise SVM except C2 class. The reason for this phenomenon is that C2 class is the feature with small area, whose spatial structure is likely damaged in the process of image segmentation. Despite this drawback, the spectral-spatial classification methods still show eminently satisfactory results. The comparison of the SVM classification and the spectral-spatial classification is given in Table 2. As is seen from Table 2, compared to the pixel-wise SVM, OA and improve about 6%~9% and 7%~11%, respectively. This table clearly shows that the proposed approach has higher classification accuracy with respect to the pixel-wise SVM (i.e., the approach proposed outperforms the pixel-wise SVM). The improvement in the volume of classification accuracy coincides with the assumption that integrating multiple information sources (i.e., spectral and spatial information) can reduce classification errors. Meanwhile, Table 2 also shows that both three dynamic band selection methods achieve higher classification accuracy than static methods. This is due to the fact that dynamic band selection methods can provide more rich spatial information than static methods. In particular, the band selection based on achieves the highest classification accuracy in this experiment.
To assess the impact of the presented algorithm on the results of hyperspectral image classification, a comparison was carried out among SVM plus majority vote method (SVMMV) , dynamic subspace method with random band selection method (DSM) , dynamic subspace method with LDA distribution method (DSMw2) , and the proposed approach. As shown in Table 3, it depicts the behavior of OA and among these four methods. The OA yielded by SVMMV, DSM, DSMw2, and the proposed approach was equal to 85.42%, 87.51%, 88.76%, and 89.49%, respectively, while values were equal to 81.30%, 85.03%, 86.22%, and 86.59%, respectively. The proposed approach outperforms SVMMV, DSM, and DSMw2 obviously, which indicates that the proposed method is more suitable for hyperspectral image classification than the other three methods.
3.2. Second Experiment
The Indiana Indian Pines hyperspectral image captured by the AVIRIS sensor on June 12, 1992, was used in the second experiment. The data and corresponding true ground data, as shown in Figure 6, are provided by Professor David A. Landgrebe from Purdue University. The AVIRIS data is composed of 220 spectral bands with a spatial resolution of 20 m per pixel . In this experiment, a subsection of the original Indian Pines with a size of 145 × 145 pixels was used. Twenty bands with water absorption were discarded, resulting in 200 bands. There are 16 different classes in the original Indiana image. Four of these 16 classes were discarded due to their sample size, resulting in 12 classes for this experiment labelled as C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, and C12. Detailed information about the 12 classes is given in Table 4, with a number of samples for each class in the available reference data.
First, SRM based on four band selection strategies were applied to segment the hyperspectral image, as shown in Figure 7. Here, the parameter of SRM was given by 8192. Next, 30% samples for each class were randomly chosen from the ground truth data as training samples. Based on these training datasets, the optimal parameters and were determined as 512 and 0.0078, respectively. Figure 8(a) shows the classification result of the pixel-wise SVM. After that, the SRM segmentation results, based on four band selection schemes, were used to refine the pixel-wise SVM classification result, producing results in Figures 8(b)–8(e). As can be seen from Figure 8, SVM classifier suffers from “salt and pepper” effects that lead to the decrease of classification performance. By contrast, spectral-spatial classifiers produce more homogeneous areas and hence tackle this limitation to a certain extent, which in turn improves the classification accuracy.
For the purpose of quantitative comprise, Table 4 summarizes the classification accuracy for each class of different methods. As hinted by Table 4, spectral-spatial classification methods can improve the classification accuracies of most classes produced by pixel-wise SVM. Table 5 comprises the pixel-wise SVM and spectral-spatial classification method based on different band selection strategies. As can be seen from Table 4, compared to pixel-wise SVM, the proposed approach based on four band selection strategies improves OA values by 5.09%, 9.95%, 9.71%, and 9.73%, respectively, while values are improved by 5.83%, 11.42%, 11.15%, and 11.18%, respectively. Similar to the first experimental results, both three dynamic band selection methods achieve higher accuracy than static band selection method, which again verifies the superiority of the proposed approach. Meanwhile, as opposed to the first case, band selection method based on uniform distribution produces the best performance in the second case. This indicates that uniform distribution is more suitable for urban areas where the spatial structural is complicated, while entropy distribution is a more proper way for plain areas without many image details.
In the second experiment, to assess the suitability of the proposed approach for the classification of hyperspectral images, a comparison analysis was carried out on three other methods (i.e., SVMMV, DSM, and DSMw2). As reported in Table 6, the proposed method resulted in the highest OA and . The quantitative analysis confirms the suitability of the proposed approach on the classification of hyperspectral images.
In the first experiment, band selection using gives the highest accuracy (see Table 2). From Table 1, it can be seen that the accuracy of classes C2, C4, and C9 of the spectral-spatial classification based on distribution decreases compared to the pixel-wise SVM classification. From Figure 9, it can be seen that the distribution method shows a good classification only in area (I) compared to the and distributions. In (II)–(VI) areas, the distribution method misclassified pixels more than the and distribution methods did.
In the second experiment, the classification method based on achieves the highest accuracy (see Table 4). Figure 10 shows the visual comparison of the spectral-spatial classification results based on different band selection strategies. As can be seen from Figure 10, both dynamic methods performed well on area (I) compared to the band selection method based on the first three PCs. Figure 10 also indicates that dynamic methods achieve similar classification results which has also been confirmed by Table 4. Hence, for images with large spatial structures, the band selection strategy has a low influence on the spectral-spatial classification method when the image has a large spatial structure.
An advanced spectral-spatial classification method for classification of hyperspectral images, which combines advances of region-based segmentation and image fusion, has been proposed in this study. The proposed approach has been achieved by (a) integrating pixel-wise support vector machine (SVM) classification and statistical region merging (SRM) segmentation results; (b) multiclassification results fusion using majority voting. Four different band selection strategies have been studied to implement the SRM algorithm to segment the hyperspectral image. The proposed approach has two advantages: it does not need to set cluster numbers in advance; the segmentation does not depend on initial values. These advantages result in higher robustness than the partitional clustering technique and thus make the proposed approach ideal for advanced spectral-spatial classification of hyperspectral images. Furthermore, the proposed approach is easy and efficient to implement. Although the developed method was used to classify hyperspectral images in this study, this method can also be used to classify multispectral images.
In this study, the spatial information is derived from the region-based segmentation results, which suffers from two main drawbacks: it is difficult to control the scale and it destroys class boundaries. To tackle these limitations, further research will therefore be conducted to improve the segmentation results. Particularly, edge information will be studied to produce precise segmentation result of remote sensing images.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
The work presented in this paper is partly supported by Ministry of Science and Technology of China (Project nos. 2012BAJ15B04 and 2012AA12A305), the National Natural Science Foundation of China (41331175), and Ling Jun Ren Cai Project of National Administration of Surveying, Mapping and Geo-Information, China. The authors would like to thank Professor David A. Landgrebe from Purdue University for providing AVRIS datasets, Professor Paolo Gamba from University of Pavia for providing the ROSIS hyperspectral images, and Dr. Hua Zhang and Dr. Yi Liu from China University of Mining and Technology for their kind discussion and support on computer programming.
- H. Z. M. Shafri, A. Suhaili, and S. Mansor, “The performance of maximum likelihood, spectral angle mapper, neural network and decision tree classifiers in hyperspectral image analysis,” Journal of Computer Science, vol. 3, no. 6, pp. 419–423, 2007.
- F. A. Kruse, A. B. Lefkoff, J. W. Boardman et al., “The spectral image processing system (SIPS)-interactive visualization and analysis of imaging spectrometer data,” Remote Sensing of Environment, vol. 44, no. 2-3, pp. 145–163, 1993.
- B. C. K. Tso and P. M. Mather, “Classification of multisource remote sensing imagery using a genetic algorithm and Markov random fields,” IEEE Transactions on Geoscience and Remote Sensing, vol. 37, no. 3 I, pp. 1255–1260, 1999.
- U. Maulik and S. Bandyopadhyay, “Fuzzy partitioning using a real-coded variable-length genetic algorithm for pixel classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 41, no. 5, pp. 1075–1081, 2003.
- G. Camps-Valls and L. Bruzzone, “Kernel-based methods for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 43, no. 6, pp. 1351–1362, 2005.
- G. Camps-Valls, L. Gomez-Chova, J. Muñoz-Marí, J. Vila-Francés, and J. Calpe-Maravilla, “Composite kernels for hyperspectral image classification,” IEEE Geoscience and Remote Sensing Letters, vol. 3, no. 1, pp. 93–97, 2006.
- G. Camps-Valls, L. Gómez-Chova, J. Muñoz-Marí, J. L. Rojo-Álvarez, and M. Martínez-Ramón, “Kernel-based framework for multitemporal and multisource remote sensing data classification and change detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 6, pp. 1822–1835, 2008.
- M. Pal and P. M. Mather, “Support vector machines for classification in remote sensing,” International Journal of Remote Sensing, vol. 26, no. 5, pp. 1007–1011, 2005.
- F. Melgani and L. Bruzzone, “Classification of hyperspectral remote sensing images with support vector machines,” IEEE Transactions on Geoscience and Remote Sensing, vol. 42, no. 8, pp. 1778–1790, 2004.
- D. Lu and Q. Weng, “A survey of image classification methods and techniques for improving classification performance,” International Journal of Remote Sensing, vol. 28, no. 5, pp. 823–870, 2007.
- M. Fauvel, Spectral and spatial methods for the classification of urban remote sensing data [Ph.D. thesis], University of Iceland, 2007.
- M. Fauvel, J. A. Benediktsson, J. Chanussot, and J. R. Sveinsson, “Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles,” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 11, pp. 3804–3814, 2008.
- Y. Tarabalka, J. Chanussot, J. A. Benediktsson, J. Angulo, and M. Fauvel, “Segmentation and classification of hyperspectral data using watershed,” in Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, pp. III652–III655, Boston, Mass, USA, July 2008.
- Y. Tarabalka, J. A. Benediktsson, and J. Chanussot, “Spectral-spatial classification of hyperspectral imagery based on partitional clustering techniques,” IEEE Transactions on Geoscience and Remote Sensing, vol. 47, no. 8, pp. 2973–2987, 2009.
- Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, “Segmentation and classification of hyperspectral images using watershed transformation,” Pattern Recognition, vol. 43, no. 7, pp. 2367–2379, 2010.
- Y. Tarabalka and J. C. Tilton, “Spectral-spatial classification of hyperspectral images using hierarchical optimization,” in Proceedings of the 3rd Workshop on Hyperspectral Image and Signal Processing (WHISPERS '11), pp. 1–4, June 2011.
- G. Camps-Valls and L. Bruzzone, Kernel Methods for Remote Sensing Data Analysis, John Wiley & Sons, Hoboken, NJ, USA, 1st edition, 2009.
- E. Frauman and E. Wolff, Segmentation of Very High Spatial Resolution Satellite Images in Urban Areas for Segments-Based Classification, 2005, http://www.ecognition.com/sites/default/files/226_frauman_urs2005.pdf.
- R. Nock and F. Nielsen, “Statistical region merging,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 11, pp. 1452–1458, 2004.
- F. A. Kruse, A. B. Lefkoff, J. W. Boardman et al., “The spectral image processing system (SIPS)-interactive visualization and analysis of imaging spectrometer data,” Remote Sensing of Environment, vol. 44, no. 2-3, pp. 145–163, 1993.
- H. Li, H. Gu, Y. Han, and J. Yang, “An efficient multiscale SRMMHR (Statistical Region Merging and Minimum Heterogeneity Rule) segmentation method for high-resolution remote sensing imagery,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 2, no. 2, pp. 67–73, 2009.
- A. M. Martinez and A. C. Kak, “PCA versus LDA,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 228–233, 2001.
- B.-C. Kuo and D. A. Landgrebe, “Nonparametric weighted feature extraction for classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 42, no. 5, pp. 1096–1105, 2004.
- J.-M. Yang, B.-C. Kuo, P.-T. Yu, and C.-H. Chuang, “A dynamic subspace method for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 48, no. 7, pp. 2840–2853, 2010.
- R. C. Gonzalez, R. E. Woods, and S. L. Eddins, Digital Image Processing Using MATLAB, Prentice Hall, Upper Saddle River, NJ, USA, 2nd edition, 2003.
- L. Devroye, Non-Uniform Random Variate Generation, Springer, New York, NY, USA, 1st edition, 1986.
- C.-C. Chang and C.-J. Lin, LIBSVM: A Library for Support Vector Machines, 2001, http://www.csie.ntu.edu.tw/∼cjlin/libsvm/.
- S. Boltz, “Image Segmentation Using Statistical Region Merging,” 2014, http://www.mathworks.com/matlabcentral/fileexchange/25619-image-segmentation-using-statistical-region-merging.
- R. O. Green, M. L. Eastwood, C. M. Sarture et al., “Imaging spectroscopy and the airborne visible/infrared imaging spectrometer (AVIRIS),” Remote Sensing of Environment, vol. 65, no. 3, pp. 227–248, 1998.
Copyright © 2016 Zelang Miao and Wenzhong Shi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.