Abstract
Abnormal cell (ABC) is a markedly heterogeneous tissue area and can be categorized into three main types: benign hyperplasia (BH), carcinoma (Ca), and intraepithelial neoplasia (IN) or precursor cancerous lesion. In this study, the goal is to determine and characterize the continuum of colorectal cancer by using a 3D-texture approach. ABC was segmented in preprocessing step using an active contour segmentation technique. Cell types were analyzed based on textural features extracted from the gray level cooccurrence matrices (GLCMs). Significant texture features were selected using an analysis of variance (ANOVA) of ABC with a value cutoff of . Features selected were reduced with a principal component analysis (PCA), which accounted for 97% of the cumulative variance from significant features. The simulation results identified 158 significant features based on ANOVA from a total of 624 texture features extracted from GLCMs. Performance metrics of ABC discrimination based on significant texture features showed 92.59% classification accuracy, 100% sensitivity, and 94.44% specificity. These findings suggest that texture features extracted from GLCMs are sensitive enough to discriminate between the ABC types and offer the opportunity to predict cell characteristics of colorectal cancer.
1. Introduction
Colorectal cancer (CRC) represents one of the most frequent cancers affecting people [1]. It is characterized by abnormal and uncontrolled cellular proliferation [2]. Surgical resection of the primary tumor with curative intent is possible in only 70% of patients. Unfortunately, up to 30% of CRC patients who undergo surgical resection of the primary tumor experience a subsequent relapse within 3 years and with a median time to death of 12 months [3, 4]. Colorectal cells are transformed by CRC into anomalous and heterogeneous shapes [5, 6]. In this context, heterogeneity is a pronounced feature of colorectal cancer that manifests as areas of high cell density. Attempts to quantify heterogeneity have been made using multiple feature functions such as Haralick features [6]. Another instance has used the link between the texture of hepatic tissue and its entropy and uniformity to predict survival using computer tomography images [7]. However, limited studies have used texture features to assess the continuum of CRC from benign to malignant cells.
Additionally, classical optical microscopy systems can detect ABC by applying advanced image processing techniques [8]. Early detection of ABC by shape or heterogeneity is of high interest in order to diagnose and start therapy early [6]. Hence, automating the process allows a faster and more precise reading of microscopic biopsies and may even allow classification of samples as BH, IN, or Ca [6, 9, 10]. In this context, numerous studies have considered developing automated reading procedures of such biopsies [5, 11–14]. The biopsies examined by these procedures can be prepared and preprocessed for automated reading using the optical microscopy system. Then, ABCs can be analyzed from their surrounding media using segmentation techniques [15]. In this context, the appropriate segmentation technique must be carefully established in order to process multispectral bioimages from microscopic system that provides high resolution gray scale images. Moreover, identification of ABC within an image should take into consideration some characteristic features that are representative of each ABC type [6]. Texture feature extraction from ABC can be a promising technique to characterize each ABC type. Then, discrimination between ABC types can be done by applying one of the classifier approaches such as the decision tree approach [16]. The analysis of the textures and structures of each ABC type permits a more accurate diagnosis of the malignant cells as they are structured in various patterns and textures.
In this work, we propose to analyze each of ABC type by extracting texture features from GLCM. Texture refers to the variability in tone within a region, or the spatial relationships among the gray levels of neighboring pixels. Three-dimensional (3D) texture analyses offer more information by using two phases and multioffset pixels to detect the variability of pixel pairs in 3D space [16, 17]. The statistical approach of image analysis based on the matrix of cooccurrence is commonly applied to optical and medical images to evaluate morphology [5, 11, 18, 19]. The texture features thus extracted from GLCM describe the texture and local variation in an image. For classification, we selected twelve principal features in order to identify ABC types, while discarding those that are either redundant or confusing, thereby improving the performance of the proposed feature based detection technique.
In summary, the purpose of this study is the derivation of quantitative texture multispectral image features from optical microscopy images that classify the continuum of CRC lesions. The novelty of this study was the first training on an automated continuum prediction of CRC. This will be foundation of radiomic maps that associate these texture features with various ABC types.
The remainder of this paper is organized as follows. Section 2 describes the texture feature extraction from 3D GLCM in detail with performance metrics. Sections 3 and 4 demonstrate experimental results and discussions. Finally, Section 5 concludes the paper.
2. Materials and Methods
We specifically analyzed 3D multispectral digital whole slide images (WSI) from 27 colorectal cancer patients. An example of spatial heterogeneity for each multispectral ABC type is seen in its histogram distribution. Clearly, there are certain characteristics and features from preliminary analysis that differentiate BH, IN, and Ca (Figure 1).
(a)
(b)
(c)
2.1. Sample Preparation and Data Acquisition
Whole tissue samples were taken from colonic glands with thickness value of 5 μm which is stained using Haematoxylin and Eosin (H & E) stains. Images were captured by a charge coupled device (CCD) camera integrated with a liquid crystal tunable filter (LCTF) in the optical microscopy system [20]. LCTF provides multispectral images of the tissue samples by changing the wavelength operation [21]. LCTF has a bandwidth of 5 nm, and its wavelength is controllable through the visible spectrum range of 400–720 nm. Multispectral images are produced through repeated image capture in various wavelengths subbands. Moreover, the impact of multispectral imaging has been shown that the classifier accuracy increases with the number of spectral bands [22]. Note that each image band is 8 bits coded and hence has 255 possible light intensity levels.
In this study, LCTF offered 16 multispectral bands using a wavelength range of 500–650 nm. Thus, from each original image, we obtained 16 images representing the wavelength range and a volume of multispectral data (Figure 1). Hence, texture extraction from each band of multispectral data enhances the lesion characterizing each abnormal cell type. Note that a colorectal pathologist views images at lower power to identify the abnormal cells which is represented by a low magnification (×40) of image samples.
2.2. Patients
After excluding samples with incomplete data, a set of 27 CRC patients were gathered for a preliminary study. We selected nine volumes of data from each ABC type, where a volume of data was structured in 16 multispectral images (Figure 1). Thus, images were filtered by an average filter (spatial filter) before further segmentation processing to minimize the effects of noise in images and other external factors. All the images were reconstructed to a 512 × 512 matrix where the volume size of 512 × 512 × 16 was taken into consideration in texture feature extraction from GLCM of ABC (Figure 2(a)).
(a)
(b)
(c)
2.3. Segmentation of Abnormal Cell
We employed active contour segmentation to accurately segment anomalous shape of cells. This technique is based on a dynamic curve that moves toward and detects the contour of the object by a number of iterative processes [23, 24]. This approach was successfully implemented to detect ABC types from similar kinds of multispectral bioimages. The computation time was improved by limiting the number of iterations, which was set automatically based on empirical calculations [6]. Computation time was further strengthened by resizing the images. For instance, an image of size 512 × 512 pixels was decreased to 64 × 64 pixels and active contour was applied to detect cells within the image. Active contour images were then resized to 512 × 512 pixels and placed on the original image (Figure 3). In fact, this technique resized the active contour and not the original image in order to enhance the computation time. Cell images in 16 multispectral images were then assessed by a board certified colorectal pathologist. A volume of a cell segmented in 16 multispectral images (2D images) was created to represent the variance details in the multispectral band (Figure 2(a)). To assess the active contour segmentation, ground truth of cell and cells segmented based on active contour were considered. In this context, evaluation of WSI segmentation considered two similarity metrics, namely, Jaccard similarity coefficient (JSC) and Dice similarity coefficient (DSC). Additionally, false positive rate (FPR) and false negative rate (FNR) were also computed. JSC and DSC measure the degree of the correspondence between ground truth cell images and segmented images.
(a) BH
(b) IN
(c) Ca
(d) BH
(e) IN
(f) Ca
JSC can be formulated according to the following:where and are the area of ground truth of cell and segmented cell, respectively.
JSC was employed to calculate the overall level of similarity between segmented cell and ground truth cell. DSC was also employed and can be expressed according to the following:Additionally, we employed FPR and FNR which were used to quantify over- and undersegmentation. Both FPR and FNR are calculated according to the following:Direct relation between JSC, FPR, and FNR is defined according to the following expression:The performance metrics of active contour technique were reported (Table 1). This volume of WSI was quantified by the texture feature extracted from GLCMs of each abnormal cell type.
2.4. GLCM Based Feature
One of the best techniques used to evaluate the relationships between image pixels is the texture feature extraction from GLCM. This technique was proposed by Haralick et al. in 1973 [19]. It is one of the most popular second-order statistical features which is based on GLCM computation and its texture features. Then, second-order statistics estimate properties of two or more pixel values occurring at specific locations relative to each other. For these reasons, we proposed to use GLCM based feature technique in this work.
2.4.1. Gray Level Cooccurrence Matrix (2D and 3D)
GLCM represents the probabilities of transition from a pixel with intensity “” to a pixel of intensity “” separated by a translation vector defined by direction “” and an offset “” (offset known as distance) [11, 16–19]. Given a two-dimensional (2D) image of size , the cooccurrence matrix can be defined as follows:where and specify the distance between the pixel of interest and its neighbor, along the -axis and the -axis of an image, respectively. GLCM is a square matrix of size , where is the number of gray levels in the image.
For 2D images, typical values used for “” equal and those for “” equal . The GLCMs corresponding to the additional directions do not add to the specification of the texture already caught by the 16 GLCMs associated with combinations of the aforementioned four offsets and four directions. This is because there is symmetry in GLCMs described by (; ; ; and ) (note that superscript “” denotes the transpose operation).
GLCM computations can be also applied to 3D images. In this case, the GLCM counts the number of pixel pairs that have intensities “” and “” for the spatial relationship specified by a translation vector , where , , and represent the number of pixel offsets along the -axis, -axis, and -axis of the 3D image. For volumetric data, two angles lead to 13 directions (Figure 2). Each segmented cell was histogram equalized to 32 levels, and then we employed the GLCM computation.
The foremost advantage of GLCMs applied to volumetric data is the ability to capture intensity relationships between the pixels in a 3D volume. Further, the number of GLCMs resulting from 3D operations is typically smaller than that corresponding to numerous 2D slices. For example, in a data cube with 10 separate 2D slices, there are a total of 80 GLCMs (8 GLCMs analogous to 2 offsets and 4 directions per slice). On the other hand, in a 3D operation, the total number of GLCMs is 26 (13 directions and 2 offsets). Supported by the benefit of GLCMs applied on volumetric data, we computed GLCMs of multispectral ABC and quantified these cooccurrence matrices by Haralick features.
2.4.2. Texture Quantification
Haralick proposed 14 texture features to be extracted from GLCMs, and the value of each extracted feature indicates the preliminary indicators of ABC in the texture image. Among the 14 texture features, we employed the 12 principal textural features: energy (), entropy (), correlation (), contrast (), homogeneity (), variance (), sum-mean (), inertia (), cluster shade (), cluster tendency (), maximum probability (), and inverse difference moment (). These features are defined by their functions as followsshows the scale of texture homogeneity. It is high when the GLCMs consist of few pixels of high amplitude and low when all the values of GLCMs are almost similar.measures the disorder or complexity of an image. The highest value of entropy is found when the values of are allocated quite uniformly throughout the matrix. measures the linear dependence of gray level values in the GLCM or describes the correlations between the rows and columns of GLCM. measures intensity contrast or the local variations present in an image to show the texture fineness. returns a value that measures the closeness of the elements distribution in GLCM to the GLCM diagonal. is expected to be large if the gray levels of the image are spread out greatly. measures the average of the gray levels. It can be high value if the sum of the gray level of the image is high. measures the inhomogeneous in image.measures the skewness (asymmetric) of the GLCM and is considered to gauge the perceptual concepts of uniformity. When the cluster shade is high, the image is asymmetric. measures the grouping of pixels that have similar gray level values. measures the dominant pair pixels in the GLCM. It can be high if the dominant pair pixel is high. measures the smoothness of the image. It can be high if the gray levels of the pixel pair are similar.
For the ABC detection problem, the aforementioned textural features are extracted from the 3D GLCMs conforming to the 13 directions and 4 types of offset. Therefore, the length of the resulting feature vector is 12 (functions) × 13 (directions) × 1 (distance or offset) = 156 features. To analyze the effect of texture feature based on GLCMs, we organized texture features into 5 groups (, , , , and ) reported in Table 2.
Moreover, we calculated the average of texture feature based on 3D GLCM within 13 directions and 4 offsets to evaluate the value of each one from ABC (Table 3). Additionally, we employed feature selection techniques on each texture feature group to demonstrate the effectiveness of texture analysis in a definite direction and offset, and the performance metrics were reported (Table 4).
2.5. Statistical Analysis
Textures quantified by twelve functions (based on those suggested by Haralick) can be found among the BH, IN, and Ca cell samples. -score normalization was employed on each of the feature vectors, which converted the features to zero mean and unit variance [25]. The mean and standard deviation () of the feature vector are calculated as follows:where is the original value, is the new value, and the mean and are the mean and standard deviation of the original data, respectively.
ANOVA was used to assess the statistical significance between texture features and ABC types [26]. This test was used to identify the significant texture feature where a value < 0.01 was deemed significant. An aggregate of 158 significant features were selected, which was further reduced using PCA. Five principal components (PCs) representing 97% of the variance among the 158 selected features were used in a decision tree classifier (Tables 3 and 4).
2.6. Classifier Setting and Performance Metrics
Classification of the ABC types based on texture features was performed using the significant features as input variables in a decision tree (DT) classifier [27]. The most important aspect of a decision tree induction strategy is the split criteria; it is a method of selecting an attribute that determines the distribution of training objects into subsets upon which subtrees are consequently built. In this study, a goodness criterion based on Gini index was used to determine how well various feature test conditions performed [28]. The reason to use the DT classifier is to find automatically the dominant features and provide the classifier metrics; however, we considered the naïve Bayes and nearest neighbors to evaluate the classifier performance metrics using the known class labels from ABC types. Due to limited data (27 patients), the classifier was validated using leave-one-out cross-validation [29]. We considered the following performance metrics of classification: accuracy, sensitivity, specificity, -score, and area under the curve (AUC), which were performed to test the reliability of the texture feature classifier. We used multiple metrics for better assessing the feasibility of abnormal cell type discrimination using texture feature based 3D GLCMs. Note that true positive (TP) and true negative (TN) are the number of positive and negative samples correctly classified; false positive (FP) and false negative (FN) are the number of positive and negative samples incorrectly classified [30]. Then, TP + FN is the total number of test samples of the considered class. For example, TP of BH cell type represents the BH samples correctly classified and FN represents the BH cell type incorrectly classified. Also, FP of BH cell type represents the IN and Ca samples classified as BH cell type, while TN of BH cell type represents the number of correctly classified IN and Ca samples and number of IN samples classified as Ca and the number of Ca samples classified as IN cell type.
Accuracy represents the correctly classified samples and can be expressed by the following:Sensitivity is a measure of the capability of a classifier to recognize the positive class patterns. It can be expressed according to the following:Specificity is a measure of the capability of a classifier to recognize the negative class patterns. It can be expressed by the following: -score is a weighted average of precision and recall and can be calculated using the following:
3. Experimental Results
ABC digital images were segmented using the active contour segmentation technique. Figure 3 shows ABC types segmented using several steps. The process of cell detection from multispectral images may appear to be a difficult task as bioimages contained some areas that have a similar range of gray shades and irregular shapes. Morphology operators were necessary to select the required cells from images by a board certified colorectal pathologist because there were multiple cell types within images.
Snake (active contour) techniques showed that ABC types were correctly detected and located (Figure 3). JSC shows a similarity range of 75.92–81.56% with the best performance achieved with Ca cell type. Meanwhile, DSC shows a similarity range of 86.31–88.21% with the best performance achieved with Ca cell type. Moreover, FPR shows a range of 05.03–07.61% with the best performance achieved with IN cell type, while we observed that FNR provided a range value of 16.11–20.26% with the best performance achieved with Ca cell type (Table 1). These metrics confirmed the feasibility of active contour segmentation method to determine the abnormal cell types and specifically the Ca cell type (Table 1).
Figure 4 shows an example case of GLCMs for corresponding ABC types in Figure 3. GLCM images showed the most pronounced texture associated with Ca cells among the three ABC types. These texture values represent a high number of pixel pairs in the original image of Ca cells, followed by IN and BH cells, respectively. Additionally, BH images had a homogenous texture that was more homogenous than IN and Ca; its corresponding GLCM showed that most BH textures were depicted in the diagonal of the GLCM image. Notably, when the texture GLCM image has more fitted data around the diagonal, the original image is less homogenous. The average of texture functions based on whole offsets and directions showed the differences between the ABC groups, which were demonstrated in each of the 12 texture features extracted from GLCMs (as shown in Table 3). For instance, energy () exhibited a higher value (0.014) in BH followed by IN and Ca cells, respectively. This is demonstrated by the high homogeneity of BH cells. Entropy () showed close values between the ABC groups with a maximum value of 2.53 for Ca cells. This is reflected by the disorder of texture in Ca cells, but the close values of ABC types can be a weak classifier feature. Thus, correlation () showed a maximum value of 0.016 for IN cells; this is reflected by the higher dependence of gray level in IN GLCM images. Compared to all texture features of ABC types, the maximum value of features (, , , , , and ), ( and ), and (, , , and ) was found for BH, IN, and Ca type, respectively. With the exception of feature (), significant differences between the ABC types were noted in all the texture features examined ( value < 0.01).
(a) BH
(b) IN
(c) Ca
An ANOVA showed that there were 35 significant classifier features among the 156 texture features for , , and , while provided 53 significant features for discrimination between ABCs. The total number of significant features was 158 () by simple concatenation of group features (Table 4).
Performance metrics range of ABC discrimination based on specific offset and direction showed accuracy, sensitivity, and specificity ranges of 55.55–77.77%, 55.55–88.88%, and 66.66–94.44%, respectively.
Maximum values of the performance metrics range were achieved by using group which represented a four-pixel offset and 13 directions of GLCMs (Table 4). Moreover, classifier accuracy for each ABC feature exhibited a range of 55.55–88.88%, 44.44–88.88%, and 55.55–66.66% for BH, IN, and Ca, respectively (Table 5).
Furthermore, five PCs features showed the highest value of 92.59% accuracy, 100% sensitivity, and 94.44% specificity (the last row in Table 4).
The highest classifier accuracy obtained for IN was 88.88% using and features (Table 5). However, BH and Ca features showed the highest values of 100% using five PCs features (Table 5).
The analysis of the ROC curve obtained for each ABC discrimination showed ranges of AUC values of 76.99–90.30% for BH versus IN, 97.97–99.90% for BH versus Ca, and 99.18–99.73% for IN versus Ca, respectively. However, the best AUC values for ABC discrimination were achieved using 5 PCs features (Figure 5).
(a) Total selected features
(b) PCs features
Moreover, using five PCs features, comparative study of the confusion matrix for abnormal cell type discrimination based on decision tree (DT), naïve Bayes (NB), and nearest neighbors (NN) classifier [26–28] showed that the nine BH and nine Ca samples are correctly classified based on DT and NB classifier, respectively. However, eight IN samples are correctly classified based on NB classifier technique (Table 6). -score showed the highest BH, IN, and Ca metric with 94.73, 94.11, and 100%, respectively, using NB classifier. This demonstrates that the best classifier technique for discriminating BH from IN and Ca is NB (Table 7).
4. Discussion
In this study, we have shown the role of texture feature extraction from GLCMs to discriminate BH, IN, and Ca. We demonstrated the use of quantitative image texture features and reported significant features with performance metrics for ABC discrimination. Additionally, we showed the power of texture quantification from GLCMs using 12 functions to indirectly associate image features with ABC types. Texture features extracted from GLCMs using four pixels offsets and 13 directions () showed a higher accuracy to discriminate between types of ABC than other features groups. This proves that the GLCMs of fourth pixel neighbors in 13 directions can offer the best automated ABC classification (Table 4). Similarly, classifier features of BH showed effectiveness to identify BH cells by texture features extracted from GLCMs using 4 pixels offsets and 13 directions ( in Table 5). Abnormal IN cell presented the best classification using and , and Ca shows the best classification using , , and . However, without using PCs features, a lack of predictive accuracy of Ca demonstrated that its texture may resemble the texture of IN, which represents the complexity of malignant diagnostic (Table 5).
According to the experiments in which different groups of texture feature were applied to the ABC discrimination process, the results showed the efficiency of PC features derived from significant texture features extracted using GLCMs for histopathology colorectal cancer image analysis (last row of Tables 4 and 5). This potential of PC features demonstrated the highest AUC values for discrimination of BH versus IN, BH versus Ca, and IN versus Ca (Figure 5). Figure 6 showed a heat map correlation between the ABC features where the highest correlation represents the resemblance between the texture features. We observed that some BH and IN features (red rectangular shape) have a high correlation value. This represented the lack of performance metrics.
This study demonstrates that texture feature extraction can be a map for ABC identification by using the techniques in image processing such as significant feature and feature selection. Previously, differentiation of human colon cancer cells was demonstrated using gene expression of B-tubulin isotypes [31]. More recently, multilabel classification of colon cancer using histopathological images was performed using several types of features. It was concluded that combined features can offer good performance for multilabel colon cancer prediction, with a precision of 73.7% [32]. Moreover, another study has proven that colon cancer prognosis can be identified by using distinct molecular subtypes and serrated precursor lesions [33]. Thus, the effort to analyze the continuum of colorectal cancer is still incomplete.
To date, few studies have directly addressed the discrimination between types of ABC for the diagnosis of colon cancer. Most have focused on their heterogeneity; several studies have suggested that increasing heterogeneity is associated with malignancy [34]. Additionally, it has been proposed that greater biologic heterogeneity may be associated with oxidative stress and genomic instability [35]. Also, a study based on hepatic texture in patients with CRC found a more heterogeneous liver texture at coarse scale (textures extracted based on Laplacian of Gaussian filter) is related to the presence of occult malignancy [36]. Moreover, in this work, it was proven that higher value of entropy function is associated with carcinoma which represents a higher heterogeneity between ABC types.
This study offers a simple approach based on texture feature analysis to evaluate the continuum of colorectal cancer from benign to malignant by using three abnormal cells. These three cell types represent the transformation from benign to malignant cancer. In this context, the results showed that radiomic texture feature is significant and provide good classifier metrics and also highlight the potential of radiomic texture feature extraction for enhanced prediction of ABC from colorectal tissues. This should trigger further research of image-based quantitative texture features in colorectal cancer. Given the reality that colorectal cancer is highly heterogeneous between patients, texture feature analysis is a more desirable approach to provide clear categorization of ABC type than the established methods.
Our study had limitations; the most important of these was the limited subjects (). Also the computation time of the segmentation, 3D GLCM, and texture feature extraction was around 15 minutes for each case. However, given the reality of the ABC, texture feature based 3D GLCM is a more preferable approach to categorize ABC type than the recognized models.
5. Conclusion
In this paper, a new method based on multitexture features for abnormal cell classification of colorectal cancer is proposed. Real data of colorectal cancer was used to validate the discrimination between ABC features. ABC was segmented by active contour technique and then texture feature extracted from GLCMs. Significant texture features were selected based on ANOVA test. The best results were obtained when combining all features together and PCA was applied to get five PC features with accuracy of 92.59% to discriminate between ABC types. This result is promising to make a bride between image features and colorectal pathology which would lead to efficient medical diagnosis and treatment.
Consent
Any necessary approvals, authorizations, and informed consent documents were obtained.
Disclosure
The materials are in compliance with all applicable laws, regulations, and policies for the protection of medical data.
Competing Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors would like to acknowledge the service Anapat of the CHU Hospital of the Nancy-Brabois and the Architecture of Embedded Systems and Smart Sensors (ASEC) team. This work was supported by the University of Lorraine.