Multi-Layer Perceptron Classifier with the Proposed Combined Feature Vector of 3D CNN Features and Lung Radiomics Features for COPD Stage Classification

Yang, Yingjian; Zeng, Nanrong; Chen, Ziran; Li, Wei; Guo, Yingwei; Wang, Shicong; Duan, Wenxin; Liu, Yang; Chen, Rongchang; Kang, Yan

doi:https://doi.org/10.1155/2023/3715603

Journal of Healthcare Engineering

On this page

Abstract Introduction Materials and Methods Results Discussion Conclusions Data Availability Disclosure Conflicts of Interest Authors’ Contributions Acknowledgments References Copyright Related Articles

Special Issue

Artificial Intelligence in Disease Diagnosis

View this Special Issue

Research Article | Open Access

Volume 2023 | Article ID 3715603 | https://doi.org/10.1155/2023/3715603

Multi-Layer Perceptron Classifier with the Proposed Combined Feature Vector of 3D CNN Features and Lung Radiomics Features for COPD Stage Classification

Yingjian Yang,^1,2Nanrong Zeng,^2,3Ziran Chen,^1,2Wei Li,²Yingwei Guo,^1,2Shicong Wang,^2,3Wenxin Duan,^2,3Yang Liu,^2,3Rongchang Chen ,^4,5,6and Yan Kang^1,2,3,7

Academic Editor: Weihua Yang

Received16 May 2022

Revised02 Aug 2022

Accepted25 Apr 2023

Published03 Nov 2023

Abstract

Computed tomography (CT) has been regarded as the most effective modality for characterizing and quantifying chronic obstructive pulmonary disease (COPD). Therefore, chest CT images should provide more information for COPD diagnosis, such as COPD stage classification. This paper proposes a features combination strategy by concatenating three-dimension (3D) CNN features and lung radiomics features for COPD stage classification based on the multi-layer perceptron (MLP) classifier. First, 465 sets of chest HRCT images are automatically segmented by a trained ResU-Net, obtaining the lung images with the Hounsfield unit. Second, the 3D CNN features are extracted from the lung region images based on a truncated transfer learning strategy. Then, the lung radiomics features are extracted from the lung region images by PyRadiomics. Third, the MLP classifier with the best classification performance is determined by the 3D CNN features and the lung radiomics features. Finally, the proposed combined feature vector is used to improve the MLP classifier’s performance. The results show that compared with CNN models and other ML classifiers, the MLP classifier with the best classification performance is determined. The MLP classifier with the proposed combined feature vector has achieved accuracy, mean precision, mean recall, mean F1-score, and AUC of 0.879, 0.879, 0.879, 0.875, and 0.971, respectively. Compared to the MLP classifier with the 3D CNN features selected by Lasso, our method based on the MLP classifier has improved the classification performance by 5.8% (accuracy), 5.3% (mean precision), 5.8% (mean recall), 5.4% (mean F1-score), and 2.5% (AUC). Compared to the MLP classifier with lung radiomics features selected by Lasso, our method based on the MLP classifier has improved the classification performance by 5.0% (accuracy), 5.1% (mean precision), 5.0% (mean recall), 5.1% (mean F1-score), and 2.1% (AUC). Therefore, it is concluded that our method is effective in improving the classification performance for COPD stage classification.

1. Introduction

Chronic obstructive pulmonary disease (COPD) is a common and non-infectious lung disease characterized by persistent airflow limitation [1–3]. Because of this characterization, the COPD stage is diagnosed from stage 0 to IV according to Global Initiative for Chronic Obstructive Lung Disease (GOLD) criteria accepted by the American Thoracic Society and the European Respiratory Society [4]. GOLD is examined by the pulmonary function test (PFT) and diagnosed by the forced expiratory volume in 1 second/forced vital capacity (FEV1/FVC) and FEV1% predicted [1, 2]. PFT can explain the impact on symptoms and life quality of COPD patients [5, 6], but it cannot reflect the change of the lung tissue in COPD patients with the COPD stage evolution. PFT changes from normal to abnormal occur when lung tissue is destroyed to a certain extent. Therefore, the PFT makes it challenging to identify the etiology of COPD.

Compared with the GOLD criteria and other imaging equipment, computed tomography (CT) has been regarded as the most effective modality for characterizing and quantifying COPD [7]. Compared with PFT, chest CT images can indicate that the patients have suffered from mild lobular central emphysema and decreased exercise tolerance in smokers without airflow limitation [8]. In addition, the chest CT images are also used to quantitatively analyze the bronchial, airway disease, emphysema, and vascular for COPD patients [7]. However, automatic multi-classification based on convolutional neural networks (CNNs) using chest CT images remains a challenging task for the COPD stage. One main reason is that the number of medical images is limited compared to natural images. In particular, few people seek medical treatment in the early stage of COPD and undergo CT scans simultaneously. Transfer learning [9] may solve the above problems. Since radiomics was proposed to mine more information from medical images using advanced feature analysis in 2007 [10], it has been widely used to analyze lung disease imaging [11–15]. However, radiomics features are extracted from medical images by specific calculation equations, preset types of images, and preset classes, limiting the forms of radiomics features. Some deep features from CNN (CNN features) are also needed to improve the classifier’s performance in multi-classification. CNN features extracted from medical images will make up for the limitations of radiomics features.

Radiomics features in COPD develop slower than those in other lung diseases, such as lung cancer and pulmonary nodules. Until 2020, reference [16] points out that radiomics features in COPD have not been extensively investigated yet. Nevertheless, there are potential applications of radiomics features in COPD for the diagnosis, treatment, and follow-up of COPD and future directions [16]. A critical reason limiting the development of radiomics features in COPD is its diffuse distribution in the lung. At the same time, radiomics features need to be extracted from the region of interest (ROI) of the chest CT images. However, the diffuse distribution of COPD makes it difficult to determine ROI. COPD results from the joint action of the peripheral airway, pulmonary parenchyma, and pulmonary vessels [17–19]. Thus, the peripheral airway, pulmonary parenchyma, and pulmonary vessels as ROI to extracting lung radiomics features are reasonable for COPD stage classification.

Currently, radiomics features also have been used in COPD for survival prediction [20, 21], COPD presence prediction [22], COPD exacerbations [23], COPD early decision [4], and analysis of COPD and resting heart rate [3]. However, as mentioned above, lung radiomics features have not been applied in the COPD stage classification. On the other hand, radiomics based on machine learning (ML) and chest CT images based on CNN have been widely and respectively used in COPD and its evaluation. However, the advantages of radiomics based on machine learning and medical images based on CNN need to be further integrated to improve the performance of COPD stage classification. Therefore, this paper proposes a feature combination strategy by concatenating three-dimension (3D) CNN features and lung radiomics features for COPD stage classification based on the multi-layer perceptron (MLP) classifier. Our contributions in this paper are briefly described as follows. (1) MLP classifier with the best classification performances is determined in the ML classifier for 3D CNN features or lung radiomics features. (2) Truncated transfer learning is proposed from the excellent segmentation model for generating nonlinear 3D CNN features. (3) The proposed feature combination strategy by concatenating 3D CNN features and lung radiomics features effectively improves the MLP classifier’s performance.

2. Materials and Methods

2.1. Materials

The participants are enrolled by the national clinical research center of respiratory diseases, China, from May 25, 2009, to January 11, 2011. Finally, 465 Chinese subjects participated in the study after being strictly selected by the inclusion and exclusion criteria [24]. The 465 subjects underwent chest HRCT scans at the full inspiration state. In addition, the 465 subjects also underwent the PFT, and the COPD stage of each subject is diagnosed by PFT in Global Initiative for Chronic Obstructive Lung Disease (GOLD) criteria 2008 accepted by the American Thoracic Society and the European Respiratory Society.

Figure 1 shows the COPD stage distribution of the subjects in this study. There are 129, 108, 121, and 107 subjects in each COPD stage (GOLD 0, GOLD I, GOLD II, GOLD III, and GOLD IV). This study was approved by the ethics committee of the national clinical research center for respiratory diseases in China. In addition, all 465 subjects have been provided written informed consent to the first affiliated hospital of Guangzhou medical university before chest HRCT scans and PFT. Refer to our previous study [4] for a more detailed description of the materials.

2.2. Methods

Figure 2 shows the proposed method in this study. The main idea of the proposed method proposed in this paper is to combine 3D CNN features and lung radiomics features for COPD stage classification. When generating the 3D CNN features, we adopt a truncated transfer learning strategy that only intercepts the encoder backbone of the pretrained Med3d [25].

(a)

(b)

(c)

(d)

2.2.1. Lung Radiomics Features Extraction

First, 465 sets of chest HRCT images are automatically segmented by a trained ResU-Net [26], obtaining 465 sets of lung images with the Hounsfield unit (Hu) [27]. The lung images include the peripheral airway, pulmonary parenchyma, and pulmonary vessels. The architecture of the ResU-Net has been described in detail in our previous study [28]. Then, lung radiomics features of 465 subjects are extracted from the lung images by PyRadiomics [29]. Refer to our previous study [4] for a more detailed description of the lung radiomics feature extraction.

2.2.2. 3D CNN Feature Extraction

A truncated transfer learning strategy is proposed to extract the 3D CNN features based on the pretrained Med3d [25]. Med3d, a heterogeneous 3D network, is used to extract general medical 3D features by building a 3DSeg-8 dataset with diverse modalities, target organs, and pathologies. Thus, we only transfer the encoder backbone of the pretrained Med3d (3D ResNet10) for generating the 3D CNN features, as shown in Figure 2(a).

Figure 2(b) shows that the 465 sets of lung images with Hu are input to the transferring encoder backbone, generating CNN feature vectors in detail. First, the lung images (512 × 512 × N) are cropped into the size 280 × 400 × N′, retaining the lung region. The non-lung images are also deleted, so N changes into N′ (N′ < N). Second, the cropped lung images are preprocessed by the method in reference [25], normalizing the lung region and generating random values outside the lung region in accordance with Gaussian distribution. Equation (1) shows the mathematical form of normalization:where x is the value of the lung region, is the mean value of the lung region, and σ is the mean square deviation of the lung region.

Third, the CNN feature maps (512 × 35 × 50 × 75) are generated by the cropped and preprocessed lung images (1 × 280 × 400 × N′) and the pretrained Med3d. Last, higher-order CNN feature maps (512 × 3 × 3 × 3) need to be extracted from the CNN feature maps (512 × 35 × 50 × 75) by 3D average pooling. Then, the higher-order CNN feature maps (512 × 3 × 3 × 3) are flattened into the CNN feature vector. Finally, each CNN feature vector (per subject) includes 13824 3D CNN features (512 × 3 × 3 × 3 = 13824).

2.2.3. Combined Feature Vector for COPD Classification

Figure 2(c) shows that the combined feature vector is generated by concatenating the CNN feature vector and the radiomics feature vector. First, the CNN feature vector (13824) and the radiomics feature vector (1316) are selected by the least absolute shrinkage and selection operator (Lasso) [30], respectively. After Lasso, the number of the selected CNN feature vector and the selected radiomics feature vector is 60 and 106, respectively. A standard python package LassoCV, with tenfold cross-validation, is performed in this paper. Equation (2) shows the mathematical form of Lasso [4]:where matrix A denotes the selected lung radiomics feature. denotes the lung radiomics features (the independent variable). y_i denotes the COPD stage (the independent variable). λ denotes the penalty parameter (λ ≥ 0). β_j denotes the regression coefficient, i ∈ [1, n], and j ∈ [0, p].

Second, the combined feature vector is generated by concatenating the selected CNN feature vector and the selected radiomics feature vector. Finally, the combined feature vector is the size 1 × 166 per subject. Figure 2(d) shows that MLP [31, 32] with the combined feature vector is used to classify the COPD stage in this paper.

2.2.4. Experiments and Evaluation Metrics

Our proposed method uses the combined feature vector of 3D CNN features and lung radiomics features for COPD stage classification based on the MLP classifier. Our experiment includes five experiments in this section to verify the effectiveness of our proposed method.

Figure 3 shows the experimental design in this paper. End-to-end CNN models based on parenchyma images are used for COPD stage classification in experiments 1-2. Specifically, two classic CNN models, DenseNet and GoogLeNet, based on parenchyma images, are adopted to compare the classification performance of the six different ML classifiers. The classification performance of DenseNet and GoogleNet has been evaluated by our previous study [33], which achieved the best classification performance for image classification. Furthermore, compared with experiment 1, multiple-instance learning (MIL) [34], a form of weakly supervised learning, is applied in experiment 2. Meanwhile, different ML classifiers based on different feature vectors are also used for COPD stage classification in experiments 3–5.

Specifically, the training parameters of 2D DenseNet and 3D DenseNet are set: 20/2 (batch size (2D/3D)), 512 × 512/512 × 512 × 20 (input size (2D/3D)), 50/50 (epoch (2D/3D)), and 0.5/0.2 (drop rate (2D/3D)) in experiment 1. The training parameters of 2D GoogleNet and 3D GoogleNet are set: 16/2 (batch size (2D/3D)), 512 × 512/512 × 512 × 20 (input size (2D/3D)), 50/50 (epoch (2D/3D)), and 0.2/0.2 (drop rate (2D/3D)) in experiment 1. MIL: each case (a set of chest HRCT images) was equally divided into 20 segments, with one slice taken equidistantly to obtain 20 slices in each case. The training parameters of 2D DenseNet with MIL (2D DenseNet_MIL) and 3D DenseNet with MIL (3D DenseNet_MIL) are set: 16/2 (batch size (2D/3D)), 512 × 512/512 × 512 × × 512 × 16 (input size (2D/3D)), 50/50 (epoch (2D/3D)), and 0.5/0.2 (drop rate (2D/3D)) in experiment 2. The training parameters of 2D GoogleNet with MIL (2D GoogleNet_MIL) and 3D GoogleNet with MIL (3D GoogleNet_MIL) are set: 16/2 (batch size (2D/3D)), 512 × 512∗∗/512 × 512 × 16 (input size (2D/3D)), 50/50 (epoch (2D/3D)), and 0.2/0.2 (drop rate (2D/3D)) in experiment 2. MIL: each case was equally divided into 10 bags, with one slice taken randomly to obtain 10 slices in each case. MIL: each case was equally divided into 16 bags, with one slice taken equidistantly to obtain 16 slices in each case.

Specifically, experiments 3–5 are designed to compare the classification performance of the six different classifiers based on the CNN feature vector (13824), radiomics feature vector (1316), their selected feature vector by Lasso, and the proposed combined feature vector (166), respectively. First, based on 3D ResNet10, we use six classic classifiers (SVM [35], MLP, RF [36], LR [37], GB [38], and LDA [39]) to determine the best COPD classification classifier by different feature vectors. Table 1 reports the six different classifiers with their definitions in this paper. The different feature vectors include the CNN feature vector (13824), CNN feature vector selected by Lasso (60), radiomics feature vector (1316), and radiomics feature vector selected by Lasso (106). The MLP classifier with the best classification performance is determined. Second, we further verify the proposed combined feature vector (166) to improve the MLP classifier’s performance. Third, 3D ResNet18 and 3D ResNet34 are also transferred to generate the CNN feature vector, and the 3D ResNet10 is determined as the encoder backbone with the best performance on the MLP classifier. The 465 subjects are divided into the train set (70%) and the test set (30%). Figure 4 shows the detailed dataset division for training and test set in each COPD stage.

(a)

(b)

Standard evaluation metrics of the CNN and ML models include the accuracy, precision, recall, F1-score, and area under the curve (AUC). The above standard evaluation metrics are defined as in equations (3)–(6). The evaluation metric AUC for multi-classification is calculated by the receiver operating characteristic curve (ROC) [40].where the true positive (TP) and false positive (FP), respectively, represent the positive and negative samples classified to be positive by the CNN and ML models and the true negative (TN) and false negative (FN), respectively, represent the positive and negative samples classified to be negative by the CNN and ML models.

3. Results

This section reports the experimental results, including (1) the classification performance of the parenchyma images based on the DenseNet and GoogleNet; (2) the classification performance of the CNN feature vector and lung radiomics vector based on different classifiers; (3) the MLP classifier’s performance with the combined feature vector; and (4) the MLP classifier’s performance with combined feature vector based on different 3D ResNet.

3.1. The DenseNet and GoogleNet’s Performance with Parenchyma Images

This section shows the classification performance of 2D/3D DenseNet, 2D/3D GoogleNet, 2D/3D DenseNet_MIL, and 2D/3D GoogleNet_MIL based on the parenchyma images, respectively.

Figure 5 intuitively shows the AUC of the CNN models by drawing the ROC curves. Tables 2 and 3 report the classification performance of CNN models. Specifically, Table 2 shows that 2D GoogleNet with parenchyma images performs the best in 2D CNN models, achieving 0.550 (accuracy), 0.562 (mean precision), 0.550 (mean recall), 0.553 (mean F1-score), and 0.809 (AUC). In addition, Table 3 shows that 3D DenseNet with parenchyma images performs the best in 3D CNN models, achieving 0.579 (accuracy), 0.614(mean precision), 0.579 (mean recall), 0.579 (mean F1-score), and 0.787 (AUC).

(a)

(b)

3.2. The Classification Performance of CNN Feature Vector and Lung Radiomics Vector Based on Different Classifiers

This section shows the classification performance of the CNN feature vector (13824), the CNN feature vector selected by Lasso (60), the lung radiomics vector (1316), and the lung radiomics vector selected by Lasso (106) based on different classifiers, respectively.

Figure 6 intuitively shows the AUC of the different classifiers by drawing the ROC curves. Tables 4–7 show that the MLP classifier is the best classifier for COPD stage classification. Specifically, Table 4 reports the classification performance of the different classifiers with the CNN feature vector (13824), respectively. The best classifier is the MLP classifier with 0.793 (accuracy), 0.798 (mean precision), 0.793 (mean recall), 0.790 (mean F1-score), and 0.790 (AUC), respectively. Table 5 reports that the classification performance of the MLP classifier with the CNN feature vector selected by Lasso has improved with 0.821 (accuracy), 0.826 (mean precision), 0.821 (mean recall), 0.821 (mean F1-score), and 0.946 (AUC), respectively. Table 6 reports that the classification performance of the MLP classifier with the radiomics feature vector selected by Lasso has improved with 0.786 (accuracy), 0.784 (mean precision), 0.784 (mean recall), 0.784 (mean F1-score), and 0.919 (AUC), respectively. Table 7 reports that the classification performance of the MLP classifier with the radiomics feature vector selected by Lasso has improved with 0.829 (accuracy), 0.828 (mean precision), 0.829 (mean recall), 0.824 (mean F1-score), and 0.950 (AUC), respectively.

(a)

(b)

(c)

(d)

Table 5 also reports that Lasso only plays a role in improving the classification performance of the MLP classifier with the CNN feature vector. It does not improve the classification performance of other classifiers with the CNN feature vector. However, Table 7 reports that Lasso does play a role in improving the classification performance of all classifiers with the radiomics feature vector.

3.3. The MLP Classifier’s Performance with Combined Feature Vectors

The best MLP classifier is determined with the CNN feature vector selected by Lasso (60) or the lung radiomics vector selected by Lasso (106) by Section 3.1. This section shows the classification performance of the MLP classifier with combined feature vectors.

Figure 7 intuitively shows the confusion matrix and ROC curves of the MLP classifier with different feature vectors based on 3D ResNet10. The MLP classifier’s performance with different feature vectors reported in Table 8 can be calculated from the confusion matrix. Table 8 reports that the proposed combined feature vectors improve the MLP classifier’s performance, achieving 0.879 (accuracy), 0.879 (mean precision), 0.879 (mean recall), 0.875 (mean F1-score), and 0.971 (AUC), respectively.

(a)

(b)

(c)

(d)

3.4. The MLP Classifier’s Performance with Combined Feature Vector Based on Different 3D ResNet

The best MLP classifier is determined with the CNN feature vector selected by Lasso (60) or the lung radiomics vector selected by Lasso (106) by Section 3.1. This section shows the classification performance of the MLP classifier with combined feature vectors.

Figure 8 intuitively shows the confusion matrix and ROC curves of the MLP classifier with combined feature vectors based on different 3D ResNet. The MLP classifier’s performance with combined feature vectors based on different 3D ResNet reported in Table 7 can be calculated from the confusion matrix. Table 9 reports that the MLP classifier with combined feature vectors based on 3D ResNet10 achieves the best classification performance.

(a)

(b)

(c)

(d)

Figure 8

The confusion matrix and ROC curves of the MLP classifier with combined feature vectors based on different 3D ResNet. (a) The confusion matrix of the MLP classifier with combined feature vector based on 3D ResNet10. (b) The confusion matrix of the MLP classifier with combined feature vector based on 3D ResNet18. (c) The confusion matrix of the MLP classifier with combined feature vector based on 3D ResNet34. (d) The ROC curves of the MLP classifier with combined feature vectors based on 3D ResNet.

4. Discussion

This paper proposes a features combination strategy by concatenating 3D CNN features and lung radiomics features for COPD stage classification based on the MLP classifier. Three sections are discussed in this section, and we also point out the limitations in this study and the future direction.

First, 2D GoogleNet with parenchyma images performs the best in 2D CNN models. The main reason is that 2D GoogleNet is designed for 2D natural image classification (RGB images). Therefore, it achieves the best classification performance in 2D parenchyma images. Meanwhile, because of the ability to extract interlayer information, 3D DenseNet with parenchyma images performs the best classification in 3D CNN models. However, CNN models with parenchyma images fail to classify the COPD stage. One main reason is that the chest HRCT image cannot fully reflect COPD’s characteristics for the CNN models. Specifically, the gold standard of COPD classification is characterized by airflow restriction with a slight difference in the chest HRCT image. The slight difference in COPD is mainly caused by small airway disease with an airway diameter<2 mm [17]. Because of the limitation of HRCT resolution, the above differential features of the small airway will be further blurred in the chest HRCT image. Another reason is that chest HRCT images can reflect the COPD anatomical characteristics, but COPD patients are with high heterogeneity and different phenotypes [1]. The heterogeneity and different phenotypes often result in different features of the chest HRCT images in the same stage. Therefore, it is hard for CNN models to learn specific COPD characteristics, resulting in bad classification performance. At the same time, a set of standard medical images is not as easy to obtain as natural images, and the number of chest HRCT images also restricts CNN models for COPD stage classification. Therefore, compared with CNN models, the ML classifier can realize the COPD stage classification with a small number of samples. This paper determines the MLP classifier with 3D CNN features or lung radiomics features, which performs the best for COPD stage classification. In addition, compared with the convolution layer in the CNN models, the MLP classifier is composed of three full connection layers, which is more efficient and more suitable for modeling long-range dependencies. The MLP classifier also can handle complex nonlinear features and discover dependencies between different input features compared with other classifiers [31, 32]. Meanwhile, the objective evaluation of the COPD stage is only the degree of airflow limitation tested by GOLD criteria [1, 2, 4]. COPD is a heterogeneous disease [41], resulting in differences in features (3D CNN features or lung radiomics features extracted from chest HRCT images) with the same degree of airflow limitation. Therefore, a nonlinear relationship exists between 3D CNN features or lung radiomics features and the COPD stage. Because of this, the MLP classifier is suitable for classifying the COPD stage and has achieved an excellent result in COPD stage classification.

Second, Lasso can improve the classification performance of the MLP classifier with the 3D CNN features and the lung radiomics features. Lasso is often used with survival analysis models to determine variables and eliminate the collinearity problem between variables [30, 42]. The results show that Lasso also can improve the MLP classifier’s classification performance by establishing the relationship between the independent variables (3D CNN features or lung radiomics features extracted from chest HRCT images) and dependent variables (the COPD stages). Furthermore, Lasso selects 3D CNN features or lung radiomics features related to COPD stages to reduce the complexity of the MLP classifiers and avoid overfitting [43]. While reducing the complexity of the MLP classifiers, the MLP classifiers can focus on the selected lung radiomics features (the radiomics feature vector selected by Lasso) or the selected 3D CNN features (the CNN feature vector selected by Lasso) and improve the classification performance. From the results of the Lasso, the number of the CNN feature vector selected by Lasso is 60, and that of the radiomics feature vector selected by Lasso is 106. We are surprised that the number of collinearity features in the CNN feature vector is more than that in the radiomics feature vector. This further shows that feature selection of 3D CNN features or the radiomics features is necessary for the COPD stage classification, especially in clinical applications.

Third, the proposed feature combination strategy can further improve the classification performance of the MLP classifier. This paper does not improve the existing classic classifiers and starts with the classification features to enhance the classifier’s performance. Many nonlinear classification features, the 3D CNN features, are obtained by a truncated transfer learning strategy. We concatenate the CNN feature vector and the radiomics feature vector for the COPD stage classification, which improves the MLP classifier’s performance. The MLP classifier is good at handling complex nonlinear features by itself [31, 32]. Therefore, based on the radiomics feature vector, we add the nonlinear CNN feature vector to the radiomics feature vector, generating a combined feature vector. The combined feature vector with the nonlinear CNN feature vector enhances the MLP classifier’s performance. Therefore, this fits the essence of the MLP classifier and is interpretable [44]. The selected encoder backbone of the pretrained Med3D is also directly related to the classification performance. Compared with the MLP classifier with 3D ResNet18 or 3D ResNet34, the MLP classifier with 3D ResNet10 performs the best, consistent with the results of multi-class segmentation task (left lung, right lung, and background) in reference [25].

Finally, this study has some limitations, and we point out the future direction. First, from the materials used in this study, there are not enough cases at the COPD stages III and IV. Second, the existing classic classifiers are not improved. Third, the classification performance of the ML classifier with the 3D CNN features is also limited by the encoder backbone of the pretrained Med3d. In our future work, the recent networks, an auto-metric graph neural network [45], will be further attempted and modified for COPD stage classification based on the 3D CNN features and/or the lung radiomics features.

5. Conclusions

This paper proposes a feature combination strategy by concatenating 3D CNN features and lung radiomics features for COPD stage classification based on the MLP classifier. First, the 3D CNN features are extracted from the lung region images based on a truncated transfer learning strategy. Then, the lung radiomics features are extracted from the lung region images by PyRadiomics. Compared with CNN models and other ML classifiers, the MLP classifier with the best classification performance is determined by the 3D CNN features and the lung radiomics features. Lasso plays a role in improving the classification performance of the MLP classifier with the CNN feature vector and the radiomics feature vector. The proposed combined feature vector also improves the MLP classifier’s performance. The MLP classifier with the proposed combined feature vector has accuracy, mean precision, mean recall, mean F1-score, and AUC of 0.879, 0.879, 0.879, 0.875, and 0.971, respectively. This shows that our method effectively improves the classification performance for COPD stage classification.

Data Availability

The datasets used and analyzed during the current study are available from the corresponding authors on reasonable request.

Disclosure

Yingjian Yang and Nanrong Zeng are co-first authors.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

Yingjian Yang and Nanrong Zeng contributed equally to this work.

Acknowledgments

Thanks are due to the Department of Radiology, the First Affiliated Hospital of Guangzhou Medical University, for providing the dataset. This research was funded by the National Natural Science Foundation of China (grant no. 62071311), Stable Support Plan for Colleges and Universities in Shenzhen of China (grant no. SZWD2021010), Scientific Research Fund of Liaoning Province of China (grant no. JL201919), Natural Science Foundation of Guangdong Province of China (grant no. 2019A1515011382), and Special Program for Key Fields of Colleges and Universities in Guangdong Province (Biomedicine and Health) of China (grant no. 2021ZDZX2008).

References

D. Singh, A. Agusti, A. Anzueto et al., “Global strategy for the diagnosis, management, and prevention of chronic obstructive lung disease: the GOLD science committee report 2019,” European Respiratory Journal, vol. 53, no. 5, Article ID 1900164, 2019.
View at: Publisher Site | Google Scholar
M. C. Matheson, G. Bowatte, J. L. Perret et al., “Prediction models for the development of COPD: a systematic review,” International Journal of Chronic Obstructive Pulmonary Disease, vol. 13, pp. 1927–1935, 2018.
View at: Publisher Site | Google Scholar
Y. Yang, W. Li, Y. Kang et al., “A novel lung radiomics feature for characterizing resting heart rate and COPD stage evolution based on radiomics feature combination strategy,” Mathematical Biosciences and Engineering, vol. 19, no. 4, pp. 4145–4165, 2022.
View at: Publisher Site | Google Scholar
Y. Yang, W. Li, Y. Guo et al., “Early COPD risk decision for adults aged from 40 to 79 Years based on lung radiomics features,” Frontiers of Medicine, vol. 9, Article ID 845286, 2022.
View at: Publisher Site | Google Scholar
P. W. Jones, “Health status measurement in chronic obstructive pulmonary disease,” Thorax, vol. 56, 2001.
View at: Google Scholar
C. D. Brown, J. O. Benditt, F. C. Sciurba et al., “Exercise testing in severe emphysema: association with quality of life and lung function,” COPD: Journal of Chronic Obstructive Pulmonary Disease, vol. 5, no. 2, pp. 117–124, 2008.
View at: Publisher Site | Google Scholar
D. A. Lynch, “Progress in imaging COPD, 2004- 2014,” Chronic Obstructive Pulmonary Diseases Journal of the Copd Foundation, vol. 1, no. 1, pp. 73–82, 2014.
View at: Publisher Site | Google Scholar
P. J. Castaldi, R. San José Estépar, C. S. Mendoza et al., “Distinct quantitative computed tomography emphysema patterns are associated with physiology and function in smokers,” American Journal of Respiratory and Critical Care Medicine, vol. 188, no. 9, pp. 1083–1090, 2013.
View at: Publisher Site | Google Scholar
K. Weiss, T. M. Khoshgoftaar, and D. D. Wang, “Finding a moral compass without a compass: evolution and ethics,” Evolutionary Anthropology, vol. 25, no. 1, pp. 1–5, 2016.
View at: Publisher Site | Google Scholar
P. Lambin, E. Rios-Velazquez, and R. Leijenaar, “Radiomics: extracting more information from medical images using advanced feature analysis,” European Journal of Cancer, vol. 43, no. 4, pp. 441–446, 2007.
View at: Google Scholar
A. N. Frix, F. Cousin, T. Refaee et al., “Radiomics in lung diseases imaging: state-of-the-art for clinicians,” Journal of Personalized Medicine, vol. 11, no. 7, p. 602, 2021.
View at: Publisher Site | Google Scholar
S. M. Rezaeijo, R. Abedi-Firouzjah, and M. Ghorvei, “Screening of COVID-19 based on the extracted radiomics features from chest CT images,” Journal of X-Ray Science and Technology, vol. 29, no. 4, pp. 1–15, 2021.
View at: Google Scholar
F. Xiao, R. Sun, W. Sun et al., “Radiomics analysis of chest CT to predict the overall survival for the severe patients of COVID-19 pneumonia,” Physics in Medicine and Biology, vol. 66, no. 10, 2021.
View at: Publisher Site | Google Scholar
F. Xiong, Y. Wang, T. You et al., “The clinical classification of patients with COVID-19 pneumonia was predicted by Radiomics using chest CT,” Medicine, vol. 100, no. 12, Article ID e25307, 2021.
View at: Publisher Site | Google Scholar
M. Tamal, M. Alshammari, M. Alabdullah, R. Hourani, H. A. Alola, and T. M. Hegazi, “An integrated framework with machine learning and radiomics for accurate and rapid early diagnosis of COVID-19 from Chest X-ray,” Expert Systems with Applications, vol. 180, Article ID 115152, 2021.
View at: Publisher Site | Google Scholar
T. Refaee, G. Wu, and A. Ibrahim, “The emerging role of radiomics in COPD and lung cancer,” Respiration, vol. 99, no. 2, pp. 1–9, 2020.
View at: Google Scholar
R. A. O'donnell, C. Peebles, and J. A. Ward, “Relationship between peripheral airway dysfunction, airway obstruction, and neutrophilic inflammation in COPD,” Thorax, vol. 59, no. 10, pp. 837–842, 2004.
View at: Publisher Site | Google Scholar
J. L. Wright and A. Churg, “Advances in the pathology of COPD: Advances in the pathology of COPD,” Histopathology, vol. 49, no. 1, pp. 1–9, 2006.
View at: Publisher Site | Google Scholar
V. I. Peinado, S. Pizarro, and J. A. Barbera, “Pulmonary vascular involvement in COPD,” Chest, vol. 134, no. 4, pp. 808–814, 2008.
View at: Publisher Site | Google Scholar
L. Huang, W. Lin, D. Xie et al., “Development and validation of a preoperative CT-based radiomic nomogram to predict pathology invasiveness in patients with a solitary pulmonary nodule: a machine learning approach, multicenter, diagnostic study,” European Radiology, vol. 32, no. 3, pp. 1983–1996, 2022.
View at: Publisher Site | Google Scholar
R. C. Au, W. C. Tan, J. Bourbeau, J. C. Hogg, and M. Kirby, “Impact of image pre-processing methods on computed tomography radiomics features in chronic obstructive pulmonary disease,” Physics in Medicine and Biology, vol. 66, no. 24, Article ID 245015, 2021.
View at: Publisher Site | Google Scholar
J. Yun, Y. H. Cho, S. M. Lee et al., “Deep radiomics-based survival prediction in patients with chronic obstructive pulmonary disease,” Scientific Reports, vol. 11, no. 1, pp. 15144–15149, 2021.
View at: Publisher Site | Google Scholar
R. C. Au, W. C. Tan, J. Bourbeau, J. C. Hogg, and M. Kirby, “Radiomics analysis to predict presence of chronic obstructive pulmonary disease and symptoms using machine learning,” TP121. TP121 Copd: From Cells To The Clinic, American Thoracic Society, New York, NY, USA, 2021.
View at: Google Scholar
Y. Zhou, P. Bruijnzeel, C. Mccrae et al., “Study on risk factors and phenotypes of acute exacerbations of chronic obstructive pulmonary disease in Guangzhou, China-design and baseline characteristics,” Journal of Thoracic Disease, vol. 7, no. 4, pp. 720–733, 2015.
View at: Publisher Site | Google Scholar
S. Chen, K. Ma, and Y. Zheng, “Med3d: transfer learning for 3d medical image analysis,” 2019, https://arxiv.org/abs/1904.00625.
View at: Google Scholar
J. Hofmanninger, F. Prayer, J. Pan, S. Rohrich, H. Prosch, and G. Langs, “Automatic lung segmentation in routine imaging is a data diversity problem, not a methodology problem,” European Radiology Experimental, vol. 4, pp. 1–13, 2020.
View at: Google Scholar
Y. Yang, Y. Guo, J. Guo, Y. Gao, and Y. Kang, “A method of abstracting single pulmonary lobe from computed tomography pulmonary images for locating COPD,” in Proceedings of the Fourth International Conference on Biological Information and Biomedical Engineering, pp. 1–6, Chengdu China, July 2020.
View at: Google Scholar
Y. Yang, Q. Li, Y. Guo et al., “Lung parenchyma parameters measure of rats from pulmonary window computed tomography images based on ResU-Net model for medical respiratory researches,” Mathematical Biosciences and Engineering, vol. 18, no. 4, pp. 4193–4211, 2021.
View at: Publisher Site | Google Scholar
J. J. M. Van Griethuysen, A. Fedorov, C. Parmar et al., “Computational radiomics system to decode the radiographic phenotype,” Cancer Research, vol. 77, no. 21, pp. e104–e107, 2017.
View at: Publisher Site | Google Scholar
R. Tibshirani, “Regression shrinkage and selection via the Lasso,” Journal of the Royal Statistical Society: Series B, vol. 58, no. 1, pp. 267–288, 1996.
View at: Publisher Site | Google Scholar
S. Wan, Y. Liang, Y. Zhang, and M. Guizani, “Deep multi-layer perceptron classifier for behavior analysis to estimate Parkinson’s disease severity using smartphones,” IEEE Access, vol. 6, pp. 36825–36833, 2018.
View at: Publisher Site | Google Scholar
M. Taki, A. Rohani, F. Soheili-Fard, and A. Abdeshahi, “Assessment of energy consumption and modeling of output energy for wheat production by neural network (MLP and RBF) and Gaussian process regression (GPR) models,” Journal of Cleaner Production, vol. 172, pp. 3028–3041, 2018.
View at: Publisher Site | Google Scholar
Q. Li, Y. Yang, Y. Guo et al., “Performance evaluation of deep learning classification network for image features,” IEEE Access, vol. 9, pp. 9318–9333, 2021.
View at: Publisher Site | Google Scholar
M. A. Carbonneau, V. Cheplygina, E. Granger, and G. Gagnon, “Multiple instance learning: a survey of problem characteristics and applications,” Pattern Recognition, vol. 77, pp. 329–353, 2018.
View at: Publisher Site | Google Scholar
V. Jakkula, “Tutorial on support vector machine (svm),” School of EECS, Washington State University, vol. 37, 2006.
View at: Google Scholar
Y. Qi, “Random forest for bioinformatics,” Ensemble machine learning, Springer, Boston, MA, USA, 2012.
View at: Google Scholar
M. P. LaValley, “Logistic regression,” Circulation, vol. 117, no. 18, pp. 2395–2399, 2008.
View at: Publisher Site | Google Scholar
V. K. Ayyadevara, “Gradient boosting machine,” in Pro Machine Learning Algorithms, pp. 117–134, Springer, Berlin, Germany, 2018.
View at: Google Scholar
W. Hu, W. Hu, and S. Maybank, “Adaboost-based algorithm for network intrusion detection,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 38, no. 2, pp. 577–583, 2008.
View at: Publisher Site | Google Scholar
Y. Yang, W. Li, Y. Guo et al., “Lung radiomics features for characterizing and classifying COPD stage based on feature combination strategy and multi-layer perceptron classifier,” Mathematical Biosciences and Engineering, vol. 19, no. 8, pp. 7826–7855, 2022.
View at: Publisher Site | Google Scholar
C. Casanova, J. P. de Torres, A. Aguirre-Jaíme et al., “The progression of chronic obstructive pulmonary disease is heterogeneous: the experience of the BODE cohort,” American Journal of Respiratory and Critical Care Medicine, vol. 184, no. 9, pp. 1015–1021, 2011.
View at: Publisher Site | Google Scholar
N. Simon, J. Friedman, T. Hastie, and R. Tibshirani, “Regularization paths for Cox's proportional hazards model via coordinate descent,” Journal of Statistical Software, vol. 39, no. 5, pp. 1–13, 2011.
View at: Publisher Site | Google Scholar
D. M. McNeish, “Using Lasso for predictor selection and to assuage overfitting: a method long overlooked in behavioral sciences,” Multivariate Behavioral Research, vol. 50, no. 5, pp. 471–484, 2015.
View at: Publisher Site | Google Scholar
J. B. Lont, Analog CMOS Implementation of a Multi-Layer Perceptron with Nonlinear synapses, ETH Zurich, Zürich, Switzerland, 1993.
X. Song, M. Mao, and X. Qian, “Auto-metric graph neural network based on a meta-learning strategy for the diagnosis of alzheimer's disease,” IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 8, pp. 3141–3152, 2021.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2023 Yingjian Yang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

220

Downloads

257

Citations