Efficient Model for Coronary Artery Disease Diagnosis: A Comparative Study of Several Machine Learning Algorithms
Background. In today’s industrialized world, coronary artery disease (CAD) is one of the leading causes of death, and early detection and timely intervention can prevent many of its complications and eliminate or reduce the resulting mortality. Machine learning (ML) methods as one of the cutting-edge technologies can be used as a suitable solution in diagnosing this disease. Methods. In this study, different ML algorithms’ performances were compared for their effectiveness in developing a model for early CAD diagnosis based on clinical examination features. This applied descriptive study was conducted on 303 records and overall 26 features, of which 26 were selected as the target features with the advice of several clinical experts. In order to provide a diagnostic model for CAD, we ran most of the most critical classification algorithms, including Multilayer Perceptron (MLP), Support Vector Machine (SVM), Logistic Regression (LR), J48, Random Forest (RF), K-Nearest Neighborhood (KNN), and Naive Bayes (NB). Seven different classification algorithms with 26 predictive features were tested to cover all feature space and reduce model error, and the most efficient algorithms were identified by comparison of the results. Results. Based on the compared performance metrics, SVM (AUC = 0.88, F-measure = 0.88, ROC = 0.85), and RF (AUC = 0.87, F-measure = 0.87, ROC = 0.91) were the most effective ML algorithms. Among the algorithms, the KNN algorithm had the lowest efficiency (AUC = 0.81, F-measure = 0.81, ROC = 0.77). In the diagnosis of coronary artery disease, machine learning algorithms have played an important role. Proposed ML models can provide practical, cost-effective, and valuable support to doctors in making decisions according to a good prediction. Discussion. It can become the basis for developing clinical decision support systems. SVM and RF algorithms had the highest efficiency and could diagnose CAD based on patient examination data. It is suggested that further studies be performed using these algorithms to diagnose coronary artery disease to obtain more accurate results.
According to the World Health Organization (WHO), in 2020, cardiovascular disease, as one of the noncommunicable diseases, was the most important cause of death globally. Among cardiovascular diseases, coronary artery disease (CAD) is one of the most common diseases and the leading cause of death in developed countries. This disease is caused by the accumulation of platelets in the arteries or atherosclerosis, which blocks blood flow and increases the risk of heart attack and stroke. After the collection of fatty plaques and calcification, the blood vessels, which are initially soft and elastic, become narrow and hard, causing myocardial infarction in addition to narrowing the arteries [1–3].
Early CAD disease and early interventions to treat the disease and prevent its complications are vital strategies to combat and reduce mortality effectively. That is why healthcare providers use strategies to diagnose the disease early and start the treatment or intervention process early. In recent decades, numerous methods have been proposed for the early diagnosis of coronary artery disease. Many researchers across a wide range of scientific disciplines have argued that one of the most successful solutions for the early diagnosis of diseases, which has become increasingly common in recent years, is artificial intelligence (AI), which can improve diagnostic accuracy [3–6]. AI uses human intelligence to explore data relationships and mimic human problem-solving patterns. An essential category of AI is machine learning techniques that have played a pivotal role in diagnosing diseases. Machine learning (ML), as a common category of AI, has many applications in diagnosing and predicting different diseases. ML techniques have been used in various health research fields, and good results have been obtained. ML algorithms are a standard tool in knowledge exploration, which isused to create prediction and diagnosis models with high accuracy [7–9]. ML classification algorithms learn to analyze data by monitoring and discovering relationships between data. Monitoring by learning from labelled data allows us to identify data output that has never been seen before. This method is a general ML technique that gives a system a list of input-output pairs, and the system tries to find a function from input to output. This method is known as supervised learning since it requires some input data. However, there are specific issues for which supervised learning systems cannot generate the required output [10–15]. Previous studies on CAD datasets have included data from various care and diagnostic methods, including demographic information, lifestyle indicators, multiple examinations, patient laboratory test results, and cardiac tests such as ECG, exercise testing, etc. They had used the information. However, in the present study, only clinical examination data of patients and their demographic data were used to reach an early diagnosis. Therefore, the present study aimed to evaluate and compare different ML classification algorithms to determine an efficient algorithm for diagnosing coronary artery disease based on examination data of patients with coronary artery disease based on a clean dataset.
2. Related Work
With the increasing prevalence of CAD and the limitations of diagnostic tests, many studies have used several machine learning techniques to improve the accuracy of the adopted diagnostic methods. Table 1 lists some relevant studies.
The studies were limited by the lack of clinical methods for selecting features. For these studies, two approaches were used to select features: either all features were used without data reduction techniques, or they were selected based on mathematical algorithms without considering their clinical relevance.
The present study aimed to design a model based on cheap clinical data for the first time in CAD diagnosis. In 2022, this study was carried out as a cross-sectional analysis of ML algorithms. To achieve the best performance in CAD data pattern recognition, different ML algorithms were used to analyze the data and compare their efficiency.
3.1. Dataset Description and Feature Selection
The used dataset is the Z-Alizadeh Sani dataset, which includes 303 data points from people suspected of having CAD, including 215 individuals who had CAD and 88 patients with normal which was their exact status confirmed using catheterization. This dataset contains 54 attributes (features) for every case in the dataset that can be utilized as CAD markers for patients (the features are arranged in four groups: demographics, symptoms and examination, ECG, and laboratory and echo features) This dataset is one of the most widely used datasets for automatic CAD detection in ML. A patient is diagnosed with CAD when one or more of his or her coronary arteries are stenosed. If the diameter of a coronary artery narrows by 50%, it is called stenosis .
In the present study, the development of a reliable selection model for diagnosing CAD disease was entirely based on clinical characteristics. These features were selected after consideration of the recommendations of three clinical cardiologists. A checklist was made to help determine the features. Following the registration of each feature using the checklist, the cost of measuring and registering each feature was calculated based on its accessibility. Finally, after this checklist was analyzed by the researchers, these clinical specialists checked and completed the checklists, and 26 features out of a total of 54 elements were chosen based on their clinical value and accessibility. By selecting these features from the original dataset, a dataset that was clinically useful and had features that had the most effects on the diagnosis of CAD was obtained. The models given in this research can be utilized to diagnose CAD using native data because access to the database with these features is more straightforward and less expensive (Table 2).
In the used dataset, the selected features were nominal (binary), and only age, weight, length, and BMI were numeric features. One of the strengths of the study is the use of cleaned data without any missing data. Utilization of this data has prevented the data preprocessing steps in the data mining process from being performed, and instead the data analysis process was performed directly using ML algorithms.
Data categorization is a crucial first step in using learning-based research models based on ML . With no normalization and a ratio of 64 : 20 : 16, the dataset was divided into training, testing, and validation sets in the current study. Of the present research dataset, 80% was used for training (learning and validation), and 20% for testing. Table 3 provides a detailed presentation of this classification. Additionally, to prevent the network from seeing data from a particular class during training, the training data were shuffled several times, each category comprising data with distinct labels.
In the present study, because the dataset is a standard public dataset and was collected under the supervision of a cardiologist, no preprocessing techniques including data cleansing and feature engineering were performed in this study.
3.2. Selective ML Algorithms
In light of the strengths of each ML algorithm and its capability to extract patterns from the datasets, several algorithms were used to find the most efficient algorithm for designing the diagnosis model of CAD. The algorithms selected include the multilayer perceptron (MLP), the support vector machine (SVM), the logistic regression (LR), the J48, the random forest (RF), the K-Nearest neighborhood (KNN), and the naive bayes (NB).
One of the most widely used ML algorithms is a neural network (NN), and the most prevalent algorithm on NN architectures is MLP which belongs to the class of supervised neural networks. An MLP network typically consists of three or more layers of nodes: an input layer that receives external inputs, one or more hidden layers, and an output layer that produces classification results. Errors are reduced using the gradient descent algorithm in this model [13, 26, 27].
SVM is one of the main supervised learning algorithms presented by Vladimir Vapnik within the area of statistical learning theory and structural risk minimization, which has been successfully applied to several classification and forecasting problems. The SVM has been applied to various problems related to pattern recognition and regression estimation, as well as medical diagnosis for disease classification. SVM is a powerful method for building classifiers. It allows the prediction of labels from one or more feature vectors by creating a decision boundary between two classes. Known as the hyperplane, this boundary is oriented to be as far away as possible from the closest data points from each class. The closest points are called support vectors .where is the class label (positive or negative) of a training compound I and is a feature vector representation. Thus, the optimal hyperplane is given by equations (1)–(3):
For all components of the training set, the and b would meet the following inequalities:
Vectors for which | | will be termed support vectors (Figure 1).
LR is a type of nonlinear regression that takes categorical data as input. COX introduced the idea of LR in 1958, based on the principle of estimating a binary response based on a set of independent features. LR uses the logistic function to predict the probability of occurrence using the input feature set .
J48 is an upgraded version of the ID3 classification ML algorithm choice tree, which is based on a calculation called ID3 (Iterative Dichotomiser variant 3), developed by the WEKA undertaking group. The J48 calculation has a clear decision tree for the C4.5 gathering. There is a double tree in the situation. It is also known as a decision tree prediction algorithm for its steadiness in grouping issues. This way, simple, easy-to-understand rules can be constructed using this algorithm .
RF was introduced in 2001. The random space approach and bagging decision trees (DT) are the two methods that make up RF. The RF classifier comprises numerous DTs that have been trained using the bagging approach. After receiving the results of all DTs and voting on the results of all DTs, the final classification result is determined. Several classification and regression trees (CART) will be created by RF, each trained on a bootstrap sample of the original training data, and each searching a randomly chosen subset of input variables to find the split. By continually dividing the data in a node into child nodes, starting with the root node that holds the entire learning sample, binary decision trees known as CARTs are created. Each tree in the RF will vote for one or more inputs, and the majority vote of the trees will determine the classifier’s output. High-dimensional data may be handled by RF, and the ensemble uses several trees. RF is a highly recommended classifier for dealing with situations like overfitting and underfitting. Noise and outliers can also be handled with RF. RF is a well-known classification technique that has been successfully applied to the categorization of a variety of medical datasets . Some key characteristics of RF include:(1)It has a good method for guessing data that are absent.(2)Weighted random forest (WRF) is a technique for balancing inaccuracy in unbalanced data.(3)It calculates the significance of the classification’s variables.
The simplest of all classifiers is KNN which belongs to the lazy learning algorithms family. Because KNN is a classifier-based instance, it may simply be constructed in parallel. In feature space, KNN utilizes majority voting among the labels for the K nearest data points, where K is an integer number. For continuous variables, Euclidean distance is used as a distance measure, while for discrete variables, hamming distance is used [15, 32] as a distance measure. The KNN classifier, also known as case-based reasoning, has been employed in a wide range of applications, including recognition and estimation. It is preferred over other classifiers due to its simplicity and high convergence speed [33, 34]. NB is a probabilistic statistical classifier with the advantages of accurate classification and excellent processing efficiency. When the input data have high dimensionality, NB is chosen. The label that optimizes the posterior probability is returned as an output in NB-based classification .
3.3. Metrics Evaluation
In this study, the performance of the selective classifier algorithms is evaluated via clinically meaningful statistical measures like precision, recall, F-measure, MCC (Matthews correlation coefficient), PRC (precision-recall curve) specificity, and F1 Score. To calculate these evaluation metrics, the following variables are required: TP (true positive), FP (false positive), which is how to calculate the indicators with the calculation formula.
The number of incorrect predictions of negative cases by the method. However, accuracy is not always a proper metric to evaluate model performance, especially in the case of an asymmetrical dataset. However, in this research, the accuracy metrics were to select the most efficient model through the selected pretrained networks. Equations (4)–(8) are used to briefly describe these measurements.
Accuracy: This parameter measures the ratio of accurately predicted cases to the total number of cases to assess a method’s performance . Mathematically, it is expressed as:
Precision, the ratio of observations in the actual classes that were correctly predicted as positive to all other observations [15, 32]. This metric shows how often different illness types are correctly classified.
F1 score, is one of the measures used frequently to assess a classifier’s effectiveness. It is the harmonic mean of precision and recall . The F-1 score is a relevant measurement for classification issues on unbalanced datasets, since it is more sensitive to data distribution.
MCC (Matthews correlation coefficient), the most important metric that has been selected as the elective metric in the USFDA-led initiative MAQCII which aims at developing and validating predictive models for personalized medicine. The MCC is calculated as (5).
In this research to reach high performance in the diagnosis of CAD using a clinical dataset, we selected seven different well-known classifiers for the diagnosis of coronary artery disease, based on the most frequently used ML algorithms in the field of diagnosis and classification of diseases. We assessed the efficacy of selective algorithms concerning patient data suspected of having CAD. The values for the performance metrics of the ML algorithms are shown in Table 3. In this table, eight proprietary metrics are used to assess the performance of ML algorithms. Based on the TP, PT, recall, F-measure, and MCC metrics for all algorithms implemented, it can be concluded that the highest amount of these metrics is attributed to the SVM algorithm, and the highest amount of ROC and PRC are related to the RF algorithm. Also, the lowest FP was related to the SVM algorithm. As a general rule, cross-validation epochs involve partitioning data into two complementary subsets. One of these sets will be used for training and fitting, and the other will be used for validation and testing. The results are averaged after numerous iterations of the validation using various subsets. The data are divided into K subsets for K-fold cross-validation, one of which is utilized for validation and the other K-1 for training. In this approach, each data fold is utilized exactly once for training and once for validation. As a result, we decide to base our final estimate on the averages of these K validation periods. Therefore, in this study, the 10-fold validation method was used, and the mean of the final results are presented. The obtained values of these metrics after testing ten times on the test dataset and averaging these ten values are shown in Table 4.
To provide a better comparison of the performance indicators of selected ML algorithms, such as precision, recall, F-measure, and ROC, a confusion matrix was used. After calculating the confusion matrix we concluded RF and SVM are the most effective approaches for the classification of CAD data. Figure 2 depicts the ROC and other metrics for selected ML algorithms.
In order to accurately evaluate chosen ML algorithms, it is better to use more accurate metrics. Therefore, correctly classified instances (CCI) and incorrectly classified instances (ICI) were used in this study. Therefore, Figure 3 generally shows the amount of correctly detected and incorrectly detected samples based on different algorithms.
Two of the factors used to compare different ML algorithms are correctly classified instances (CCI) and incorrectly classified instances (ICI). In many studies, these metrics determine the performance of ML algorithms . Based on the two indicators, CCI and ICI, SVM and RF algorithms had the highest efficiency among the run algorithms (Figure 2). Selected ML models employ the confusion matrix as a performance evaluation metric. The confusion matrix of the model evaluation on the test data is shown in Figure 4, which offers a clearer insight of the outcomes.
ROC is an important index that includes a range of values for receiver operating characteristics (ROC). This term is used in signal detection to describe the tradeoff between hit rate and false alarm rate when the channel is noisy. ROC curves illustrate the performance of a classifier without regard to class distribution or error costs. They plot the true positive rate against the true negative rate. In Figure 5, we show the ROC diagram for both RF and SVM algorithms. Based on this diagram, it appears that these two algorithms have produced extraordinary results.
Each algorithm has specific parameters and settings. In this study, algorithms have been recorded to maintain the validity and reliability of the parameters and settings. In Table 5 the parameter value is presented. The selected algorithms were manually tuned with different parameters in order to improve their efficiency and minimize their errors. By varying these values, the speed of convergence, the learning steps, and the categories of feeding data to the models changed, resulting in the models reaching the most optimal state and the least error.
The algorithms were implemented and run using Weka v 3.5.9 software (The University of Waikato, Hamilton, New Zealand) and the results were presented to select the optimal algorithm (s) from the analysis in the form of graphs and comparison tables. In order to increase the efficiency of the selected algorithm (s), the properties of the algorithms, which are shown in Table 5, were manually adjusted and changed.
5. Discussion and Conclusion
Many studies have been conducted on CAD using ML algorithms, and significant results have been obtained in recent years. Many of these studies used paraclinical data to analyze CAD, some of which also used heart tissue imaging data. Using clinical examination criteria to design a CAD diagnosis model drastically reduces the cost and time of the diagnosis process. According to the findings of this study, when the CAD diagnostic model is used in a specific instance, the chance of its correctness is close to 90%. Therefore, the cardiologist can employ the ML models as an additional diagnostic tool to get a definitive determination.
ML algorithms have played a pivotal role in diagnosing coronary artery disease. It can become the basis for developing clinical decision support systems. SVM and RF algorithms had the highest efficiency and could diagnose CAD based on patient examination data. It is suggested that further studies be performed using these algorithms to diagnose coronary artery disease to obtain more accurate results. More precise disease prediction tools will be required to avoid coronary artery disease. Imaging methods, such as echo and esophageal echo, can play a decisive role in diagnosing CAD disease. Therefore, a combination of clinical data and cardiac imaging data and the use of newer artificial intelligence methods such as deep learning as a powerful tool can play an important role in predicting the occurrence of CAD.
Such ML models could be used with more extensive data, such as ECG features and other data, and examination data can help specialists detect coronary artery disease more correctly. Demographic data play a significant role in implementing ML-based models for disease prediction and diagnosis. According to the results, it can be claimed that variables such as age, sex, patient weight, BMI, and FH are effective in the initial diagnosis of CAD. So, it is suggested that these predictor variables for the diagnosis of coronary artery disease and patients’ clinical examination variables should be used in studies based on ML. In other words, these variables, along with the variables of patients’ clinical examinations, are somewhat indicative of the disease status and can help the physician make an accurate diagnosis. It seems that using a more comprehensive data set can increase the accuracy of patient prediction and diagnosis models. Because additional data can lead to greater pattern extraction and, consequently, a better understanding of the data's complexity in learning-based models; therefore, one of the limits of researching the limited amount of the dataset was employed because using big volume data will be one of the influential variables in achieving models with better accuracy and precision. Future research should take advantage of a more comprehensive data collection. Another drawback of the study was that the factors were limited to clinical examination and patient identifying information. Tests and ECGs are instrumental in detecting CAD; nonetheless, the variables were chosen based on the study’s aims.
ML algorithms have played a pivotal role in diagnosing coronary artery disease. It can become the basis for developing clinical decision support systems. SVM and RF algorithms had the highest efficiency and could diagnose CAD based on patient examination data. It is suggested that further studies be performed using these algorithms to diagnose coronary artery disease to obtain more accurate results. More precise disease prediction tools will be required to avoid coronary artery disease. Imaging methods such as echo and esophageal echo can play a decisive role in diagnosing CAD disease. Therefore, a combination of clinical and cardiac imaging data and the use of newer artificial intelligence methods such as deep learning as a powerful tool can play an important role in predicting the occurrence of CAD.
The data used to support the findings of this study are available publicly at https://archive.ics.uci.edu/ml/datasets/Z-Alizadeh+Sani.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
S. S. Virani, A. Alonso, H. J. Aparicio et al., “American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee,” Heart disease and stroke statistics—2021 update: a report from the American Heart Association Circulation, vol. 143, no. 8, pp. e254–e743, 2021.View at: Publisher Site | Google Scholar
Y. Kumar, A. Koul, P. S. Sisodia, J. Shafi, V. Kavita, and M. Gheisari, “Heart failure detection using quantum-enhanced machine learning and traditional machine learning techniques for internet of artificially intelligent medical things,” Wireless Communications and Mobile Computing, vol. 2021, 9 pages, 2021.View at: Google Scholar
X. Teng and Y. Gong, “Research on application of machine learning in data mining,” IOP Conference Series: Materials Science and Engineering, IOP Publishing, Article ID 62202, 2018.View at: Google Scholar
E. Alpaydin, Introduction to Machine Learning, MIT press, Cambridge, Massachusetts, 2020.
M. Ghaderzadeh, F. Sadoughi, and A. Ketabat, “Designing a Clinical Decision Support System Based on Artificial Neural Network for Early Detection of Prostate Cancer and Differentiation from Benign Prostatic Hyperplasia,” Stud Health Technol Inform, vol. 2013, pp. 192–928, 2012.View at: Google Scholar
M. Ghaderzadeh, M. A. Eshraghi, F. Asadi, A. Hosseini, R. Jafari, and D. Bashash, “Efficient Framework for Detection of COVID-19 Omicron and Delta Variants Based on Two Intelligent Phases of CNN Models,” Comput Math Methods Med, vol. 2022, Article ID 4838009, 2022.View at: Publisher Site | Google Scholar
M. Ghaderzadeh, M. Aria, A. Hosseini, F. Asadi, D. Bashash, and H. Abolghasemi, “A fast and efficient CNN model for B‐ALL diagnosis and its subtypes classification using peripheral blood smear images,” International Journal of Intelligent Systems, 2021.View at: Google Scholar
J. Iong and Z. Chen, “Early prediction of coronary artery disease (CAD) by machine learning method - a comparative study,” BMC Public Health, vol. 19, no. 1, 2021.View at: Google Scholar
X. Chen, Y. Fu, J. Lin, Y. Ji, and Y. Fang, applied sciences Coronary Artery Disease Detection by Machine Learning with Coronary Bifurcation Features, 2020.
C. Ricciardi, R. Cuocolo, R. Megna, M. Cesarelli, and M. Petretta, “Machine learning analysis: general features, requirements and cardiovascular applications,” Minerva Cardiology and Angiology, vol. 70, no. 1, pp. 67–74, 2022.View at: Google Scholar
H. Ramchoun, Y. Ghanou, M. Ettaouil, and M. A. Janati Idrissi, “Multilayer Perceptron: Architecture Optimization and Training,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, no. 1, pp. 26–30, 2016.View at: Google Scholar
M. Ghaderzadeh, “Clinical decision support system for early detection of prostate cancer from benign hyperplasia of prostate,” Studies in Health Technology and Informatics, vol. 192, p. 928, 2013.View at: Google Scholar
A. Christmann and I. Steinwart, Support Vector Machines, 2008.
M. Ghaderzadeh, F. Asadi, R. Jafari, D. Bashash, H. Abolghasemi, and M. Aria, “Deep convolutional neural network–based computer-aided detection system for COVID-19 using multiple lung scans: design and implementation study,” Journal of Medical Internet Research, vol. 23, no. 4, Article ID e27468, 2021.View at: Publisher Site | Google Scholar
M. L. Zhang and Z. H. Zhou, “A k-nearest neighbor based algorithm for multi-label classification,” in Proceedings of the 2005 IEEE International Conference on Granular Computing, pp. 718–721, IEEE, Manhattan, New York, 2005.View at: Google Scholar
K. P. Murphy, “Naive bayes classifiers,” Univ Br Columbia, vol. 18, no. 60, pp. 1–8, 2006.View at: Google Scholar
B. E. V. Comendador, L. W. Rabago, and B. T. Tanguilig, “An educational model based on Knowledge Discovery in Databases (KDD) to predict learner’s behavior using classification techniques,” in Proceedings of the 2016 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), pp. 1–6, IEEE, Manhattan, New York, 2016.View at: Google Scholar