Abstract
Introduction. The ability of risk models to predict in-hospital mortality and the influence on downstream therapeutic strategy has not been fully investigated in Chinese Non-ST-segment elevation myocardial infarction (NSTEMI) patients. Thus, we sought to validate and compare the performance of the Global Registry of Acute Coronary Events risk model (GRM) and China Acute Myocardial Infarction risk model (CRM) and investigate impacts of the two models on the selection of downstream therapeutic strategies among these patients. Methods. We identified 2587 consecutive patients with NSTEMI. The primary endpoint was in-hospital death. For each patient, the predicted mortality was calculated according to GRM and CRM, respectively. The area under the receiver operating characteristic curve (AUC), Hosmer–Lemeshow (H–L) test, and net reclassification improvement (NRI) were used to assess the performance of models. Results. In-hospital death occurred in 4.89% (126/2587) patients. Compared to GRM, CRM demonstrated a larger AUC (0.809 versus 0.752, ), less discrepancy between observed and predicted mortality (H–L χ2: 22.71 for GRM, and 10.25 for CRM, ), and positive NRI (0.3311, ), resulting in a significant change of downstream therapeutic strategy. Conclusion. In Chinese NSTEMI patients, the CRM provided a more accurate estimation for in-hospital mortality, and application of the CRM instead of the GRM changes the downstream therapeutic strategy remarkably.
1. Introduction
Non-ST-segment elevation myocardial infarction (NSTEMI) is a leading cause of mortality, morbidity, and hospitalization from cardiovascular disease both worldwide and in China, which has a major influence on health economies [1–4]. Unfortunately, lower rather than higher risk NSTEMI patients are more likely to receive more aggressive therapeutic strategies, which is the so-called risk-treatment paradox [5–7]. To diminish the impact of this paradox, the current guidelines consider the risk assessment by the Global Registry of Acute Coronary Events (GRACE) risk model as a fundamental component to select the most appropriate therapeutic strategy for NSTEMI patients [1–3]. However, several observations have suggested that the performance of the GRACE risk model (GRM) was unsatisfactory among Chinese NSTEMI patients [8, 9]. Recently, a novel risk model has been developed for the risk evaluation of NSTEMI patients [8] based on multicenter data from the China Acute Myocardial Infarction (CAMI) registry [10]. To date, the CAMI-NSTEMI risk model (CRM) has not been systematically validated in an external cohort. Moreover, it has not been fully demonstrated how these risk assessment models influenced clinical management. Thus, the present study tends to validate and compare the performance of GRM and CRM and investigate the impacts of the two proposed models on the selection of downstream therapeutic strategies in Chinese NSTEMI patients.
2. Materials and Methods
2.1. Study Participants
The details of inclusion and exclusion criteria for the study participants are illustrated in Figure 1. According to the most recent guidelines, NSTEMI was defined as symptoms of ischemia and detection of an elevation of cardiac troponin values without new persistent ST-segment elevation [1–3]. Three thousand and twenty-five patients with a primary clinical diagnosis of NSTEMI were included. Then, the patients were excluded if they fulfilled one of the following criteria: data missing (217), acute or chronic infectious diseases (116), cancer (68), and acute cerebrovascular disease (37). Among the 217 patients excluded from the analysis for missing data, medical histories for 68% (147/217), results of blood test for 28% (61/217), and clinical presentations for 24% (52/217) were not available. At last, 2587 patients were included in this study and were divided into in-hospital survival and death groups. This observational study complied with the Declaration of Helsinki and was approved by the local Ethics Committees.
2.2. Data Collection and Definitions
The following variables were registered through review of the electronic medical record: age, sex, history of diabetes mellitus, hypertension, hyperlipidemia, smoking, previous revascularization, and previous myocardial infarction. The body mass index (BMI), Killip class, heart rate (HR), systolic blood pressure (SBP), creatinine (Cr), white blood cell count (WBC), cardiac arrest, and ST-segment depression on an 18-lead electrocardiogram at admission were also recorded. Hyperlipidemia was defined as a total cholesterol of at least 220 mg/dl, low-density lipoprotein cholesterol of at least 140 mg/dl, fasting triglycerides of at least 150 mm/dl, or receiving treatment with oral lipid-lowering agents. Diabetes was defined as fasting glucose levels over 7 mmol/l or treatment currently with diet, oral glucose-lowering agents, or insulin. The smoking status included current smoker, nonsmoker, and previous smoker (quit >6 month). The primary endpoint of this study was in-hospital death, defined as all-cause death during hospitalization.
2.3. Risk Assessment by the CRM and GRM
The GRM included 8 independent risk factors: age, Killip class, SBP, ST-segment depression, cardiac arrest, Cr, initial cardiac enzyme findings, and HR [11]. The initial cardiac enzyme findings were positive because all patients have been diagnosed with NSTEMI. According to the GRM and guideline recommendations, we classified patients into 3 risk groups: the low-, medium-, and high-risk group [1]. The CRM identified 11 independent predictors of in-hospital mortality: age, BMI, SBP, Killip class, cardiac arrest, ST-segment depression, Cr, WBC smoking status, previous MI, and previous percutaneous coronary intervention, and all patients were classified into 3 risk groups based on the CRM [8].
2.4. Statistical Analysis
All statistical analyses were carried out by MedCalc (version 15.2.2; MedCalc Software, Mariakerke, Belgium) and R (version 3.2.4; R Foundation for Statistical Computing, Vienna, Austria). A two-tailed value less than 0.05 was considered statistically significant. Continuous variables were compared using Student’s t-tests or Mann–Whitney U-tests as appropriate. Count variables were assessed using the χ2-test or Fisher’s exact test as appropriate. This study conducting validation and comparison of multivariable prediction models strictly followed Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement [12]. To validate and compare the predictive value of the GRM and CRM, we used three characteristics: discrimination, calibration, and classification [13]. Discrimination refers to how well the model differentiates those having an endpoint from those not having. The area under receiver-operator characteristic curve (AUC) was used to quantify the improvement in discrimination [14]. Calibration reflects the extent to which the values predicted by the model agree with the observed values. We used Hosmer–Lemeshow (H–L) tests which divided patients into ten groups according to deciles of mortality and calculated a chi-square statistic (H–L χ2) to assess calibration [15]. As the selection of downstream therapeutic strategy usually bases on the risk classification, we established a reclassification table to evaluate the net reclassification improvement (NRI), determining how correctly a model reclassifies patients into various risk categories compared with another [16].
3. Results
Baseline characteristics of the study cohort are listed in Table 1. Overall, hospital death occurred in 4.89% (126/2587) patients. Compared to the in-hospital survival group, the in-hospital dead group were older, had more male, lower BMI and SBP, and a higher prevalence of diabetes mellitus, hypertension, current smoker, cardiac arrest, high Killip class, and ST-segment depression. Also, HR, Cr, WBC, and troponin T were higher in the in-hospital dead group. All the differences were statistically significant ().
The receiver-operator characteristic curves of 2 models are exhibited in Figure 2. The AUC for the CRM (0.809, 95% confidence interval: 0.789 to 0.829, ) was significantly () larger than that for the GRM (0.752, 95% confidence interval: 0.729 to 0.774, ).
The predicted mortality was compared with the observed mortality in deciles of predicted mortality as illustrated in Figure 3. The GRM manifested a predominance of underestimation, resulting in a poor calibration (H–L χ2 = 22.71, ). The differences between the observed and predicted mortality were not evident in H–L calibration plots for CRM, so the calibration of the CRM was good (H–L χ2 = 10.25, ).
(a)
(b)
Table 2 shows the reclassification table comparing the CRM to GRM. For the 126 positive patients, compared to the GRM, the CRM correctly reclassified 23 from the medium- to high-risk category, 12 from low to high, and 6 from low to medium, but 2 from medium to low and 2 from high to medium. Of the 2461 negative patients, 174 were correctly reclassified to a lower risk category but 82 to a higher PTP category. As a result, compared to the GRM, the NRI for the CRM was 0.2937 in positive, 0.0374 in negative, and 0.3311 overall (). In other words, the replacement of the GRM by CRM for every 2587/(23 + 12 + 6 + 174)≈13 patients would result in 1 correct reclassification.
4. Discussion
This observational analysis determined that the CRM provided a more effective prediction for in-hospital mortality in Chinese NESTEMI patients. Compared to the GRM, the CRM demonstrated a larger AUC, less discrepancy between the observed and predicted mortality, and a positive NRI. It was worth noting that the application of the CRM instead of the GRM may pronouncedly change the downstream therapeutic strategy in these NESTEMI patients.
Compared to ST-segment elevation myocardial infarction (STEMI) patients, NSTEMI patients present with more heterogeneous variation in ischemic risk and comorbidities so that the risk classification has been considered as a fundamental component to select the most appropriate therapeutic strategy for NSTEMI patients [1–3, 17]. However, in clinical practice the treatment-risk paradox is widespread [5–7], which is partly due to the suboptimal risk assessment underestimating the ischemic risk [18, 19], especially in China [4, 6, 20]. In conformity with this, according to the H–L calibration plots in the present study, the GRM dramatically underestimated the in-hospital mortality and classified nearly 40% in-hospital death into the low- or medium-risk category, which may cause underuse of invasive strategy for patients at high risk [1, 2]. Two reasons may potentially account for the unfavorable performance when applying GRM to the Chinese NSTEMI patients. First, GRM was developed in patients mainly from America, Europe and Australia twenty years ago [11]. There are significant differences in characteristics and management between these and contemporary Chinese patients [3, 11]. Second, the GRM was developed to assess the risk of patients with acute coronary syndrome, including STEMI, NSTEMI, and unstable angina pectoris [11].
To address the limitations mentioned above, the CRM was developed to predict in-hospital mortality, particularly for Chinese NSTEMI patients, and its diagnostic performance was superior to that of the GRM in two internal validation studies using data from CAMI registry [8, 9]. Our effort further extended this conclusion in an external validation cohort by indicating a larger AUC, positive NRI, and less disagreement between the observed and predicted mortality for the CRM. Moreover, the GRM classified 15% (19/126) in-hospital dead into the low-risk category, for which invasive strategy was not recommend according to current guidelines and 22% (28/126) into the medium-risk category, for which immediate (<2 h) and early (<24 h) invasive strategy were not recommend [1]. Using the CRM instead of the GRM would imply a significant change for downstream therapeutic strategy in the in-hospital dead: 87% (36/47) of these patients would be reclassified into the higher risk category, for which more aggressive strategies were recommended. Thus, application of the CRM instead of the GRM may have the potential to optimize the referral of aggressive intervention and lead to an evident diminution of the treatment-risk paradox in NSTEMI patients.
This study was subjected to the limitations of its single-center, retrospective, and observational design. The indication of clinical management and downstream therapeutic strategy were based on the individual physician decision. Thus, the strategic discordance of downstream management for NSTEMI patients should not be ignored, and the actual impact of applying the CRM was complicated. To further investigate the generalizability and reliability of CRM-guided therapeutic strategy, more pragmatic and cost-effective randomized control trials are needed, such as the UK GRACE Risk Score Intervention Study (UKGRIS) [21].
5. Conclusions
In Chinese NSTEMI patients, the CRM provided a more effective estimation for in-hospital mortality due to the improvement in discrimination, classification, and calibration compared to the GRM. The application of the CRM instead of the GRM could change the diagnostic strategy and the potential to optimize the referral of a more aggressive intervention.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This study was funded by the Key Project of Scientific and Technological Support Plan of Tianjin in 2018 (No.16ZXMJSY00140).