Abstract

Background. Severe acute pancreatitis (SAP) can cause various complications. Septic shock is a relatively common and serious complication that causes uncontrolled systemic inflammatory response syndrome, which is one of the main causes of death. This study aimed to develop a nomogram for predicting the overall survival of SAP patients during the initial 24 hours following admission. Materials and Methods. All the data utilized in this study were obtained from the MIMIC-III (Medical Information Mart for Intensive Care III) database. The data were analyzed using multivariate Cox regression, and the performance of the proposed nomogram was evaluated based on Harrell’s concordance index (C-index) and the area under the receiver operating characteristic curve (AUC). The clinical value of the prediction model was tested using decision-curve analysis (DCA). The primary outcomes were 28-day, 60-day, and 90-day mortality rates. Results. The 850 patients included in the analysis comprised 595 in the training cohort and 255 in the validation cohort. The training cohort consisted of 353 (59.3%) males and 242 (40.7%) females with SAP. Multivariate Cox regression showed that weight, sex, insurance status, explicit sepsis, SAPSII score, Elixhauser score, bilirubin, anion gap, creatinine, hematocrit, hemoglobin, RDW, SPO2, and respiratory rate were independent prognostic factors for the survival of SAP patients admitted to an intensive care unit. The predicted values were compared using C-indexes, calibration plots, integrated discrimination improvement, net reclassification improvement, and DCA. Conclusions. We have identified some important demographic and laboratory parameters related to the prognosis of patients with SAP and have used them to establish a more accurate and convenient nomogram for evaluating their 28-day, 60-day, and 90-day mortality rates.

1. Background

Acute pancreatitis (AP) is a clinically common condition that is characterized by sudden inflammation of the pancreas and the release of a series of digestive enzymes, leading to local tissue damage and multiple organ dysfunction syndrome (MODS) [1]. It is a critical disease with a high fatality rate, and especially in severe acute pancreatitis (SAP) the tissues release various factors including activated trypsin, a large number of cytokines, and inflammatory mediators, which aggravate the inflammatory necrosis of the pancreas and can even involve lesions in adjacent organs and tissues. Approximately 25% of AP patients develop SAP, and the mortality rate has been reported to vary from 10% to 60% [2, 3]. SAP can cause various complications, among which septic shock is a relatively common and serious one, causing uncontrolled systemic inflammatory response syndrome, which is one of the main causes of death [4]. Early effective treatment can improve the prognosis of patients [5]. Literature reviews have found that factors such as age, RDW, WBC, albumin, ALT, MODS, and SPO2 are significantly associated with a higher risk of death in SAP patients [6, 7].

Effective diagnostic methods and optimal strategies for SAP have not yet been determined, which requires the development of a severity assessment model to stratify patients at risk [8]. Although the existing critical-care scoring tools are designed for clinical use, there is no effective and rapidly calculated bedside prognostic scoring tool for predicting the prognosis of SAP patients in general intensive care units (ICUs). Thus, knowledge of the prognostic factors for SAP may play a critical role in helping clinicians identify high-risk patients and make better medical judgments.

A nomogram is a convenient mathematical tool for predicting certain endpoints, such as disease progression or death [9], based on several various key parameters. It is used to calculate the probability of a clinical event based on the prognostic weights of multiple factors [10]. A nomogram is a powerful and easy-to-use clinical decision-maker that can predict the outcomes in individual patients, thus benefiting both patients and clinicians alike [11].

The main aim of this study was to integrate multiple independent risk factors of SAP. These risk factors come from the Medical Information Mart for Intensive Care III (MIMIC-III) database, and they were used to establish a prognostic nomogram for improving predictions of the overall survival of SAP patients.

We present the following article/case in accordance with the CONSORT reporting checklist.

2. Methods

2.1. Data Source

All data in this study were obtained from the MIMIC-III database, which includes approximately 60,000 pieces of unconfirmed health-related data from the ICU of the Beth Israel Deaconess Medical Center (BIDMC) in the United States [12, 13]. All of the subjects had been admitted to the ICU of BIDMC between 2001 and 2012 [14]. Researchers around the world can use the data in the MIMIC-III database for free as long as they pass the assessment established by this database and obtain the data-use protocol [15, 16].

We obtained permission to access the database after completing the online training course of the National Health Protection Human Research Institute (certification number: 38292153).

2.2. Patients and Inclusion Criteria

Code 577.0 of the ICD-9 specification was used to identify 961 patients who were admitted to an ICU with AP for the first time [17]. We excluded patients with no major laboratory test results or ICU admission severity score within the first 24 hours and also patients below 18 years of age. Finally, 850 people who met the inclusion criteria were selected. The procedure of data selection according to the criteria mentioned above is presented in Figure 1.

We identified the all-cause death rates at 28 days, 60 days, and 90 days after ICU admission as the primary outcomes of our study. We randomly selected 70% of the eligible patients as the training cohort and used the remaining 30% of patients to independently validate the data.

2.3. Data Extraction and Management

Navicat Premium software was used to extract the raw data at 24 hours after ICU admission, and R software (version 3.6.1) was used for further data processing. This software supports MIMIC-III documents and other websites that are publicly available [18]. The code used to generate the descriptive statistics can be accessed at https://github.com/MIT-LCP/mimic-code/tree/master/concepts [12].

The data collected within 24 hours of ICU admission included demographics, vital signs (e.g., mean blood pressure, temperature, SPO2, respiratory rate, and heart rate), laboratory tests (e.g., RDW, chloride, glucose, sodium, and WBC), severity scores on scoring systems, and comorbidities. Demographic characteristics including age, sex, race, insurance status, marital status, readmission records, and survival time were collected from the original data set. The baseline characteristics, severity scores on scoring systems, and Elixhauser score of the patients were calculated as described previously [1921]. The primary endpoints of this study were the mortality rates at 28 days, 60 days, and 90 days from the date of ICU admission.

2.4. Statistical Analysis

Continuous variables conforming to a normal distribution are presented as mean ± SD or median and interquartile-range (IQR) values, while categorical variables are presented as frequencies and proportions. Continuous variables were compared using the t-test or Wilcoxon rank-sum test, as appropriate, while categorical variables were compared using χ2 or Fisher’s exact tests. Backward stepwise selection was applied in a Cox regression model to select variables in the training cohort. The prognostic factors that were identified as being significant were used to construct a nomogram for predicting the 28-day, 60-day, and 90-day survival rates of AP patients.

Harrell’s concordance index (C-index) and the area under the receiver operating characteristic curve (AUC) were used to evaluate the predictive accuracy of the constructed nomogram. A calibration curve was used to evaluate the consistency between the predicted probabilities and the actual outcomes [15]. For the new prediction model, the net reclassification improvement (NRI) was used to compare the accuracy of two models, while the integrated discrimination improvement (IDI) was used to determine the effectiveness of the improvements. The clinical value of the prediction model was tested using decision-curve analysis (DCA).

We used R software (version 3.6.1, CRAN) and SPSS (version 24.0, Chicago, IL) for the statistical analyses. Probability values of in two-sided tests were regarded as being statistically significant.

3. Results

3.1. Baseline Characteristics of Patients

The study included 850 eligible AP patients: 595 in the training cohort and 255 in the validation cohort. The training cohort consisted of 353 (59.3%) males and 242 (40.7%) females with AP with a median age of 60 years (IQR = 48–71 years). The validation cohort comprised 130 (51.0%) males and 125 (49.0%) females with a median age of 59 years (IQR = 45–71 years). Most of the patients in both cohorts were white (>65%), male (>50%), and married (>60%) and had Medicare (46.1%) or private (36.0%) insurance. Most patients did not have an infection or MODS.

Only the sex distribution and glucose differed significantly between the training and validation cohorts. The baseline clinicopathological data were similar in the training and validation cohorts, as indicated in Table 1. The median length of stay in the ICU was 6 days (IQR = 0–102 days). Long-term outcome data were available for all 850 patients: the 28-day, 60-day, and 90-day mortality rates were 12.9% (n = 110), 18.7% (n = 159), and 53.2% (n = 452), respectively.

3.2. Prognostic Factors for 28-Day, 60-Day, and 90-Day Mortality

Univariate analyses revealed that the significant variables were weight, sex, insurance status, explicit sepsis, SAPSII score, Elixhauser score, bilirubin, anion gap, creatinine, hematocrit, hemoglobin, RDW, SPO2, heart rate, and respiratory rate, which were included in multivariate Cox regression analyses. The multivariate analyses showed that the positive factors for survival were weight (HR = 0.989, ), being female (HR = 0.728 versus male, ), having private insurance (HR = 0.496 versus Medicare, ), having Medicaid (HR = 0.674 versus Medicare, P = 0.096), having government insurance (HR = 0.269 versus Medicare, ), creatinine (HR = 0.820, ), hemoglobin (HR = 0.678, ), SPO2 (HR = 0.938, ), and heart rate (HR = 0.986 ). In addition, the risk factors affecting survival were explicit sepsis (HR = 2.052 versus without explicit sepsis, ), SAPSII score (HR = 1.037, ), Elixhauser score (HR = 1.024, P = 0.002), bilirubin (HR = 1.043, ), anion gap (HR = 1.038, ), hematocrit (HR = 1.116, P = 0.001), RDW (HR = 1.148, ), and respiratory rate (HR = 1.045, ).

3.3. Prognostic Nomogram for 28-Day, 60-Day and 90-Day Mortality

The results of the multivariate regression model presented in Table 2 were used to establish a nomogram (Figure 2). The nomogram contained all of important independent factors predicting 28-day, 60-day, and 90-day mortality in the training cohort. The nomogram indicates that hemoglobin is the most important factor affecting prognosis, and it also includes sex, insurance status, explicit sepsis, Elixhauser score, weight, bilirubin, anion gap, creatinine, hematocrit, hemoglobin, RDW, heart rate, respiratory rate, and SPO2.

3.4. Performance and Clinical Usefulness of the Nomogram

The C-index analysis of the training cohort indicated that the nomogram provided high 28-day, 60-day, and 90-day survival C-indexes of 0.705, 0.713, and 0.720, respectively. The C-indexes of the nomogram were similarly high in the internal validation cohort, at 0.722, 0.737, and 0.751, indicating the good discriminative ability of the model (Figure 3). All of the C-indexes exceed 0.700, and the calibration curve has good consistency with the 45-degree ideal line (Figure 4). The DCA curves in Figure 5 display the large net benefits of the new model in predicting survival at 28 days, 60 days, and 90 days.

3.5. Predictive Accuracy of the Nomogram

The NRI values at the 28-day, 60-day, and 90-day follow-ups were 0.501, 0.704, and 0.732, respectively, in the training cohort and 0.170, 0.299, and 0.314, respectively, in the validation cohort. The results show that the new model has better prediction performance than the SAPSII model. Moreover, the IDI values at the 28-day, 60-day, and 90-day follow-ups were 0.084, 0.107, and 0.118, respectively, in the training cohort, and 0.041, 0.077, and 0.085, respectively, in the validation cohort.

4. Discussion

AP is a sudden inflammatory process of the pancreas with a mortality rate of 10%. The severity of AP ranges from a mild self-limiting disease to systemic complications and MODS. Up to 25% of SAP patients become critically ill, with a mortality rate of 20–30% [22, 23]. Moreover, the clinical symptoms vary with the body weight, with lighter people experiencing pancreas edema that manifests as abdominal pain, diarrhea, and vomiting, whereas in heavier people the pancreas exhibits necrosis, sepsis, or hemorrhage, resulting in shock and peritonitis, which is a dangerous condition with even higher mortality [24]. The onset age is between 20 years and 50 years, and it is more common in females than in males. Bleeding and necrosis are the most serious findings in AP, since they can readily cause peritonitis. Some patients will experience MODS or even sudden death.

The current clinical evaluation indicators for the severity and prognosis of acute pancreatitis are Ranson score [25] and SAPS score [26, 27]. Ranson score requires 48 h of inpatient observation, thereby resulting in delayed triage and management [28, 29]. The SAPSII score involves 17 clinical variables. The above two scoring systems have the characteristics of too many variables involved, being expensive, complex calculation process, and poor feasibility [30]. It is therefore necessary to establish a model for the prognosis that is simple to use and more accurate than the traditional SAP scoring system.

This study used the publicly available MIMIC-III database [12], which contains detailed information on 38,161 patients, including their vital signs, severity scores on scoring systems, laboratory tests, and diagnostic information and so can be used to obtain adequate clinical information about patients.

A nomogram is a prediction tool in the form of a simple chart based on statistical forecasting models. The prognostic weight of each factor is taken into account when calculating the probability of a particular clinical event [10]. The simplicity of the diagrammatic form of a nomogram additionally helps clinicians to provide patients and their families with effective consultations about risks, allowing them to make objective decisions. The present study established a nomogram for predicting the 28-day, 60-day, and 90-day mortality rates for AP based on the MIMIC-III database.

Our nomogram model is based on readily available clinical factors. We compared the prognostic model with the traditional APSIII scoring system using several commonly used model validation parameters: C-index, calibration curves, NRI, IDI, and DCA curve.

The multivariate Cox regression performed in this study showed that weight, sex, insurance status, explicit sepsis, SAPSII score, Elixhauser score, bilirubin, anion gap, creatinine, hematocrit, hemoglobin, RDW, SPO2, and respiratory rate are independent prognostic factors for the survival of SAP patients admitted to ICUs. The influence of overweight and obesity on the severity of AP has been controversial for a long time [3134]. Interestingly, current research shows that greater weight reduces the risk of poor prognosis in AP patients. We believe that this may be due to the poorer physical condition of AP patients in the ICU, and patients with larger weight at the time of admission tend to have better physical nutritional status, so the long-term prognosis of patients is better. Sex differences in AP have been found in previous studies, with males being more likely to develop the disease and have a worse prognosis than females. Consistent with previous studies [35], the present study found that females have better survival rates compared to males.

A particularly interesting finding of the present research was the type of insurance for patients in the ICU also being a significant prognostic factor. Patients with private, Medicaid, or government medical insurance had a better prognosis than patients with only a self-pay plan. This is in line with patients with good insurance and financial conditions usually receiving better treatment and care and recovering more quickly.

The disease severity has a major influence on the mortality rate in AP [36], with this being higher in patients with SAP and comorbidities than in those with mild AP. This is consistent with the present study finding that SAPSII and Elixhauser scores are positively correlated with the mortality of AP patients. Moreover, patients with AP may develop sepsis as their condition worsens, and patients with concurrent sepsis often have a poor prognosis. Similarly, our study found that patients with significant sepsis were at an increased risk of poor outcomes.

Regarding laboratory indexes, we found that creatinine, hemoglobin, bilirubin, anion gap, hematocrit, RDW, and SPO2 were significant independent prognostic indicators. Previous studies have shown that serum creatinine level is a good predictor of AP, and persistently high serum creatinine levels indicate poor prognosis for AP patients [37, 38]. The reason why our research results are inconsistent with previous studies may be that, in the MIMIC-III database, the time to collect patient biochemical indicators was earlier time, and continuous monitoring results were not obtained, so it cannot accurately reflect the prognosis of long-term survival. Hemoglobin is another laboratory test used, which may reflect changes in the intravascular volume status and is taken into account in cases of cardiovascular dysfunction [39]. In our study, we found that a decrease in hemoglobin was correlated with increased mortality.

Our study suggests that elevated bilirubin levels are a risk factor for SAP. Previous studies [3] have found that when AP occurs, obstruction of the bile duct can block bile excretion, and bilirubin will be deposited in liver cells, which affects the normal metabolism of liver cells and leads to the degeneration and necrosis of liver cells and impaired liver function. This increases the bilirubin level and results in a poor prognosis. It has also been reported [40] that increased hematocrit is associated with pancreatic necrosis and organ failure. We similarly found that hematocrit is a risk factor for the prognosis in AP patients.

The relationships between the anion gap and the clinical outcomes of various diseases have been explored previously [41, 42]. The anion gap is a traditional tool for assessing the acid-base status, and most previous studies have linked it to acid-base disorders, which have a main impact on the morbidity and mortality of critically ill patients [43]. Similarly, we found that the serum anion gap is a risk factor for death in patients with SAP. RDW has traditionally been an important indicator in complete blood count testing, representing the variability of the red blood cell volume [44]. A recent systematic review [45] found that the admission RDW can be used as an independent biomarker of a high risk of death in AP patients. Our results support this conclusion. Also, a lower SPO2 contributing to a higher mortality rate is similar to previous findings [37].

The main strength of our study is that it has produced the first comprehensive nomogram for the prognosis of ICU patients with AP. The nomogram represents a further improvement over the traditional SAPSII score and includes more prognostic indicators. Nevertheless, our study also had several limitations. Firstly, we used data from a single center in the United States, which may have resulted in selection bias and thus limits the applicability of our findings to other regions. However, performing the study in a single center increased the likelihood of the patient treatments being more consistent. Secondly, we only collected the laboratory parameters that were measured when a patient entered the ICU, and there were no laboratory follow-up data. It is possible that measurement data were incorrectly classified, which may have affected the results of the study. Finally, because the data in the MIMIC-III database are relatively old, the database could only be used for internal validation. Therefore, in future research we need to conduct external verification based on our own data to further verify the performance and accuracy of the new nomogram [46].

5. Conclusions

This study identified important demographic and laboratory parameters related to the prognosis of patients with SAP and used this information to establish a more accurate and convenient nomogram for evaluating the prognosis. The new nomogram clearly shows the 28-day, 60-day, and 90-day all-cause mortality rates for SAP patients in ICUs. The prognostic value of the single SAPSII scoring system was found to be inferior to that of the novel nomogram. Applying our new nomogram in the clinical care environment can help doctors who are making treatment and management decisions about SAP. However, larger prospective studies with longer follow-up times are needed to further confirm the present findings.

Abbreviations

SAP:Severe acute pancreatitis
MIMIC-III:Medical Information Mart for Intensive Care III
C-index:Harrell’s concordance index
AUC:Area under the receiver operating characteristic curve
DCA:Decision-curve analysis
NRI:Net reclassification improvement
IDI:Integrated discrimination improvement
AP:Acute pancreatitis
MODS:Multiple organ dysfunction syndrome
ICUs:Intensive care units
IQR:Interquartile range
SAPSII:Simplified Acute Physiology Score II
LOS:Length of stay
Vent:Mechanical ventilation
CHF:Chronic heart failure
RDW:Red blood cell distribution width
WBC:White blood cell
MBP:Mean blood pressure.

Data Availability

The data are available at https://physionet.org/content/mimiciii-demo/1.4/.

Ethical Approval

The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The MIMIC III database has received ethical approval from the institutional review boards (IRBs) at BIDMC and MIT, and, because the database does not contain protected health information, a waiver of the requirement for informed consent was included in the IRB approval.

Disclosure

This manuscript is available in a preprint version (https://www.Researchsquare.com/article/rs-135817/v1) [46]. The authors have completed the Observational Studies in Epidemiology Reporting checklist.

Conflicts of Interest

All the authors have completed the ICMJE uniform disclosure form. The authors have no conflicts of interest to declare.

Authors’ Contributions

Didi Han and Luming Zhang contributed to the study’s conception and design. Fengshuo Xu provided administrative support. Chengzhuo Li and Zichen Wang provided study materials or patients. Rui Yang contributed to collection and assembly of data. Shuai Zheng and Zichen Wang performed data analysis and interpretation. All the authors wrote and approved the manuscript.