Table of Contents Author Guidelines Submit a Manuscript
BioMed Research International
Volume 2015, Article ID 169870, 6 pages
Research Article

Predicting 30-Day Readmissions: Performance of the LACE Index Compared with a Regression Model among General Medicine Patients in Singapore

1Department of Family Medicine & Continuing Care, Singapore General Hospital, Academia Level 4, 20 College Road, Singapore 169856
2Department of Emergency Medicine, Singapore General Hospital, Outram Road, Singapore 169608
3Health Services and Systems Research, Duke-NUS Graduate Medical School, Singapore
4Integrated Health Information Systems, Singapore
5Department of Rheumatology & Immunology, Singapore General Hospital, Singapore
6Centre for Quantitative Medicine, Duke-NUS Graduate Medical School, Singapore

Received 30 August 2015; Accepted 3 November 2015

Academic Editor: Fabrizio Montecucco

Copyright © 2015 Lian Leng Low et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


The LACE index (length of stay, acuity of admission, Charlson comorbidity index, CCI, and number of emergency department visits in preceding 6 months) derived in Canada is simple and may have clinical utility in Singapore to predict readmission risk. We compared the performance of the LACE index with a derived model in identifying 30-day readmissions from a population of general medicine patients in Singapore. Additional variables include patient demographics, comorbidities, clinical and laboratory variables during the index admission, and prior healthcare utilization in the preceding year. 5,862 patients were analysed and 572 patients (9.8%) were readmitted in the 30 days following discharge. Age, CCI, count of surgical procedures during index admission, white cell count, serum albumin, and number of emergency department visits in previous 6 months were significantly associated with 30-day readmission risk. The final logistic regression model had fair discriminative ability c-statistic of 0.650 while the LACE index achieved c-statistic of 0.628 in predicting 30-day readmissions. Our derived model has the advantage of being available early in the admission to identify patients at high risk of readmission for interventions. Additional factors predicting readmission risk and machine learning techniques should be considered to improve model performance.

1. Introduction

Patients at high risk of readmission are a strain to healthcare system and contribute to bed shortage. Healthcare resources are finite and such frequent readmissions can overwhelm even developed health systems. Patients with frequent admissions also experience significant psychological stress and financial burden. In the United States, 30-day readmissions are considered an accountability measure and quality indicator, and the Centers for Medicare and Medicaid Services (CMS) adjusts reimbursements to hospitals according to readmission rates.

Reducing readmissions is a priority of the Ministry of Health (MOH), Singapore. Singapore’s population is one of the most rapidly ageing in Asia and faces the challenge of healthcare demand exceeding supply. An estimated one million or 20% of the country’s population will be elderly by 2030 [1]. The all-cause 30-day readmission rate in Singapore in 2010 was 11.6% [2], rising to 19.0% in those aged 65 and older. This rate is only slightly lower than in the United States [3]. The gap between the lesser developed community care and the high specialized acute care services is widely acknowledged. Transitional care programs are started by acute care hospitals to ensure a safe transition of patients from hospital to home. Currently, the planning for transitional care interventions by various hospitals is challenging as there is a lack of understanding on the prevalence of patients at various risk strata for readmission. Without accurate stratification, there could be under- or overdevelopment of the relevant transitional care programs that are meant to target patients of a certain risk stratum.

The reasons behind readmission are many and can range from unresolved medical issues and poor transition of care to unanticipated new problem in the postdischarge period. Interventions to mitigate this readmission risk are resource intensive and should be targeted at patients at high risk of readmission. A model to predict patients at risk of readmission will help healthcare administrators and providers to allocate appropriate resources to patients at highest risk of readmission.

Several countries have developed predictive models for readmission, for example, LACE and LACE+ index [4, 5] in Canada, PARR-30 [6] by the UK National Health System, and the HOSPITAL score in the United States [7]. Many of these models have limited generalizability to other health systems due to the unique variables derived from their respective specific settings [6, 8, 9]. The LACE index (length of stay, acuity of admission, Charlson comorbidity index, and emergency department visits in past 6 months), derived in Ontario, Canada (Table 1), is simple to use and may have potential clinical utility in Singapore. However, the LACE index was developed from middle-aged Canadian patients free of serious comorbidities, and its applicability to another population needs to be validated before clinical utility. Tan et al. [10] found that medical patients in Singapore with a LACE score ≥ 10 had an almost 5 times higher risk of 30-day unplanned readmission after index discharge. But the LACE index can only be calculated on discharge and performance of the LACE index was not validated in our population. A simple predictive model using variables available early in the admission will identify patients to receive intensive interventions to reduce their readmission risk.

Table 1: Components of the LACE index.

Our primary objective is to validate the LACE index and compare its performance with a derived regression model to predict 30-day readmission risk among general medicine patients in Singapore.

2. Materials and Methods

2.1. Setting and Study Population

Singapore is a multiethnic Asian country with a population of 5.6 million. The public healthcare system is divided into six regional health systems, each responsible for a specific geographical region. Singapore General Hospital (SGH) is the flagship hospital of the SingHealth regional health system and the largest tertiary hospital in Singapore with 37 clinical specialities and 88,000 inpatient admissions each year [11].

All adult patients ≥21 years, with alive-discharge episodes from the Department of Internal Medicine, SGH, in 2012 were eligible for inclusion. Internal Medicine is the general medical service and the largest medical speciality in SGH that discharges about 18% of all hospitalized patients per year. Patients who died during the index admission, who were nonresidents, or who had a discharge destination other than home at index discharge were excluded from analysis. The first admission in 2012 is the index admission and we counted no more than one readmission for each internal medicine patient discharged in 2012.

2.2. Data Collection, Variables, and Outcome Measures

We extracted deidentified data from the electronic health intelligence system (eHINTS) of SGH using the Oracle Business Intelligence and Enterprise Edition (OBIEE) Software. The eHINTS consolidates and analyses patient and healthcare data that are uploaded on the web-based business intelligence software. Variables were chosen a priori and according to medical literature. These variables are capable of extraction from the hospital’s electronic health records (EHR) and available to all hospitals in Singapore, with potential for external validation of our model. The comorbidities were identified using International Classification of Diseases (ICD-10) codes in any primary or secondary diagnosis fields dating back to one year preceding the index admission. The Charlson comorbidity index (CCI) was computed for each patient.

In addition to the four variables in the LACE index, variables extracted included patient demographics (age, gender, marital status, race, and admission ward class as a marker of socioeconomic status), comorbidities (major diseases listed under the Charlson comorbidity index, number of medications on discharge), clinical and laboratory variables during the index admission (haemoglobin, white cell count, c-reactive protein, serum sodium, serum creatinine, serum albumin, and number of surgical procedures), and prior healthcare utilization in the preceding year (number of admissions, number of emergency department visits, and number of specialist outpatient clinic visits).

2.3. Statistical Analysis

The demographics and baseline characteristics of the study population were described and compared among patients with and without meeting the outcome (30-day readmission). Continuous variables were presented as means (standard deviation) by using Student’s -test in the analysis or medians (interquartile range) by using Wilcoxon rank-sum test. Categorical variables were analysed with chi-square test or Fisher’s exact test when appropriate and were presented as numbers (percentage). Statistical significance was set as .

Significant variables (e.g., age, length of stay, Charlson comorbidity index, LACE score, emergency department visits in previous 6 months, admission ward class, and laboratory variables) selected from univariate analysis were associated with 30-day readmissions using multivariate logistic regression where stepwise forward variable selection was implemented to build the prediction model. In the final regression model, we only included variables that would be available early in the admission for the model to be useful to clinicians to identify their patients at high risk of readmission. The receiver operating characteristic (ROC) analysis was conducted to evaluate the predictive performance for both LACE index and the regression model, and c-statistic (i.e., area under the ROC curve) was reported. Data analyses were conducted in IBM SPSS Statistics 21 (IBM Corporation, Armonk, NY) and MATLAB R2014b (Mathworks, Natick, MA).

3. Results

6,377 unique patients were admitted to the internal medicine wards in 2012. Of these, 515 (8.1%) were excluded from analysis because of death during the index admission, nonresidential status, or having a discharge destination other than home at index discharge (Figure 1).

Figure 1: Study flow chart showing number of included, excluded, and readmitted patients.

Of the 5,862 (91.9%) patients remaining in the cohort, 572 patients (9.8%) were readmitted in the 30 days following discharge (Figure 1). Patient characteristics are shown in Table 2. The mean age of all patients was 60.0 () years. The majority were female (55.3%, ,241). Most admissions were emergent (99.4%, ,792). Median length of stay was 3 days (IQR 2–5). On average, patients had a median LACE score of 6 (IQR 5–8). The readmitted patients were older compared to the nonreadmitted patients and had significantly longer length of stay during the index admission, higher CCI, more ED admissions in the preceding 6 months, and higher LACE scores (Table 2).

Table 2: Patient characteristics and their association with readmission within 30 days.

In a multivariate logistic regression, six variables were found to be significantly associated with readmission within 30 days of discharge (Table 3). Five were included in the final model: age, CCI, white cell count, serum albumin, and emergency department visits in previous 6 months. We excluded count of surgical procedures during index admission as this information is not available early in the admission to be clinically useful. The final logistic regression model had fair discriminative ability (c-statistic of 0.650, 95% CI: 0.625–0.676) (Figure 2). As a comparison, the LACE index achieved a c-statistic of 0.628 (95% CI: 0.602–0.653) in predicting 30-day readmissions. The optimal cut-off for the LACE index is a score of 6 or more with a sensitivity of 66.3% and specificity of 53.3%. The optimal cut-off for the regression model has comparable sensitivity and specificity of 61.4% and 60.3%, respectively (Table 4).

Table 3: Variables in the final logistic regression model for prediction of 30-day readmissions.
Table 4: Sensitivity and specificity for the regression model and LACE indices greater than 10 and 6.
Figure 2: Comparison of the receiver operating characteristic (ROC) curves for the LACE score and the derived logistic regression model.

4. Discussion

We were able to identify key patient-level predictors of readmission and derive and internally validate a model for assessing readmission risk in general medicine patients hospitalized for a variety of medical conditions and discharged home. Several patient-level factors identified as significant predictors were known from the published literature, such as age, the Charlson comorbidity index [4], and number of emergency department visits in the preceding 6 months. The number of surgical procedures, white cell count, and serum albumin levels are clinical markers of patients’ condition during the index admission and the frailty of the patients [12], respectively. In contrast to the LACE index, length of stay and acuity of admission were not associated with risk of readmission within 30 days after adjusting for covariates in the multivariate logistic regression model. It is possible that, in our cohort, the duration of admission was affected by other factors such as social circumstances and did not reflect the severity of illness entirely. Almost all admissions were emergent and reduced its predictive ability for readmission risk.

The discriminative ability of our model is only fair but comparable to the discriminative ability of other commonly cited readmission risk prediction models [4, 6, 13, 14]. It is likely that important predictors of readmission such as level of social support, function, and financial payment modes, for example, insurance, government welfare, or self-payment, affect patients’ healthcare utilization patterns [15] but are not available from our administrative database. A model that incorporated these variables with eight interaction terms had a c-statistic of 0.83 but may be difficult to deploy clinically due to its complexity and possible overfitting for the population used to derive it [16].

When we applied the LACE index to our cohort of general medicine patients, we found that the LACE index had poorer discriminative ability (c-statistic 0.628), compared to our derived model (c-statistic of 0.650) and the Canadian population from which it was derived (c-statistic of 0.69). This poorer performance in our cohort is consistent in other populations that validated the LACE index [17]. The difference in performance is not surprising, as the characteristics of the cohort enrolled in the Canadian study were of a lower risk profile and differed from those of the usual patients who attend general medicine wards in Singapore. Patients in the Canadian cohort were mainly free of serious comorbidities, while our general medicine cohort had higher CCI scores, more medications dispensed on discharge, and greater utilization of hospital services preceding the index admission. When we evaluated the discriminative ability of the LACE index in our cohort stratified by age, the performance of LACE in older patients ≥ 65 years dipped to c-statistic 0.59. This is consistent with the performance of LACE in an older UK and Denmark population [17, 18].

The fair discriminative ability of both the LACE index and our derived model suggests much room for improvement to the model. Future models should consider more clinical, social, and functional variables for inclusion but need to be balanced against the costs of collecting these additional risk factors prospectively. The SingHealth regional health system has developed its enterprise analytic platform featuring a single enterprise data repository that integrates information from multiple healthcare transactional systems including applications from administration and clinical and ancillary services [19]. The quality information is hosted on user-friendly web-based Business Intelligence/Dashboard front-end software, and can be easily accessed online by researchers and healthcare providers. Analytical tools with “self-service” drill down capabilities enable faster and more efficient analysis. With the expansion of the data repository in our health system and health systems worldwide, advanced machine learning techniques will be needed to handle the data and to improve the performance of prediction models for various health outcomes [20, 21] and should be considered as an alternative method for model derivation.

4.1. Limitations

Our study has a number of limitations. Firstly, our prediction model was derived from administrative data, which may have coding errors. Secondly, as a retrospective study, our database is limited to variables routinely collected into the EHR. Other factors that may contribute to readmission risk such as the level of social support, functional limitations, and frailty of patients are not included in this study. Thirdly, we did not adjudicate whether each readmission was elective versus unplanned. However, based on our collective experience, we would expect the rate of elective readmissions on general medicine services to be low. Fourthly, we were unable to confirm out of hospital deaths or readmissions to hospitals other than ours, although only a minority utilized services from more than one hospital and majority of elderly die in hospitals in Singapore [22, 23]. Finally, although our study was conducted at the largest hospital in Singapore, caution should be exercised in generalizing its findings to other medical departments or other hospitals without externally validating the model.

5. Conclusions

Our derived model had comparable discriminative ability compared to the LACE index in our cohort of general medicine patients in Singapore. Our model has the advantage of being available early in the admission to identify patients at high risk of readmission to receive interventions to prevent potentially avoidable readmissions. Additional factors that predict readmission risk and advanced machine learning techniques should be considered to improve readmission models. The developing enterprise analytic platform in our health system will further build capabilities in predictive analytics in Singapore.

Conflict of Interests

The authors declare that there in no conflict of interests regarding the publication of this paper.


  1. Health Facts Singapore, Healthcare Institution Statistics, Ministry of Health Singapore, 2012,
  2. E. Lim, N. Matthew, W. Mok, S. Chowdhury, and D. Lee, “Using hospital readmission rates to track the quality of care in public hospitals in Singapore,” BMC Health Services Research, vol. 11, supplement 1, article A16, 2011. View at Publisher · View at Google Scholar
  3. S. F. Jencks, M. V. Williams, and E. A. Coleman, “Rehospitalizations among patients in the medicare fee-for-service program,” The New England Journal of Medicine, vol. 360, no. 14, pp. 1418–1428, 2009. View at Publisher · View at Google Scholar · View at Scopus
  4. C. van Walraven, I. A. Dhalla, C. Bell et al., “Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community,” CMAJ, vol. 182, no. 6, pp. 551–557, 2010. View at Publisher · View at Google Scholar · View at Scopus
  5. C. van Walraven, J. Wong, and A. J. Forster, “LACE+ index: extension of a validated index to predict early death or urgent readmission after hospital discharge using administrative data,” Open Medicine, vol. 6, no. 3, pp. e80–e90, 2012. View at Google Scholar · View at Scopus
  6. J. Billings, I. Blunt, A. Steventon, T. Georghiou, G. Lewis, and M. Bardsley, “Development of a predictive model to identify inpatients at risk of re-admission within 30 days of discharge (PARR-30),” BMJ Open, vol. 2, no. 4, Article ID e001667, 2012. View at Publisher · View at Google Scholar · View at Scopus
  7. J. Donze, D. Aujesky, D. Williams, and J. L. Schnipper, “Potentially avoidable 30-day hospital readmissions in medical patients: derivation and validation of a prediction model,” JAMA Internal Medicine, vol. 173, no. 8, pp. 632–638, 2013. View at Publisher · View at Google Scholar · View at Scopus
  8. E. Ben-Chetrit, C. Chen-Shuali, E. Zimran, G. Munter, and G. Nesher, “A simplified scoring tool for prediction of readmission in elderly patients hospitalized in internal medicine departments,” Israel Medical Association Journal, vol. 14, no. 12, pp. 752–756, 2012. View at Google Scholar · View at Scopus
  9. D. He, S. C. Mathews, A. N. Kalloo, and S. Hutfless, “Mining high-dimensional administrative claims data to predict early hospital readmissions,” Journal of the American Medical Informatics Association, vol. 21, no. 2, pp. 272–279, 2014. View at Publisher · View at Google Scholar · View at Scopus
  10. S. Y. Tan, L. L. Low, Y. Yang, and K. H. Lee, “Applicability of a previously validated readmission predictive index in medical patients in Singapore: a retrospective study,” BMC Health Services Research, vol. 13, article 366, 2013. View at Publisher · View at Google Scholar · View at Scopus
  11. Hospital Overview: Singapore General Hospital, 2015,
  12. J. Afilalo, K. P. Alexander, M. J. Mack et al., “Frailty assessment in the cardiovascular care of older adults,” Journal of the American College of Cardiology, vol. 63, no. 8, pp. 747–762, 2014. View at Publisher · View at Google Scholar · View at Scopus
  13. O. Hasan, D. O. Meltzer, S. A. Shaykevich et al., “Hospital readmission in general medicine patients: a prediction model,” Journal of General Internal Medicine, vol. 25, no. 3, pp. 211–219, 2010. View at Publisher · View at Google Scholar · View at Scopus
  14. N. L. Novotny and M. A. Anderson, “Prediction of early readmission in medical inpatients using the probability of repeated admission instrument,” Nursing Research, vol. 57, no. 6, pp. 406–415, 2008. View at Publisher · View at Google Scholar · View at Scopus
  15. E. Kroch, M. Duan, J. Martin, and R. A. Bankowitz, “Patient factors predictive of hospital readmissions within 30 days,” Journal for Healthcare Quality, In press.
  16. E. A. Coleman, S.-J. Min, A. Chomiak, and A. M. Kramer, “Posthospital care transitions: patterns, complications, and risk identification,” Health Services Research, vol. 39, no. 5, pp. 1449–1465, 2004. View at Publisher · View at Google Scholar · View at Scopus
  17. T. Cooksley, P. Nanayakkara, C. Nickel et al., “Readmissions of medical patients: an external validation of two existing prediction scores,” Quarterly Journal of Medicine, 2015. View at Publisher · View at Google Scholar
  18. P. E. Cotter, V. K. Bhalla, S. J. Wallis, and R. W. S. Biram, “Predicting readmissions: poor performance of the LACE index in an older UK population,” Age and Ageing, vol. 41, no. 6, Article ID afs073, pp. 784–789, 2012. View at Publisher · View at Google Scholar · View at Scopus
  19. Electronic Health Intelligence System (eHINTs), Integrated Health Information Systems 2012,
  20. A. G. Singal, A. Mukherjee, B. J. Elmunzer et al., “Machine learning algorithms outperform conventional regression models in predicting development of hepatocellular carcinoma,” The American Journal of Gastroenterology, vol. 108, no. 11, pp. 1723–1730, 2013. View at Publisher · View at Google Scholar · View at Scopus
  21. B. Amalakuhan, L. Kiljanek, A. Parvathaneni, M. Hester, P. Cheriyath, and D. Fischman, “A prediction model for COPD readmissions: catching up, catching our breath, and improving a national problem,” Journal of Community Hospital Internal Medicine Perspectives, vol. 2, no. 1, 2012. View at Publisher · View at Google Scholar
  22. N. Saxena, A. X. You, Z. Zhu et al., “Singapore's regional health systems—a data-driven perspective on frequent admitters and cross utilization ofhealthcare services in three systems,” The International Journal of Health Planning and Management, 2015. View at Publisher · View at Google Scholar
  23. A. K. L. Beng, C. W. Fong, E. Shum, C. R. Goh, K. T. Goh, and S. K. Chew, “Where the elderly die: the influence of socio-demographic factors and cause of death on people dying at home,” Annals of the Academy of Medicine Singapore, vol. 38, no. 8, pp. 676–683, 2009. View at Google Scholar · View at Scopus