Administrative and Claims Data Help Predict Patient Mortality in Intensive Care Units by Logistic Regression: A Nationwide Database Study

Hsu, Yu-Ting; He, Yi-Ting; Ting, Chien-Kun; Tsou, Mei-Yung; Tang, Gau-Jun; Pu, Christy

doi:https://doi.org/10.1155/2020/9076739

BioMed Research International

On this page

Abstract Introduction Methods Results Discussion Limitations Conclusions Abbreviations Data Availability Conflicts of Interest Authors’ Contributions Acknowledgments Supplementary Materials References Copyright Related Articles

Research Article | Open Access

Volume 2020 | Article ID 9076739 | https://doi.org/10.1155/2020/9076739

Administrative and Claims Data Help Predict Patient Mortality in Intensive Care Units by Logistic Regression: A Nationwide Database Study

Yu-Ting Hsu,¹Yi-Ting He,²Chien-Kun Ting,¹Mei-Yung Tsou,¹Gau-Jun Tang,^2,3and Christy Pu²

Academic Editor: Nasimul Noman

Received19 Aug 2019

Revised14 Jan 2020

Accepted30 Jan 2020

Published26 Feb 2020

Abstract

Background. Increasing attention has been paid to the predictive power of different prognostic scoring systems for decades. In this study, we compared the abilities of three commonly used scoring systems to predict short-term and long-term mortalities, with the intention of building a better prediction model for critically ill patients. We used the data from the National Health Insurance Research Database (NHIRD) in Taiwan, which included information on patient age, comorbidities, and presence of organ failure to build a new prediction model for short-term and long-term mortalities. Methods. We retrospectively collected the medical records of patients in the intensive care unit of a regional hospital in 2012 and linked them to the claims data from the NHIRD. The Acute Physiology and Chronic Health Evaluation II (APACHE II) score, Elixhauser Comorbidity Index (ECI), and Charlson Comorbidity Index (CCI) were compared for their predictive abilities. Multiple logistic regression tests were performed, and the results were presented as receiver operating characteristic curves and C-statistic. Results. The APACHE II score has the best predictive power for inhospital mortality (0.79; ) and 1-year mortality (0.77; ). The ECI and CCI alone have poorer predictive power and need to be combined with other variables to be comparable to the APACHE II score, as predictive tools. Using CCI together with age, sex, and whether or not the patient required mechanical ventilation is estimated to have a C-statistic of 0.773 (95% CI 0.744-0.803) for inhospital mortality, 0.782 (95% CI 0.76-0.81) for 30-day mortality, and 0.78 (95% CI 0.75-0.80) for 1-year mortality. Conclusions. We present a new prognostic model that combines CCI with age, sex, and mechanical ventilation status and can predict mortality, comparable to the APACHE II score.

1. Introduction

Intensive care units (ICUs) provide crucial medical care to critically ill patients. Owing to the advances in diagnostics and therapeutics, human life expectancy has been extending, leading to increasing demand for ICU care [1]. The increase in ICU care is accompanied by a growing use of risk assessment tools, aimed at evaluating treatment [2], triage patients [3], and achieve better resource allocation [4].

In the past decades, several risk scoring systems have been introduced [5]. These include the Acute Physiology and Chronic Health Evaluation (APACHE) II score [6], Charlson Comorbidity Index (CCI) [7–9], and Elixhauser Comorbidity Index (ECI) [10, 11]. However, the predictive power of these scoring systems varies according to previous surveys, and how to choose the best system is still not clear.

This study compares the abilities of these three scoring systems to predict short-term and long-term mortalities by combining data from ICU medical records with claims data from Taiwan’s National Health Insurance Research Database (NHIRD) [12]. In this study, we have examined and evaluated the different variables using multiple logistic regression tests to identify and compare the strongest predictors. We aim to improve the predictive power of the current scoring systems and to build a new prediction model for ICU mortality.

2. Methods

2.1. Data Acquisition and Extraction

All ICU admissions () in 2012 at the National Yang-Ming University Hospital, a regional hospital in Eastern Taiwan, were identified. The study was approved by the Institutional Review Board of the National Yang-Ming University Hospital (IRB number: 2014A021). Informed consent was obtained from the patients or their guardians for all data used in this study.

The enrolled patients were admitted to our ICU between January 1 and December 31, 2012. If a patient was admitted more than once during the study period, only the first admission was included to avoid a small group of patients dominating the characteristics of the study population. As a result, 591 repeat admissions were excluded. Whether some of the remaining patients had been admitted to an ICU other than ours could not be determined because we lacked access to the medical records of other hospitals. We also excluded the following patients: (a) those who did not have a Taiwanese citizenship, as noncitizens did not have the national identification number required to link the ICU medical records to the claims and mortality data of NHIRD; (b) those who were under 20 years of age; and (c) those whose data could not be linked to NHIRD because of administrative errors (). Finally, we enrolled 1,608 patients in our study.

The ICU medical records were linked to the claims data of NHIRD from 2010 to 2013. The Taiwan NHI program is a public insurance system in which the enrollment is compulsory for Taiwanese citizens [12]. We used the patients’ national identification numbers for linkage to the NHIRD in our study. Doing so reduced the likelihood of duplication and mismatching. The medical claims for all inpatient and outpatient medical services in Taiwan are the same. The medical claim and diagnosis codes in the NHI system are used nationwide and have been validated by the NHI Administration of Taiwan [13].

2.2. Study Variables

2.2.1. Acute Physiology and Chronic Health Evaluation (APACHE) II Score

Each patient admitted to the ICU was evaluated using an APACHE II score form, which included 12 items (Table 1). These items assessed the acute physiological state, age, and chronic health conditions of the patient [6].

2.2.2. Charlson Comorbidity Index (CCI)

The CCI was calculated based on 17 disease categories [7] (Table 1). The look-back period for comorbidities was 1 year before ICU admission. A 1-year look-back period is thought to improve the ability of a model to predict posthospitalization mortality according to previous studies [14, 15]. A patient was considered to have a comorbid condition in a certain year if there were at least two claim records with an ICD-9 code for that condition during that year [15, 16]. A higher CCI score indicates a higher number of comorbidities. See Appendix 1 for the ICD-9-CM codes for the different comorbidities [11].

2.2.3. Elixhauser Comorbidity Index (ECI)

The ECI was based on 30 comorbidities [10] (Table 1). Appendix 1 lists the ICD-9-CM codes for these conditions [11].

2.2.4. Other Variables

The other variables evaluated in the study were age, sex, hemodialysis, surgery, number of outpatient and emergency department visits in the previous year, number of inpatient admissions in the previous year, admissions department, and use of a ventilator.

2.3. Outcomes

The primary outcome measured in this study was inhospital mortality, which was defined as death during the hospital stay; the censoring point was discharge from the hospital. The secondary outcomes were all-cause 30-day mortality (defined as death within 30 days after hospital discharge) and overall 1-year mortality (defined as death within 1 year after hospital discharge).

2.4. Statistical Analysis

Data were expressed as absolute numbers and percentages. The categorical variables of surviving and deceased subjects were compared using the chi-square test. To evaluate the risk of mortality, an odds ratio with a 95% confidence interval (CI) was determined for each variable via linear regression analysis. We developed regression models to identify the strongest predictors of mortality, which were then entered into multiple logistic regression models to evaluate the overall model performance and to predict the risk of mortality. We computed the areas under receiver operating characteristic (ROC) curves (AUROCs) [17, 18] as a measure of the ability of a model to predict mortality over different risk categories. The AUROC is often referred to as the concordance index number (C-statistic) and ranges from 0.5 (no discrimination) to 1.0 (perfect discrimination), with values above 0.7, 0.8, and 0.9 considered reasonable, strong, and exceptional, respectively [18].

The discrimination performance of the CCI and ECI (with/without additional variables) was compared to that of the APACHE II score, which was used as the reference model. Differences in the AUROC between the fitted models were analyzed. Statistical analysis was performed using the SPSS statistical software (SPSS Inc., Chicago, IL, USA).

3. Results

The patient () characteristics stratified by inhospital mortality have been summarized in Table 2. The inhospital mortality rate was 16.11%. The inhospital mortality and no inhospital mortality groups differed significantly in terms of age, department, use and duration of the use of a ventilator, number of inpatient admissions, and CCI, ECI, and APACHE II scores.

Tables 3 and 4 show the results of univariate analyses (odds ratios, 95% confidence intervals (CIs), and values) assessing the association between mortality and the CCI and ECI, respectively, alone and in combination with other variables.

The predictive abilities of the CCI and ECI, along with other variables, as represented by the C-statistic are shown in Tables 5 and 6, respectively. For inhospital mortality, the predictive power of the APACHE II score was 0.79 (95% CI: 0.77–0.82), whereas that of the CCI alone was 0.61 (95% CI: 0.57–0.64). The C-statistic for inhospital mortality based on CCI increased to 0.77 (95% CI: 0.74–0.80) when age, sex, and mechanical ventilation were added (CCI: MODEL 2) and further increased to 0.79 (95% CI: 0.76–0.82) when all variables were included (CCI: MODEL 1). The APACHE II score had a higher predictive power for 30-day mortality (C-statistic: 0.80, 95% CI: 0.78–0.82) than did the CCI alone (C-statistic: 0.62, 95% CI: 0.59–0.66).

The ECI as the sole independent variable had a lower predictive power for inhospital, 30-day, and 1-year mortality than that of the APACHE II score and CCI. It had a C-statistic of 0.55 (95% CI: 0.52–0.59) for inhospital mortality, which increased slightly when age, sex, and mechanical ventilation were added to the model (ECI: MODEL 2). It was highest (0.78, 95% CI: 0.75–0.80) when all variables were included (ECI: MODEL 1). The ECI alone had a predictive power of 0.57 (95% CI: 0.54–0.61) for 30-day mortality and 0.62 (95% CI: 0.59–0.65) for 1-year mortality.

Figures 1–3 present the analysis of the AUROC for the prediction of inhospital, 30-day, and 1-year mortality. Contrast tests revealed that the APACHE II score and CCI (CCI MODEL 1) had comparably predicted the three mortality outcomes.

4. Discussion

This study investigated and compared the predictive power of the APACHE II score, CCI, and ECI for short-term and long-term mortalities in ICU patients. The impact of comorbidities on predictive ability was analyzed using administrative data, which are more accurate than are ICU medical records. Our results show that the CCI and ECI have less predictive power than does the APACHE II score. However, when the CCI was combined with age, sex, and ventilator use, its ability to predict mortality increased and was comparable to that of the APACHE II score. The ECI had a poorer predictive power than that of the CCI, even when combined with age, sex, and mechanical ventilation. However, further adding the number of outpatient visits, emergency department visits, and inpatient admissions during the past year improved its predictive power slightly.

The APACHE II score has been used worldwide for measuring ICU performance. The scoring system was validated and outlined in 1985 by Knaus et al. [6] and is still a popular prognosis evaluation tool in ICU settings [19]. The APACHE II score takes into account various parameters, including acute physiological variables and chronic health conditions, all of which have significant effects on the outcome prediction for ICU patients. The CCI was introduced in 1987 to predict 1-year mortality using comorbidity data from medical charts [7]. The ECI was developed using patient data from 438 acute care hospitals in California in 1992 [10], and its outcome measures were selected from those commonly available in administrative databases.

Previous studies have investigated the predictive power of the various scoring systems and their ability to determine mortality rates using administrative data. Quach et al. [20] compared the discriminative ability of the CCI and APACHE II score in predicting hospital mortality in adult multisystem ICU patients and found the former to be less effective. In their study, the CCI did not provide significantly better results even when adjusted for age, sex, and acute physiology score. However, its predictive power slightly improved when it was added to the full APACHE II model. Despite its underperformance, the CCI can be considered an alternative method of risk assessment when data for the variables included in the APACHE II score are unavailable or not recorded in a standard manner. Christensen et al. [21] studied 469 adult patients admitted to a tertiary university-affiliated ICU and found that there were no major differences in predictive power for mortality between physiology-based systems and the CCI combined with other administrative data. Fortin et al. [22] studied the predictive performance of the ECI for inhospital mortality in adult patients at a health center and found that it demonstrated excellent discrimination for all-cause inhospital mortality. Comorbidity indices and APACHE II scores have also been used to study the severity of and mortality risk adjustments for specific health conditions such as ischemic stroke [23], acute intracerebral hemorrhage [24], trauma [25], and cancer [26].

Previous studies that enrolled different types of patients have shown that the APACHE II score has the highest predictive power for mortality compared with other comorbidity indices [20, 27, 28]. These studies usually focused on short-term mortality, and studies on longer-term mortality are limited [29].

One possible reason for the significantly higher predictive power of the APACHE II score for mortality in ICU patients when compared to that of the CCI and ECI is that the acute physiology status is usually more critical in ICU patients than in other patients and varies significantly among patients. Our study results are similar to those of Ho et al. [27], who showed that replacing the chronic condition measures of the APACHE II score with those of the CCI or ECI did not significantly improve the mortality-predicting power of the APACHE II score. On the other hand, Quach et al. [20] found that if the CCI was combined with the APACHE II score, its predictive power increased from 0.626 to 0.74. Our study shows that acute physiology variables should not be replaced by other comorbidity measures when predicting mortality in ICU patients.

When comparing short-term and long-term mortalities, we found that the APACHE II score had a slightly better predictive power for short-term mortality, whereas the CCI had a higher predictive power for long-term mortality. However, the predictive power of the CCI for long-term mortality was still lower than that of the APACHE II score. The poorer performance of the CCI may be because of its coarse weights. If we put the comorbidity variables of the CCI in a regression model as a 0–1 binary indicator, the predictive ability of the CCI may strengthen. Like the CCI, the ECI also predicted long-term mortality better than short-term mortality. This supports our conclusion that acute physiology status is more important for short-term versus long-term mortality. While the APACHE II score was originally designed to measure the severity of the conditions in critical care patients, the CCI and ECI were not. This probably explains the higher predictive power of the APACHE II score in our study [21, 28]. However, considering the time and cost involved in data collection, comorbidity measures derived from administrative data (such as those used in the CCI) may still have substantial advantages in terms of data accessibility [30].

5. Limitations

This study has some limitations. First, this study retrospectively collected patient information from a single regional hospital in Eastern Taiwan. Considering the limited study period and single geographic location, our findings cannot be extrapolated to other ICUs in Taiwan. A study design including different hospitals would, therefore, produce more reliable data. Second, our findings need to be validated by prospective analysis of subsequent ICU admissions. Finally, although the use of the NHIRD has advantages (e.g., large sample sizes, long observation periods, updated information, and easy access to different information sources), it also has some disadvantages, such as possible misclassification of diseases and difficulty in controlling confounding factors [31].

6. Conclusions

For ICU patients, the APACHE II score has the strongest predictive power for short-term mortality, followed in turn by the CCI and ECI. The ability of our new model, which combines the CCI with age, sex, and use of mechanical ventilation, to predict short-term and long-term mortality in ICU patients is comparable to that of the APACHE II score.

Abbreviations

APACHE:	Acute Physiology and Chronic Health Evaluation
AUROC:	Area under the receiver operating characteristic curve
CCI:	Charlson Comorbidity Index
CI:	Confidence interval
ECI:	Elixhauser Comorbidity Index
ICU:	Intensive care unit
NHI:	National Health Insurance
NHIRD:	National Health Insurance Research Database
OR:	Odds ratio
ROC:	Receiver operating characteristic.

Data Availability

The data sets used and analyzed in this study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors declare that they have no competing interests.

Authors’ Contributions

Yu-Ting Hsu and Yi-Ting He contributed equally to this study.

Acknowledgments

We thank our colleagues who provided their insight and expertise, as well as those who helped with the data collection and cared for the study patients. This study was supported by the Department of Anesthesiology of the Taipei Veterans General Hospital and the ICU of the National Yang-Ming University Hospital, Taiwan.

Supplementary Materials

ICD-9 codes for different comorbidities evaluated in the study. (Supplementary Materials)

References

R. A. Gooch and J. M. Kahn, “ICU bed supply, utilization, and health care spending: an example of demand elasticity,” JAMA, vol. 311, no. 6, pp. 567-568, 2014.
View at: Publisher Site | Google Scholar
E. Mery and J. M. Kahn, “Does space make waste? The influence of ICU bed capacity on admission decisions,” Critical Care, vol. 17, no. 3, p. 315, 2013.
View at: Publisher Site | Google Scholar
H. T. Stelfox, B. R. Hemmelgarn, S. M. Bagshaw et al., “Intensive care unit bed availability and outcomes for hospitalized patients with sudden clinical deterioration,” Archives of Internal Medicine, vol. 172, no. 6, pp. 467–474, 2012.
View at: Publisher Site | Google Scholar
L. M. Chen, M. Render, A. Sales, E. H. Kennedy, W. Wiitala, and T. P. Hofer, “Intensive care unit admitting patterns in the veterans affairs health care system,” Archives of Internal Medicine, vol. 172, no. 16, pp. 1220–1226, 2012.
View at: Publisher Site | Google Scholar
M. T. Sharabiani, P. Aylin, and A. Bottle, “Systematic review of comorbidity indices for administrative data,” Medical Care, vol. 50, no. 12, pp. 1109–1118, 2012.
View at: Publisher Site | Google Scholar
W. A. Knaus, E. A. Draper, D. P. Wagner, and J. E. Zimmerman, “APACHE II: a severity of disease classification system,” Critical Care Medicine, vol. 13, no. 10, pp. 818–829, 1985.
View at: Google Scholar
M. E. Charlson, P. Pompei, K. L. Ales, and C. MacKenzie, “A new method of classifying prognostic comorbidity in longitudinal studies: development and validation,” Journal of Chronic Diseases, vol. 40, no. 5, pp. 373–383, 1987.
View at: Publisher Site | Google Scholar
R. M. Poses, D. McClish, W. R. Smith, C. Bekes, and W. E. Scott, “Prediction of survival of critically ill patients by admission comorbidity,” Journal of Clinical Epidemiology, vol. 49, no. 7, pp. 743–747, 1996.
View at: Publisher Site | Google Scholar
D. M. Needham, D. C. Scales, A. Laupacis, and P. J. Pronovost, “A systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research,” Journal of Critical Care, vol. 20, no. 1, pp. 12–19, 2005.
View at: Publisher Site | Google Scholar
A. Elixhauser, C. Steiner, D. R. Harris, and R. M. Coffey, “Comorbidity measures for use with administrative data,” Medical Care, vol. 36, no. 1, pp. 8–27, 1998.
View at: Publisher Site | Google Scholar
H. Quan, V. Sundararajan, P. Halfon et al., “Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data,” Medical Care, vol. 43, no. 11, pp. 1130–1139, 2005.
View at: Publisher Site | Google Scholar
T. M. Cheng, “Taiwan's new national health insurance program: genesis and experience so far,” Health Affairs, vol. 22, no. 3, pp. 61–76, 2003.
View at: Publisher Site | Google Scholar
C. L. Cheng, Y. H. Kao, S. J. Lin, C. H. Lee, and M. L. Lai, “Validation of the National Health Insurance Research Database with ischemic stroke cases in Taiwan,” Pharmacoepidemiology and Drug Safety, vol. 20, no. 3, pp. 236–242, 2011.
View at: Publisher Site | Google Scholar
D. B. Preen, C. D. Holman, K. Spilsbury, J. B. Semmens, and K. J. Brameld, “Length of comorbidity lookback period affected regression model performance of administrative health data,” Journal of Clinical Epidemiology, vol. 59, no. 9, pp. 940–946, 2006.
View at: Publisher Site | Google Scholar
R. A. Deyo, D. C. Cherkin, and M. A. Ciol, “Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases,” Journal of Clinical Epidemiology, vol. 45, no. 6, pp. 613–619, 1992.
View at: Publisher Site | Google Scholar
M. Norena, H. Wong, W. D. Thompson, S. P. Keenan, and P. M. Dodek, “Adjustment of intensive care unit outcomes for severity of illness and comorbidity scores,” Journal of Critical Care, vol. 21, no. 2, pp. 142–150, 2006.
View at: Publisher Site | Google Scholar
M. J. Pencina, R. B. D'Agostino, and R. S. Vasan, “Statistical methods for assessment of added usefulness of new biomarkers,” Clinical Chemistry and Laboratory Medicine, vol. 48, no. 12, pp. 1703–1711, 2010.
View at: Publisher Site | Google Scholar
J. A. Hanley and B. J. McNeil, “A method of comparing the areas under receiver operating characteristic curves derived from the same cases,” Radiology, vol. 148, no. 3, pp. 839–843, 1983.
View at: Publisher Site | Google Scholar
M. Rué, A. A. MD, M. Álvarez, S. Quintana, and C. Valero, “Performance of the Mortality Probability Models in assessing severity of illness during the first week in the intensive care unit,” Critical Care Medicine, vol. 28, no. 8, pp. 2819–2824, 2000.
View at: Publisher Site | Google Scholar
S. Quach, D. A. Hennessy, P. Faris, A. Fong, H. Quan, and C. Doig, “A comparison between the APACHE II and Charlson Index Score for predicting hospital mortality in critically ill patients,” BMC Health Services Research, vol. 9, no. 1, 2009.
View at: Publisher Site | Google Scholar
S. Christensen, M. B. Johansen, C. F. Christiansen, R. Jensen, and S. Lemeshow, “Comparison of Charlson comorbidity index with SAPS and APACHE scores for prediction of mortality following intensive care,” Clinical Epidemiology, vol. 3, pp. 203–211, 2011.
View at: Publisher Site | Google Scholar
Y. Fortin, J. A. Crispo, D. Cohen, D. McNair, D. R. Mattison, and D. Krewski, “External validation and comparison of two variants of the Elixhauser comorbidity measures for all-cause mortality,” PLoS One, vol. 12, no. 3, pp. e0174379–e0174379, 2017.
View at: Publisher Site | Google Scholar
U. Ludwigs, M. Csatlos, and J. Hulting, “Predicting in-hospital mortality in acute myocardial infarction: impact of thrombolytic therapy on APACHE II performance,” Scandinavian Cardiovascular Journal, vol. 34, no. 4, pp. 371–376, 2000.
View at: Publisher Site | Google Scholar
Y. Huang, J. Chen, S. Zhong, and J. Yuan, “Role of APACHE II scoring system in the prediction of severity and outcome of acute intracerebral hemorrhage,” The International Journal of Neuroscience, vol. 126, no. 11, pp. 1020–1024, 2016.
View at: Publisher Site | Google Scholar
L. A. Dossett, L. A. Redhage, R. G. Sawyer, and A. K. May, “Revisiting the validity of APACHE II in the trauma ICU: improved risk stratification in critically injured adults,” Injury, vol. 40, no. 9, pp. 993–998, 2009.
View at: Publisher Site | Google Scholar
L. Chang, C. F. Horng, Y. C. Huang, and Y. Y. Hsieh, “Prognostic accuracy of Acute Physiology and Chronic Health Evaluation II scores in critically ill cancer patients,” American Journal of Critical Care, vol. 15, no. 1, pp. 47–53, 2006.
View at: Google Scholar
K. M. Ho, J. Finn, M. Knuiman, and S. A. Webb, “Combining multiple comorbidities with Acute Physiology Score to predict hospital mortality of critically ill patients: a linked data cohort study,” Anaesthesia, vol. 62, no. 11, pp. 1095–1100, 2007.
View at: Publisher Site | Google Scholar
K. M. Ho, K. Y. Lee, T. Williams, J. Finn, M. Knuiman, and S. A. Webb, “Comparison of Acute Physiology and Chronic Health Evaluation (APACHE) II score with organ failure scores to predict hospital mortality,” Anaesthesia, vol. 62, no. 5, pp. 466–473, 2007.
View at: Publisher Site | Google Scholar
J. A. Johnston, D. P. Wagner, S. Timmons, D. Welsh, J. Tsevat, and M. L. Render, “Impact of different measures of comorbid disease on predicted mortality of intensive care unit patients,” Medical Care, vol. 40, no. 10, pp. 929–940, 2002.
View at: Publisher Site | Google Scholar
J. F. Farley, C. R. Harley, and J. W. Devine, “A comparison of comorbidity measurements to predict healthcare expenditures,” The American Journal of Managed Care, vol. 12, no. 2, pp. 110–119, 2006.
View at: Google Scholar
C. Mazzali and P. Duca, “Use of administrative data in healthcare research,” Internal and Emergency Medicine, vol. 10, no. 4, pp. 517–524, 2015.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2020 Yu-Ting Hsu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

959

Downloads

1378

Citations