Abstract
Various research methods have been used to obtain skeletal fracture data and report the incidence of fractures. A large number of British studies have used data collected in emergency departments, and not data derived from orthopaedic units. We hypothesised that fracture data will differ depending upon the methodology employed to capture it. Two commonly used sources of fracture data at our institution were compared, (the Emergency Department (ED) database and the Orthopaedic Trauma Unit (OTU) database), using a cohort of adult patients from our defined population as the study sample. We performed univariate analyses to identify differences between groups with accurate and inaccurate ED fracture diagnoses. We then performed a binary logistic regression analysis to determine the best predictors of diagnostic accuracy. In one year, 7,449 patients were referred to the OTU. Three-quarters were referred with fractures. The overall false positive fracture referral rate was 25%. Several fracture subtypes were commonly overdiagnosed in the ED. Regression analysis showed that patient age, patient gender, and the seniority of the referring clinician were independently predictive of an accurate fracture diagnosis. We suggest that studies making use of ED fracture data may potentially overestimate the incidence of adult fractures.
1. Introduction
The systematic collection and analysis of skeletal fracture data is the essence of fracture epidemiology. Its application to clinical practice allows clinicians to compare affected with unaffected patient groups, determine definable and preventable characteristics that predispose to skeletal fracture, and ensure the provision of appropriate treatment strategies.
Cummings et al. [1], in their review of the methodological challenges facing all injury epidemiologists, identified several key areas including the definition and classification of injuries and the importance of defining the population at risk. The analysis of fracture data can only occur if fractures are identified accurately. This process relies upon the ability of clinicians to make clinical judgements and accurately interpret standard plain radiographs. It has been suggested that experienced clinicians are able to identify fractures with greater accuracy than those less experienced [2, 3]. In adult patients, only a small number of studies have analysed the incidence of fractures in a given population. Reported rates vary considerably, with Donaldson et al. [4] suggesting a rate four times higher than Brinker and O’Connor [5] (36.0/1000/yr versus 8.5/1000/yr).
It is highly likely that variation exists between populations in different countries, and between regions within the same country, but we feel the reported differences in adult fracture incidence in the literature are unlikely to be explained by population demographics alone. Notably, many studies have used different research methods in obtaining skeletal fracture data.
In four studies the fracture diagnoses were made from radiographs examined by orthopaedic specialists and radiologists, and the authors reported similar results (Court-Brown et al., 13.7/1,000/yr [6]; Court-Brown and Caesar, 11.2/1000/yr [7]; Singer et al., 13.8/1000/yr [8]; Donaldson et al., 9.1/1000/yr [9]). Three investigations obtained fracture data from emergency department databases and coding systems, and reported notably higher rates of fracture (Johansen et al., 21.1/1000/yr [10]; Sahlin, 22.8/1000/yr [11]; Fife and Barancik, 21.0/1000/yr [12]). Donaldson et al. employed a patient questionnaire strategy to determine fracture incidence (36.0/1000/yr [4]). Brinker and O'Connor obtained medical insurance company data (8.5/1000/yr [5]), while van Staa et al. made use of the United Kingdom (UK) general practice research database (GPRD) and reported a fracture incidence of 10.3/1000/yr [13].
In order to investigate the discrepancy in adult fracture incidence between studies, a prospective cohort study was undertaken at our institution. The ascertainment of fractures in the emergency department (ED) was compared with that of the orthopaedic trauma unit (OTU) in the same cohort of adult patients. Our primary research objective was to test our hypothesis that a difference exists between the ED and the OTU fracture databases. Our secondary objective was to identify predictors of ED diagnostic accuracy by examining variables such as patient age, patient gender, fracture subtype, and the grade of referring ED clinician.
2. Materials and Methods
2.1. Participants
The study was conducted with the approval of the South East Scotland research ethics service. The Royal Infirmary of Edinburgh (RIE) serves a defined adult population. All adult patients presenting to the ED, with onward referral to the OTU, were prospectively recorded for one year (July 2007 to June 2008). This is the only OTU for a population of 545,081 adults (52% women). Population data were obtained from the General Register Office for Scotland [14].
All patients in Edinburgh aged less than 13 years, and a proportion aged less than 15 years, are treated at a separate paediatric institution. Therefore only patients aged 15 years or older were included. Patients residing out with the defined catchment area of the RIE and those referred from other institutions were excluded. All patient records in the ED are held electronically and can be accessed and reviewed as required. Details of OTU fracture clinic review are also held electronically. Patients for whom no ED notes or fracture clinic notes could be obtained were excluded, as were those for whom the grade of referring ED clinician was unclear. Patients who failed to attend fracture clinic or chose to cancel their appointment were also excluded.
Patients with fractures requiring immediate admission and inpatient treatment were deemed unsuitable for inclusion in this study. Such patients were reviewed by the ED and OTU clinicians simultaneously, which limited the identification of any discrepancy in fracture ascertainment. In contrast, patients requiring outpatient OTU fracture treatment were rarely reviewed by the OTU clinician prior to fracture clinic attendance. These patients therefore made up the study population. In order to pick up possible misdiagnoses, all fracture and nonfracture patient referrals were included in the analysis.
2.2. Classification of Injury
Demographic data were recorded for all patients referred to the OTU fracture clinic. ED diagnostic data were obtained from the electronic patient record, and details of the ED attendance, clinicians’ examination, provisional diagnosis and reason for referral were recorded. Patient injury types were defined as fractures, dislocations, soft tissue injuries, musculotendinous ruptures, wounds, or other injury types. Fracture subtypes were determined, and are detailed in Section 3.
2.3. Data Handling
The grade of the referring ED clinician was recorded. On presenting to the ED, patients were seen and referred by a nurse practitioner (NP), junior grade physician (SHO/FY/ST1-2), middle grade physician (SpR/ST3-6), or consultant grade physician (Cons). In cases where the patient had been reviewed by a number of ED clinicians prior to referral, the grade of the most senior clinician was recorded. In cases where the OTU clinician had been asked to review the patient prior to referral, the grade was recorded as “OTU”.
The final OTU diagnosis was made by the OTU clinician in fracture clinic, with or without the additional information provided by the radiology report of accompanying plain radiographs. In cases where the OTU diagnosis remained unclear pending further review or repeat imaging, recording of the definitive diagnosis was deferred until such time as the diagnosis could be confidently made.
In order to test the hypothesis that a difference in fracture ascertainment exists between the ED and the OTU at our institution, we defined a “correct” ED diagnosis (true positive case) as one where both the injury type and fracture subtype agreed with the final OTU diagnosis. Any diagnostic discrepancy was deemed a false positive. In particular, when the ED diagnosis of “fracture” injury type was correct, but the fracture subtype was incorrectly diagnosed, this was deemed a false positive. The recording of true and false negatives was not possible, as uninjured patients were not referred for OTU assessment.
2.4. Statistical Analysis
No estimate of sample size was made on the basis of assumptions about the primary research objective. A twelve month sample of convenience was used. For the purposes of statistical analysis, multiple events (multiple injuries at presentation; recurrent injury in the same patient over time) were treated as distinct entities.
2.5. Primary Research Objective
Continuous data were presented in terms of the median and range. Median values between groups were compared using the Mann-Whitney (MWU) test. Groups of categorical variables were compared using the chi-square test. A two-tailed value of <0.05 was considered statistically significant. The positive predictive value (PPV) of ED diagnosis was calculated by dividing the number of true positive (TP) diagnoses by the number of true positive plus false positive (FP) diagnoses, expressed as a percentage according to the following:
2.6. Secondary Research Objective
To investigate the potential effect of patient age, patient gender, injury type, fracture subtype, and grade of referring ED clinician on the chance of a correct ED diagnosis, two binary logistic regression analyses were performed. Only those variables that proved nearly significant (<0.01) or significant (<0.05) after univariate analyses were entered into the regression analyses. The first regression analysis was performed to determine the best predictors of a correct ED injury type diagnosis, and the second to determine predictors of a correct ED fracture subtype diagnosis. The regression models produced an odds ratio for each of the independent variables, the Exp(B), along with the accompanying 95% confidence interval.
3. Results
3.1. Patients
In one year, 7,762 patients were referred from ED to the OTU fracture clinic. Two-hundred and fifty-one patients either cancelled their appointment or failed to attend for review. In twelve cases, no ED records were available. In fifty cases, the grade of the referring ED clinician was unclear. The patient pathway is summarised in the flow diagram (see Figure 1).
3.2. Primary Research Objective
After exclusions, 7,449 patients had an ED diagnosis and OTU diagnosis available for comparison. The overall PPV for an ED diagnosis was 78.0% (5,811 true positives and 1,638 false positives). Patients attributed a false positive ED diagnosis (age 33 yrs, 20–52 yrs) were significantly younger than those given a true positive ED diagnosis (age 38 yrs, 22–58 yrs), (, MWU).
Male patients accounted for 4,127 (55.4%) of 7,449 ED referrals. The PPV of a correct ED diagnosis was higher in men (PPV = 79.0%) than in women (PPV = 76.8%), (). A correct ED diagnosis was highest when the patient had been reviewed by the OTU clinician in the ED prior to referral (Table 1). However, the OTU clinician referred the fewest number of patients. Senior ED clinicians (Consultants and SpRs) were more likely to refer a patient with a correct diagnosis than junior colleagues (SHOs and NPs). Junior ED clinicians made 5,620 (75.4%) referrals to the OTU.
Seventeen patients were referred to fracture clinic with an ED diagnosis of soft tissue wounding. All had been correctly diagnosed (Table 2). “Other” diagnoses accounted for 247 referred cases, and contained a wide variety of conditions that did not meet the inclusion criteria for another injury type category. Commonly encountered cases included suspected soft tissue infection, postoperative pain and swelling, joint or limb pain in the absence of trauma, exacerbations of arthropathies, and the presence of symptomatic orthopaedic implants.
Traumatic joint dislocations were accurately diagnosed by the ED clinician. The three commonest dislocation final diagnoses were those affecting the glenohumeral joint (), fingers or thumb (), and the acromioclavicular joint ().
Musculotendinous rupture was diagnosed in 172 patients, but in thirty patients the OTU diagnosis differed from this. The ascertainment of this injury type was less accurate than that of a traumatic dislocation (PPV 82.6% versus 92.4%). The three commonest rupture final diagnoses involved the Achilles/gastrocnemius complex (), the extensor apparatus in the fingers (), and the ulnar collateral ligament of the thumb (). Of note, patients with extensor mechanism ruptures of the knee were all admitted to the OTU for operative treatment. Where a diagnostic discrepancy was apparent, this was most commonly due to referred ruptures of the Achilles’ tendon receiving an OTU diagnosis of soft tissue injury.
Emergency department clinicians diagnosed 854 soft tissue injuries with a PPV of 98.5%. The majority of soft tissue injury referrals were knee injuries () directed towards dedicated acute knee clinics. Soft tissue injury to the shoulder () and ankle () were also commonly encountered.
Three-quarters of ED patient referrals to the OTU fracture clinic arrived with a diagnosis of skeletal fracture. Of 5,695 patients referred with a fracture, 1,491 (26.2%) were attributed a false positive diagnosis. A small number had been miscoded as fractures from a different anatomical region. The majority of false positive diagnoses received an OTU diagnosis of soft tissue injury. Eight fracture subtype diagnoses were significantly better diagnosed than the average, while a further eight were diagnosed with less accuracy (Table 3).
3.3. Secondary Research Objective
To identify significant predictors of accurate ED diagnoses and injury ascertainment, all independent variables that had proven nearly significant (<0.01) or significant (<0.05) after univariate analysis were entered into binary logistic regression models. The first model aimed to identify predictors of a correct ED injury type diagnosis, and the results are shown in Table 4.
The second regression model identified predictors of a correct ED fracture subtype diagnosis. Patient age (Exp(B) 1.01, CI 1.01-1.02), male gender (Exp(B) 1.17, CI 1.01–1.36), SHO grade clinician (Exp(B) 0.76, CI 0.66–0.88) and ortho grade clinician (Exp(B) 3.60, CI 2.00–6.48) remained significantly predictive variables.
4. Discussion
At our institution, we discovered a significant difference between the ascertainment of skeletal fractures in adult outpatients when comparing ED and OTU data sources. The PPV of a correct ED fracture diagnosis was 73.8%. Several common fracture subtype diagnoses had PPVs that were less accurate than the average (fractures of the carpus, proximal tibia, proximal radius, calcaneus, talus, and midfoot). Regression analysis showed that patient age, patient gender, and the grade of referring ED clinician were independently predictive of a correct fracture diagnosis.
These results suggest that studies on the epidemiology of adult fractures which use ED data sources may overestimate adult fracture incidence. The published studies using ED methodology [10–12] have reported adult fracture incidence to be 50% greater than studies involving orthopaedic data collection [6–9]. At our institution, two-thirds of adult fractures encountered were treated on an outpatient basis during the study period, and the number of outpatient fractures has been overestimated by 26.2% (the overall false positive rate).
We accept this investigation has been limited to outpatient OTU referrals, and has not investigated the potential discrepancy between inpatient admission and discharge diagnoses. However, a systematic review of inpatient discharge coding accuracy in the UK [15] concluded that discharge coding performed by administrative staff is on average highly accurate. Furthermore, at our institution all inpatient fracture coding is routinely performed by OTU clinicians rather than administrative staff. Further work will focus on identifying a diagnostic discrepancy for inpatient fracture data.
These results apply to our institution and the Edinburgh population. They may not be applicable to populations whose social demographics vary substantially from our own, but we believe they will apply to many similar regions. Additionally, differences in staffing of EDs between regions will also affect the discrepancy between ED and orthopaedic fracture ascertainment. We accept that in clinical practice senior clinicians are called upon to review more severe or difficult cases, and that this may have an effect on the accuracy of referral. Notably, nurse practitioner clinicians performed equally as well as junior ED doctors, further confirming previous studies in this area [16–18]. At our institution the majority of patients with minor musculoskeletal injuries are seen and treated by junior ED clinicians. In departments where the proportion of middle grade and senior clinicians is higher, the overall false positive fracture diagnosis rate is likely to be lower.
Despite the inherent limitations of this study, it has involved the close examination of a large cohort of outpatients referred to fracture clinic and identified a significant difference between two commonly used fracture data sources employed in the fracture epidemiology literature. We believe it is the first study to attempt a comparison of this type.
Other research methods described in the literature have included the use of patient questionnaires, insurance company data, and the GPRD. Donaldson et al. [4] relied upon a patient questionnaire to estimate overall fracture incidence in the population. This type of methodology suffers from patient recall bias [19–21], but had the advantage of including fractures that are often treated by the family physician (e.g., osteoporotic vertebral body fractures). Brinker and O’Connor [5] obtained insurance company data in an attempt to estimate the incidence of fractures in Texas, USA. Importantly, uninsured individuals were not included.
The GPRD was used by van Staa and colleagues in an attempt to define fractures patterns in England and Wales [13]. This national database comprises the computerised medical records of a large number of participating general practitioners, and is widely used in epidemiological research. In more recent years, the GPRD has proved a popular source of fracture data as researchers have attempted to identify risk factors for fracture in various patient groups. As an illustration of its popularity, a search of the recent fracture literature was undertaken using the Ovid MEDLINE search engine. From 2004 to 2008, 26 peer-reviewed studies were found containing “fracture(s)” in the title, and “general practice research database” as a keyword. Five studies dealt with the paediatric population [22–26], while the remaining 21 studies analysed adult fractures [27–47].
The GPRD has been reportedly validated [48–50], with a true positive rate ranging from 70% to 95% depending on the diagnosis or disease of interest. However, there is very little information in the literature regarding the validity of GPRD skeletal fracture data. The GPRD derives its fracture data from two sources: inpatient fracture data are derived from secondary care discharge letters, and outpatient fracture data are obtained directly from ED discharge records. Van Staa and colleagues, in 2000, analysed hip fractures in the GPRD [51]. They reported that 90.7% of GPRD entries were confirmed as accurate by the general practitioner, and 86.5% were confirmed by secondary care discharge letter. To our knowledge, no such data exist for fractures treated on an outpatient basis.
The results of this study highlight the potential error that epidemiologists might expect to encounter if obtaining fracture data from the ED or GPRD. Where possible, we believe future epidemiological fracture research should make use of data obtained from orthopaedic departments in order to minimise the potential error in fracture ascertainment.
Conflict of Interests
The authors have no conflicts of interests to declare.