Abstract
Objectives and Background. Scoring systems in Emergency Departments (EDs) are rarely validated. This study aimed to examine the Paediatric Observation Priority Score (POPS), a method of quantifying patient acuity, in EDs in the United Kingdom, and determine baseline performance characteristics. Methods. POPS was implemented in 4 EDs for children (ages of 0 to 16) with participants grouped into 3 categories: discharged from ED, discharged but with return within 7 days, and admitted for less or more than 24 hours. Results. 3323 participants with POPS scores ranging from 0 to 11 (mean = 2.33) were included. The proportion of each POPS score varied between sites with approximately 10–20% being POPS 0 and 12–25% POPS greater than 4. Odds ratio of readmission with POPS 5–9 against 0–4 was 2.05 (CI 1.20 to 3.52). POPS 0–4 showed no significant difference (p = 0.93) in relation to admission/discharge rates between sites with a significant difference found (p < 0.01) for POPS > 5. Conclusion. It is feasible to implement POPS into EDs with similar performance characteristics to the original site of development. There is now evidence to support a wider health service evaluation to refine and improve the performance of POPS.
1. Background
In developed countries serious childhood illness is uncommon but increasing numbers of families seek urgent face-to-face medical assessment [1, 2] for minor illness, and there continue to be missed opportunities in staff identifying seriously ill children [3]. Risk-averse strategies of referring all children of “potential concern” for specialist paediatric assessment overload an already stretched out-of-hours system and lead to unnecessary hospital admissions. Rational and efficient healthcare requires there to be an objective system of measuring how unwell a child is. Although “scoring tools” exist most are ward-based “early warning scores (EWS)” which identify children needing high-level critical care. They are not validated for use early in illness [4], or through the range of childhood acute illness (mild to severe). Attempts to implement ward-based scoring systems have resulted in disappointing results [5, 6].
The Paediatric Observation Priority Score (POPS) is a method of identifying the range of severity of childhood illness, to support staff in taking the decision to redirect the child to primary care or discharge to self-care and to help them in expediting senior/specialist assistance for deteriorating children [7]. POPS is not intended to determine a course of action but to support clinical decisions, especially for staff not familiar with dealing with child.
POPS uses a combination of physiological, behavioural, and known-risk parameters to generate a score of either 0, 1, or 2 (Figure 1), giving a total score (0–16). POPS has undergone an initial single centre validation in a cohort of 942 patients [8], a larger group than any previous Emergency Department (ED) study [9]. Further local data on over 20,000 patients has demonstrated a relationship between length of admission and increasing POPS, with an increase when POPS > 4, and good safety profile [10].
Scoring systems often perform less well outside the organisation that developed the score. This study aimed to validate how the POPS score performed in other EDs in the United Kingdom (UK).
2. Methods
The objectives of the study were to describe the distribution of scores in EDs outside our institution, confirm a threshold indicator of POPS > 4 for hospital admission risk (which was the threshold in local data), and examine discharge and representation rates following uptake of POPS. POPS was implemented in 4 EDs. The characteristics of each of the EDs is shown in Table 1. The implementation and education of local staff about POPS was left to the discretion of the local team.
All children (0–16 years) presenting to these departments were included on a convenience basis from February 2013 to June 2013. The department’s normal patient assessment/triage procedure was followed as per local protocol with the POPS observations being taken at the first available opportunity. National Standards in UK require an initial assessment of patients to occur within 15 minutes of arrival which all departments worked towards. A report form was used for prospective collection of each child’s POPS score (Figure 1), age, and unique study identifier. The local research team later recorded whether or not the patient was admitted, length of admission, and representation in the next 7 days.
For the analysis patients were grouped into those that (a) were discharged from ED (including outpatient referrals) and who did not return within 7 days, (b) were admitted for less than 24 hours or represented within 7 days after discharge, and (c) were admitted to hospital for more than 24 hours. For validation of the admission threshold patients were separated into two groups based on an odds of admission calculated for the groups above and below a POPS of 4.
Ethical approval was granted via the East Midlands REC committee 2 and need for consent waived. Descriptive and statistical data were examined using SPSS v22.0.
3. Results
Of 3338 patients submitted 15 were excluded (7 self-discharged and 8 unclear outcomes). The remaining patients are described in Table 1. Overall discharge rate was 61.7% across all sites with a range of 49.1% to 72.1% at individual sites. 111 (5.4%) of patients were represented (Table 2) with increasing representations rates as POPS increases, that is, a trend to being linear (Figure 2). POPS scores ranged from 0 to 11 with a median score 2 and interquartile range of 2. The POPS scores in the 4 sites are shown in Figure 3. The proportion of each POPS score varied between sites with approximately 10–20% being POPS 0 and 12–25% POPS greater than 4. Admission rates (both for less than and more than 24 hours) increased as POPS increased in all 4 sites (Figure 4). In relation to admission and discharge the receiver operating characteristic (ROC) curve area across all sites for admission was 0.66 (CI 0.64 to 0.69).
In relation to threshold analysis across all centres, with POPS dichotomised into 0–4 and 5–11, odds ratio of admission was 5.13 (CI 4.06 to 6.49). Controlling for location there was no significant difference in the odds ratios for POPS 0–4 (). However, there was a significant difference between admission/discharge for POPS > 5 () (Table 3). Odds ratio of readmission with POPS 5–9 against 0–4 was 2.05 (CI 1.20 to 3.52).
4. Discussion
This data demonstrates it is feasible to introduce POPS into EDs and maintain similar performance characteristics between sites. Across all sites POPS performed moderately well at predicting admission and safe discharge with similar characteristics to the site of derivation. Despite no specific education in the use of POPS, its performance was similar between sites with some expected variation due to different populations and size of units. Its performance, despite the absence of a well-defined, implementation strategy, would suggest it is a system that is relatively easy to implement successfully.
Unlike inpatient Early Warning Scores designed to detect deterioration, POPS has demonstrated an ability to aid detection of patients requiring hospitalisation in the ED safely, while also supporting discharge decision making for children with minor illness. The strong relationship between return rates and initial POPS allows a clinician or individual unit to define their own management thresholds, depending on their attitude to risk. It may be that children with intermediate POPS scores (3–6) would benefit from a mandated senior review prior to discharge, and identifying this middle group would assist each site in fine-tuning their discharge/admission threshold. At POPS above 4 in all the sites 30% of children were admitted for over 24 hours’ rate. An admission for over 24 hours infers that ongoing management was deemed necessary by the paediatric team and admission was unlikely to be related to any service or capacity pressures. While an individual score should never on its own determine the decision of a child these results support the use of a cut-off of around 4 as a decision making aid for admission and discharge. Senior clinicians could obviously make their own adjustments depending on local factors.
There are many potential benefits to wide scale implementation of this system aside from the rapid detection of critically unwell children. Potential cost savings include fewer referrals for inpatient treatment (current risk-averse practice results in overreferral), fewer serious adverse events (missed serious illness), and reducing numbers or costs of litigation claims. The combination of POPS with inpatient EWS is being investigated and will further enhance serious illness detection in the future (personal communication).
Although data was collected prospectively the time periods were not uniform between sites and there was no measure to determine standardisation of assessment between staff in a given Emergency Department. The ROC score remains low but comparable with other Paediatric Early Warning Scores used in children’s Emergency Departments [5] and an external group have utilised a modified version of POPS and demonstrated its predictive value and safety profile [11]. It is not the intention for the POPS to be decision making arbiter, more support of the assessment of a child, especially for junior medical and nursing staff.
5. Conclusion
We have demonstrated it is feasible to implement POPS into several EDs. This data can now be used to inform a formal prospective external validation study and assist in ongoing rollout to interested sites.
Conflicts of Interest
Dr. Roland and Dr. Davies own copyright to the Paediatric Observation Priority Score which is available via a creative commons attribution noncommercial share-alike 4.0 license.
Acknowledgments
The contribution of the Principal Investigators Dr. Elizabeth Gilby (Bath), Liza Keating (Reading), Mark Lyttle (Bristol), and Victoria Stacey (Gloucestershire) to the success of this study should be recognised. The initial contribution of Ashley Douglas to data amalgamation is also acknowledged and was recognised in an early abstract of this work (Roland D., Arshad F., Douglas A., et al. Spreading innovation in children’s emergency care: an external pragmatic validation of the paediatric observation priority score. Emerg Med J 2015; 32:978-979).