Abstract
Background. Random safety audits (RSAs) are a safety tool but have not been widely used in hospitals. Objectives. To determine the frequency of proper use of equipment safety mechanisms in relation to monitoring and mechanical ventilation by performing RSAs. The study also determined whether factors related to the patient, time period, or characteristics of the area of admission influenced how the device safety systems were used. Methods. A prospective observational study was conducted in a level III-C Neonatal Intensive Care Unit (NICU) during 2012. 87 days were randomly selected. Appropriate overall use was defined when all evaluated variables were correctly programmed in the audited device. Results. A total of 383 monitor and ventilator audits were performed. The Kappa coefficient of interobserver agreement was 0.93. The rate of appropriate overall use of the monitors and respiratory support equipment was 33.68%. Significant differences were found with improved usage during weekends, OR 1.85 (1.12–3.06, ), and during the late shift (3 pm to 10 pm), OR 1.59 (1.03–2.4, ). Conclusions. Equipment safety systems of monitors and ventilators are not properly used. To improve patient safety, we should identify which alarms are really needed and where the difficulties lie for the correct alarm programming.
1. Introduction
Random safety audits (RSAs) are a safety tool widely used in industry due to their ability to identify errors and potentially hazardous situations, thereby enabling prevention of adverse events. RSAs consist of continuously monitoring high-risk procedures in order to identify and address error-prone points in the system, difficult to detect with other methods, before they cause patient harm [1]. The application of this method in a hospital setting could be extremely valuable as it evaluates real-time clinical practice and provides immediate feedback to the staff in the unit [2]. Moreover, this method, which is applied by clinical staff, only requires simple training and involves a low cost of implementation. Despite all these advantages, RSAs have rarely been applied in the hospital setting and there is very little published data on their use as a tool for safety and quality control [3, 4].
Neonatal Intensive Care Units (NICU) have a large variety of different technological devices, most of which have built-in safety systems that help ensure appropriate use. In the field of neonatology, much attention has been paid to other types of error but less to the potential for technology-related errors, despite the fact that good patient outcomes largely depend on how the technology is used.
Consequently, we decided to implement RSAs as a safety tool to analyse the use of technology in our NICU. The primary aim of this study was to determine the frequency of proper use of equipment safety mechanisms and other measures involved in patient safety in relation to monitoring and invasive and noninvasive mechanical ventilation by performing RSAs. The study also determined whether factors related to the patient, time period, or characteristics of the area of admission influenced how the device safety systems were used.
2. Materials and Methods
A prospective observational study was conducted from January 1 to December 31, 2012, in a level III-C NICU with 1000 admissions annually. 120 patients with birth weight less than 1500 g per year were admitted to the NICU with a survival rate of 90%, bronchopulmonary dysplasia (BPD) rate at 36 weeks of 16.4%, and a rate for retinopathy of prematurity (ROP) grade 3 or higher of 1.4%. Our NICU is divided into three areas for critical care: a large one with 10 beds (NICU-A) and two small ones (NICU-B and NICU-C), one with 4 and the other with 5 beds, and two additional areas with 24 medium care cots. Physicians and nurses were surveyed according to the modified Delphi technique [5] on the technological devices and procedures in which the recommendations for use were apparently often not met, ensuring that the most relevant equipment and procedures were included. A total of 23 technological devices and procedures were selected (shown in Table 1) and 23 cards were produced, each containing the variables to be evaluated for each device or procedure.
During the study period, 2 days a week were randomly selected and included week days, weekends, and holidays. The shifts, early, from 8 am to 3 pm, or late, from 3 pm to 10 pm, were also randomly selected. Each study day, 2 cards were randomly selected for audit. On one day, 2 kinds of technological devices, 2 kinds of procedures, or 1 kind of device and 1 kind procedure would be audited. As it was used less frequently, the high-frequency oscillatory ventilator was audited every other day whenever it was in use. Each day of the research, a researcher (EBS or MCPG) identified selected devices or procedures in use by patients which had to be audited in order to check all study variables. The NICU staff did not know the purpose of the audit but if an error was detected which might involve a potential danger to the patient, the caregivers were immediately informed. The degree of reproducibility of the 2 investigators was analysed by simultaneous rounds of audits. In this study, the results presented are for the monitoring and respiratory support equipment, so only the audit procedure for this specific equipment is described.
2.1. Monitoring Equipment and Equipment for Invasive and Noninvasive Mechanical Ventilation
Information about Masimo RADICAL-7 pulse oximeters and Philips IntelliVue MP70 multimeasurement monitors was collected (see Table 2). Consideration was given as to whether the monitored patients had some type of assisted ventilation and supplemental oxygen. The outcome variable “appropriate overall use” was defined to describe when all the variables were correctly programmed in the audited monitor. Information about Dräger Babylog 8000plus conventional mechanical ventilator, Carefusion SensorMedics high-frequency oscillatory ventilator, and Viasys Infant Flow system continuous positive airway pressure (CPAP) was collected (see Table 2). The outcome variable “appropriate overall use” was defined to describe when all the variables were correctly programmed in the audited ventilator. A critical incident was defined as any situation detected concerning the devices reviewed that could put the patient in immediate risk.
2.2. Other Variables
Information was collected on the patient, the time, and characteristics of the NICU area where the patient was admitted in order to evaluate whether or not they influenced the use of the safety systems of these devices: birth weight; gestational age; sex; occupancy of the NICU at the time of the audit; and location of the patient.
2.3. Work Organization in the NICU
In the NICU, the ratio patient/nurse is 2.1 during all the shifts. The nurses are registered nurses with university education and neonatal experience. Paediatric nurse specialists were introduced 3 years ago in Spain. In relation to the doctors, 7 neonatologist and 7 paediatrics residents are in charge of the 19 intensive care beds and 24 medium care beds. During the duty (from 3 pm to 8 am) 2 neonatologist consultants and 1 resident are in charge of the neonatal unit including attention to delivery room. The nurses are responsible for programming monitor and pulse oximeter alarms following the unit’s protocols or medical instructions, for controlling the water and heater temperatures, and for changing respirator and CPAP tubing. The respirator and CPAP parameters and alarms are programmed by doctors.
2.4. Ethical Issues
This study involved quality strategies for improving patient security and thus did not require institutional review board approval. The study consent was obtained from the head doctor and the head nurses of the unit.
2.5. Analysis Plan
Continuous variables are presented as mean ± SD and categorical variables as absolute and relative frequencies. The reproducibility of the observations made by the two study investigators was estimated with the kappa coefficient. The statistical significance of the comparison of proportions was determined using chi-square or Fisher’s exact test from contingency tables. Comparisons of the distributions of ordinal and continuous measurements were made using the Wilcoxon-Mann-Whitney test or Student’s t-test, as appropriate. Multivariable logistic regression analysis was used to estimate the strength of association between appropriate overall use and several covariables such as gestational age, birth weight, sex, location in the NICU, normal working day or nonworking day, shift, month, and occupancy. Results are presented as odds ratios and 95% confidence intervals (CI).
3. Results
During the study period, a total of 383 assessments were made of the monitoring and respiratory support equipment (see Figure 1). The overall interobserver kappa coefficient between the 2 investigators carrying out the audits was 0.93. The rate of appropriate use of the equipment safety systems and other aspects of the technology as a whole relating to patient safety was 33.68% (129/383). Table 3 shows the rates for appropriate use of the items assessed in each device.
The variables with the worst compliance rates were in the pulse oximeters, the programming of the oxygen saturation (SpO2) alarms at 64.89% (61/94); in the monitors, the respiratory rate alarm at 18.18% (12/104); in the conventional mechanical ventilation respirators, the volume/minute alarm at 33.96% (18/53); in the high-frequency oscillatory ventilator, the MAP alarm at 51.56% (33/64); and in CPAP, the change of tubing at 48.53% (33/68).
Table 4 shows the results for appropriate overall use of the different pieces of equipment in relation to patient characteristics and location. Table 5 shows the results in relation to time characteristics. Significant differences were found in the multivariate analysis, with improved overall usage during weekends and holidays, OR 1.85 (1.12–3.06, ), and on the late shift, OR 1.59 (1.03–2.4, ). The occupancy rate when appropriate use was found was 95.56% and 93.02% when inadequate use was detected (). There were no significant differences in any of the other variables studied.
Critical incidents were found in 9 (2.34%) of the 383 devices reviewed. These were pulse oximeter not working in 2 patients requiring monitoring; alarms not programmed in invasive blood pressure monitoring in 3 unstable patients; and no water in the heater of 4 respirators.
4. Discussion
This prospective observational study shows that only a third of evaluated monitors and respiratory support equipment had all the parameters well programmed. Contrary to what might be expected, weekend and holidays and late shifts, when there are fewer neonatologists in the NICU, were the periods when we found that the evaluated equipment parameters were best programmed. During the study period the NICU occupancy rate was higher than 90%. Patient characteristics and location in the unit did not influence the appropriate use of the monitors and respiratory support equipment.
To our knowledge, this is the first published study with data on real-time audits of monitoring and respiratory support equipment in a level III-C NICU. Compared to the large number of published studies on other aspects of neonatal patient safety [10–12], there are surprisingly few literature references reporting errors directly related to the equipment used [3, 13, 14]. For this reason, it has not been possible to compare many of our results with other studies. When we designed the study, we believed that our results were going to be much better than those we finally obtained after the audits.
One of the most disturbing findings in relation to the monitoring equipment was the poor protocol compliance for programming SpO2 alarms, results similar to those found by other authors [3, 13]. In most cases, this consisted of not programming the upper SpO2 alarm limit in patients on supplemental oxygen, which exposes the patient to a greater risk of hyperoxia and the widely described adverse effects on lung, retina, and brain [15–17]. There is currently a great deal of concern about what the optimal SpO2 ranges are for premature infants and numerous studies are being conducted in this area [16–19]. However, all the efforts will be of little effect if, on implementing the recommendations in different NICUs, the SpO2 alarm limits are not properly set.
Despite the high rate of inadequate programming of SpO2 alarms, our BPD and grade 3 or higher ROP rates are not high. Nevertheless, this does not justify the poor compliance in programming these alarms in view of the other repercussions on the newborns in our unit.
One of the most surprising results was finding a higher percentage of appropriate overall use at the weekend and during the late shift. It is not easy to explain this result. At the weekend and during late shifts, there are only two neonatologists on duty. The number of nurses is always the same on all shifts. Programmed procedures are usually not performed. There are, therefore, fewer interruptions, allowing a greater degree of concentration on the part of the staff. The nurses probably work better in the quieter, calmer atmosphere on the weekend. These aspects need to be confirmed since it does not seem logical that the use of technology should be better when there are fewer doctors working.
Also remarkable is that appropriate use of the equipment is better when there is higher occupancy. A tendency towards better use would be expected with lower occupancy and not the other way round. In our NICU, the occupancy rate is always very high and although the difference is statistically significant, it may not be clinically relevant.
We also wanted to find out whether the use of technology was different in the months of July/August/September compared to the rest of the year, as this is considered a high-risk period for adverse events due to the fact that, in Spain, it is a vacation period during which less-experienced nurses are recruited. For example, our group has reported a higher rate of unplanned extubation over the summer months [20]. However, no differences were found in the use of technology.
With regard to adverse events, higher rates have been reported in the most premature patients and those weighing less than 1500 g [21]. In our study, these patient characteristics had no influence on the use of the equipment safety systems or on other safety-related aspects of the technology evaluated. The incidents found in 2.34% of the devices reviewed were considered critical for the serious consequences they could have had for the patient. This rate is not insignificant and NICUs are highly complex units, in which high-risk situations continually occur. In all the above cases, the patient’s care provider was immediately informed to resolve the problem.
A major problem in the NICUs is the large number of alarms used which produce a high level of noise. There are reports that exposure to high levels of noise in newborns, particularly when premature, can have potentially negative effects on their stability and their future neurodevelopment [22, 23]. Moreover, there are studies which show that 72–99% of the alarms that go off are not actually related to any real patient event [24]. This can lead to a situation where NICU staff suffer from “clinical alarm fatigue,” the term used to describe the phenomenon of medical providers becoming too desensitized due to the constant noise emitted from monitors [25, 26]. It could also lead to staff becoming less rigorous and not programming alarms which they believe often sound without signifying a real clinical problem for the patient. In light of the above, there is an attempt nowadays to design “smart alarms” which manage information in an integrated manner using algorithms, thereby improving the sensitivity and specificity of the alerts [27].
One limitation to our study was that although the staffs were unaware of the objectives of the observation they were, however, aware of being watched and this may have influenced their usual practices. Nevertheless, we do not believe that this aspect influenced the results. When the different quarters of the year were compared, no changes were found in staff behaviour over the course of the study period. Another limitation is that the night shift was not included in the observations. Perhaps our results cannot be generalized because the organization of other units and the degree of professional training may differ between countries and units, but these results may encourage other units to audit the technology they use in an effort to identify areas for improvement.
The results of this study have been disseminated to the NICU staff and different strategies have been introduced to improve the appropriate use of equipment. Probably, as in other aspects concerning safety, checklists are useful to evaluate the appropriate settings of monitors and respirators. The companies that design monitoring and respiratory support systems are trying to incorporate an increasing number of alerts to ensure proper use of the equipment and to improve the safety of the newborn. However, our study shows that a large number of these safety systems are not properly used and do not benefit the patient. To improve patient safety, we should seek to identify which alarms are really needed and where the difficulties lie for the correct alarm programming in the monitors and support respiratory equipment.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors thank the Spanish Collaborative Maternal and Child Health Research Network (SAMID) for the support and Ann Marie Strigari and Abraham Pavón for the editing assistance.