Abstract
Introduction. Multidisciplinary simulation-based team training (STT) provides a powerful training method to train technical and team skills during emergencies. Effectiveness of STT depends on transfer of learned skills to clinical practice. In this study we examined three important prerequisites to enhance transfer from STT into clinical practice, intervention readiness, realism, and performance self-efficacy. Methods. For the quantitative part of the study, 131 participants (pediatric nurses and physicians) were asked to fill out an online questionnaire before and after training. For the qualitative part of the study we organized three one-hour focus group sessions in which participants were interviewed on attitude, realism, and self-efficacy. Results. Providing adequate preparation material and extensive debriefing of scenarios is important in creating this positive learning experience. The perspective of realism depends strongly on setting and learning goals. During STT team assembly and role playing can become more important to participants, while physical aspects become less important. Performance self-efficacy for all participants increases significantly regarding team skills. Conclusions. STT can be a very positive multidisciplinary learning experience, which creates the possibility of enhancing confidence, skills, and team performance within the clinical context. STT combines three important prerequisites for transfer of training to take place.
1. Introduction
Multidisciplinary simulation-based team training (STT) provides a powerful training method [1–4], which is incorporated into the annual educational curriculum of all health care professionals (nurses, residents, and medical staff) at our children’s hospital.
There are several reasons why simulation learning appears to be an excellent tool in teaching team skills. STT takes place in a learning environment that closely matches the clinical setting, which has been identified as an important factor on memory retrieval and transfer [5]. Secondly, this type of training uses different educational tools, including hands-on practice and instant debriefing, that could stimulate activation of prior knowledge [6].
The key to success of these training programs is the extent to which trainees use their training on the job, leading to an improvement in job performance [7], in other words transfer of skills. The literature on simulation-based training methods shows evidence that skills acquired by simulation-based training seem to be transferable to the operative setting [8, 9]. There is also evidence of transfer of nontechnical skills in simulation-based trauma team training and pediatric emergency training [10–12]. However none of the studies included in these reviews were randomized, controlled, and blinded, which limits the strength of conclusions. Furthermore evidence for participant- and simulation-relatedprerequisites, necessary for transfer from learning environment to patient care, is sparse [4, 12–14] and results in terms of actual benefit to safety and quality of care are contradictory [10, 11]. As STT asks high investments (in terms of time and money) a good understanding of the factors that enhance trainees’ use of learned skills and knowledge on the job is needed in order to use and further optimize training programs.
We focused our study on three transfer factors that can be influenced by the intervention of STT itself, intervention readiness, simulation realism, and self-efficacy [3]. Individual learner characteristics and personal traits such as conscientiousness, emotional stability, openness to experience, and job attitude are also predictors of transfer [15] but are difficult to influence by the training itself and were therefore not included in this research.
1.1. Intervention Readiness and Learners Attitude
A positive attitude towards training or the so-called “intervention readiness” was found to be essential to create learning moments [16]. After a positive learning experience, participants might strive to structure and redesign their working environment in order to enable transfer of training [13, 17], directly improving patient care.
1.2. Realism and Training in Context
STT seems to have a high level of content validity compared to other training methods [1, 2]. However, published data to date have not clearly supported the concept that simulated physical features, for example, the ability to communicate with the manikin or perform skills and monitor vital signs, distinct from other elements of a simulation experience, are of particular educational benefit [18, 19]. It is not uniformly the case that more (physical) realism allows for better attainment of educational goals [20].
1.3. Performance Self-Efficacy
Self-efficacy is defined as “a person’s belief in their capabilities to organize and execute the courses of actions required to produce given attainments” [3]. Self-efficacy influences choices of behavior, especially the degree of effort and perseverance invested in a task. A strong sense of self-efficacy correlates with positive thinking, allowing a person to visualize success and control anxiety.
We conducted a prospective quantitative study to investigate change in attitude, perception of realism, and self-efficacy among pediatric nurses and physicians before and after STT. After analyzing the quantitative data we conducted three structured focus group sessions among neonatology nurses, pediatric nurses, and physicians to gain more insight in the educational needs of our participants, with the ultimate goal to promote transfer of skills to patient care. Measurement of actual staff performance was not one of our study aims.
2. Materials and Methods
2.1. Participants
This study was conducted among health care professionals of the Pediatric Department at the Radboud University Medical Center. The study group consisted of neonatal intensive care nurses, pediatric care nurses, and physicians (pediatric residents and consultants) working as well in the pediatric unit as in the Neonatal Intensive Care Unit (NICU). We chose to include both ICU and ward professionals while both groups participate in STT with a similar set-up (performance on a high fidelity simulator with video debriefing afterwards) but work in a different environment, whereas the physicians have to perform their skills in both ICU and pediatric wards. Further descriptive statistics of our participants are shown in Table 1.
For the quantitative part of the study every participant that entered our structured pediatric simulated team training program during a four-month period was included after informed consent and asked to fill out an online questionnaire before and after training.
For the qualitative part of the study a sample of nine neonatology nurses, nine pediatric nurses, and six pediatricians (consultants/residents) volunteered to participate in three two-hour focus group sessions in which they were interviewed on attitude, realism, and self-efficacy items before and after simulated team training.
2.2. Setting
The study was conducted in the Skills and Simulation Unit of our center, which is equipped with cameras and microphones. Scenarios took place using a METI Pediasi manikin for the pediatric teams and an enhanced Laerdal Babysi manikin for the neonatology teams. The scripted scenarios had duration of approximately 10 minutes per session and covered different cases with medium complexity of critically ill children in a pediatric unit or neonatology ward (e.g., anaphylaxis, hypoglycemia, and sepsis). Afterwards an extended debriefing of one hour took place focusing on team performance and nontechnical skills. The “performing team” consisted of four persons (2 nurses, 1 resident, and 1 staff member), while the other participants watched the team performing on a video screen in another room. After the first session, participants changed places. Scenarios were written by training experts who are pediatricians involved in acute care for children at our centre.
2.3. Materials
The online questionnaire was designed and tested by an expert panel of three educational scientists together with three experienced training instructors in the field of acute pediatric care. Answers for the items on the constructs “Attitude” and “Value of Realism” were given at a six-point Likert-scale. During the design stage, Cronbach’s alpha was used to quantify the internal reliability of the questionnaire. The reliability of each construct was good, with Cronbach’s alpha ranging from 0.75 to 0.9.
A visual analogue scale (VAS) of 100 mm was used to score the items of “Self-efficacy” [21].
2.4. Procedure
After obtainment of informed consent, participants had to fill out the first online questionnaire before the training. The training program lasted for four hours. Then, the training participants filled out the online questionnaire for the second time. One month later the participants filled out the online questionnaire for the last time, only for the self-efficacy construct.
Confidentiality was assured and made clear to the participants as required by ethical protocol.
After analyzing quantitative results, a semistructured interview guide to clarify quantitative results was constructed by an expert panel of two educational scientists and two pediatricians. Three focus group interviews were organized 2 to 12 weeks after participants had participated in STT. The structured sessions were guided by a moderator and contained questions about perceived (dis)advantages of STT, their attitude towards this type of training, and importance of realism aspects and self-efficacy regarding resuscitation skills (technical and nontechnical) before and after training.
2.5. Statistical Analysis
Quantitative analyses were done with the computer statistics program IBM SPSS Statistics, version 18.0. Paired sample t-tests were done for each item for each group of participants to look for differences between the means on pretest and posttest scores for attitude, realism, and self-efficacy. A two-sided value < 0.05 was considered to be statistically significant.
All interviews were transcribed verbatim and analyzed with qualitative data analysis software ATLAS Ti, version 5.2 (Scientific Software Development GmbH, Berlin). We used an open coding approach, to look for patterns and clarifications for our quantitative results during the interviews, through comparing, categorizing, and examining data. During the open coding process we used a provisional list of codes based on the interview guide. Two researchers discussed the results to reach final consensus on significance and relevance of phrases and sentences.
3. Results
We included a total of 131 pediatric health care professionals in our study after informed consent was obtained. The group consisted of 33 neonatology nurses, 50 pediatric nurses, 11 pediatric residents, and 37 pediatric consultants. They all participated in a neonatal or pediatric STT course depending on their primary working environment (nurses) or rotation (residents). Further descriptive statistics such as gender, working experience, and experience with simulation team training are shown in Table 1.
3.1. Intervention Readiness and Learners Attitude
3.1.1. Quantitative Results
All groups rated the training session significantly safer () after SST compared to before SST. Nurses rated STT less stressful after the training compared to before the training .
Participants with training experience, like most of the neonatal nurses, still perceived a level of stress similar to novice participants (Table 2). Pediatric nurses and physicians found the training sessions to be more enjoyable and inspiring than expected. Neonatology nurses, who had the most experience with STT, also rated the sessions enjoyable and inspiring but showed no significant difference in these areas before and after training (Table 2).
3.1.2. Qualitative Results
During the focus group session neonatal nurses stated that during video-assisted feedback a structured debriefing decreases the level of perceived stress afterwards but it could not prevent feeling of being stressed at the start of the scenario. The stress factor of not knowing what to expect before training in a completely new environment was brought up by all groups as the most important reason for stress.
A decent and well-structured debriefing session lead by good, qualified trainers who focus on team competences and creating an environment, in which we can give constructive feedback to each other, makes it feel safer and less stressful afterwards.
Adequate information and more possibilities for preparation, such as an e-learning program to get up to date with certain procedures, were mentioned in all three focus group sessions, as way to put participant more at ease at the start of a training session.
If I could read back relevant guidelines or view important procedures beforehand I would know better what is to be expected of me. I think I would feel less stressed and it would enhance my learning experience.
Still a level of experienced stress is necessary for participants, according to remarks made by the physicians during their focus group session, to be able to perform at a realistic level.
If I wouldn’t experience any stress during a scenario, it wouldn’t be realistic. Experiencing stress up until a certain level is not necessarily negative. If I’m paged to see a critically ill child in the emergency room, you automatically feel adrenaline running, because you want to be focused and feel responsible to bring the situation to a good end.
Training team skills, like leadership, effective communication, and situational awareness, were found to be highly relevant and inspiring and the most important reason to promote STT to colleagues. Learned skills were highly applicable in their real-life jobs.
If this type of training wasn’t obligatory I [sic] still would come to this type of training, even in my own time. It is a very valuable additional tool to what you can learn in clinical practice and the only type of training that focuses on teamwork. For me this is the most important reason to recommend this type of training to other residents and staff members.
3.2. Realism Aspects of Simulation
3.2.1. Quantitative Results
All groups rated the value of several physical realism aspects of the simulation higher before training than afterwards (Table 3). Physicians found it more important to perform in their own professional role as doctors () and others to perform in their own profession as nurses () as opposed to other staff members or course instructors playing the role of nurses during the scenario.
Results of the investigated realism aspects of STT show that after training the use of realistic scenarios, the possibility to perform skills real time, and performing the role of your own profession are mentioned most frequently by participants. The least important aspects of realism were found to be appearance of the room and manikin and communication with the manikin (Table 3).
3.2.2. Qualitative Results
Physical aspects, like appearance of the manikin, were found to be more important at the start of the scenario, while role performing took over in the second part. Physicians suggested that nurses need higher levels of physical realism, because of the fact that physicians are called in the scenario when there’s already an urgent state of acting. This was confirmed by nurses’ comments on the difficulty of starting interaction with the patient without real physical interaction.
Essential visual aspects for early recognition like arousal, consciousness, color and breathing difficulty are missing. At the start of the scenario you feel as if you miss parts of your clinical view. But when the patient deteriorates and the action starts it all becomes very real.
At the start of the scenario nurses are alone with the patient and there’s more need for interaction with the manikin and higher levels of physical realism to be drawn into the scenario…For nurses it is more important to work in their own environment, in a familiar place with familiar materials. Physicians have to work at different units, so to train in a familiar environment is less important for us.
Other aspects that were brought up to increase the level of realism were team assembly and a parent role. Finally, all groups suggested that training scenarios at different difficulty levels would be desirable, starting with more common situations to get familiar with the concept of STT.
3.3. Self-Efficacy of Team Skills
3.3.1. Quantitative Results
The self-efficacy for performing team skills or nontechnical skills (NTS) during a pediatric emergency increased significantly () for both groups (physicians and nurses). However, our data showed a significant difference in pretraining data between nurses and physicians, with nurses scoring themselves at a higher level before training (SE score > 75 overall, range 0–100) than physicians (Figure 1).
A comparison of participants’ data, filled in one month after training, with the posttest data shows us no significant decrease in SE scores for neonatology nurses () and a significant increase in SE scores for pediatric nurses and physicians ( values: 0.037 and 0.05, resp.). This implicates that an increased self-efficacy after training lasts for at least one month (Figure 1).
3.3.2. Qualitative Results
During focus group sessions physicians suggested that nurses experienced difficulty in rating themselves on team skills before training, because they have little experience in managing acutely ill children by themselves. Physicians and nurses indicated that physicians would be more suited to indicate their level in advance, because they train more often and have more experience in real-life crisis situations.
Physicians have more training experience in actually performing emergency skills and therefore have probably better insights on what they can or can’t do in real life.
Residents indicated that they perceived STT to be better suited as a learning tool in training team skills than technical skills, because the team and content of the scenario are very realistic, while the manikin can only simulate the emergency up to a certain extend.
The challenge of this type of training for physicians is not how to judge an airway, but learning how to coordinate a team. These things are very important in emergency situations, but we never seem to get the chance to train these things during actual clinical practice…These situations are either too rare or we simply don’t get a shot on trying without a staff member intervening [sic], so I’m anxious to find out how I will perform in real life during these type of situations [sic].
4. Discussion
In this study we tried to obtain insight into STT related factors that can be of influence in optimizing multiprofessional STT and enhancing transfer of skills into daily clinical practice.
The questionnaire at different time intervals and focus group interviews gave us an insight into how participants perceive the team training method before and after participating in STT on aspects of safety, relevance, and enjoyment.
4.1. Intervention Readiness and Learners Attitude
In general participants gave high scores to relevance and enjoyment of the training. Perceived stress caused by introducing a new training method decreases for all groups after participating in training. Existing literature suggests that receiving more preparatory information prior to a stressful event can reduce negative responses to stress even more [22]. Even so most of our neonatology nurses already had participated in STT and still experienced stress levels similar to the other less experienced groups. Fear of the unknown does not seem to be the only factor causing stress. The anticipation of an acute problem going to occur during the course of the scenario brings participants to a high state of alertness or even anxiousness. According to our participants, this adds to the realism of the scenario, because during the management of acutely ill patients it is normal to perceive a certain amount of stress. Other studies have found that a certain level of emotional stress during simulation correlates with enhanced performance of life support skills after the course [23].
On the other hand insufficient stress coping strategies may lead to a defensive posture of the participant leading to reduced learning, avoidance, and lack of confidence to perform certain skills [16]. During the interviews participants told us that creating a safe learning environment and extensive debriefing by an experienced instructor adds to the decrease in stress and promotes learning. Creating a positive learning attitude by providing adequate information and possibilities for preparation, such as providing candidates with pretraining e-learning modules or up-to-date guidelines on the specific training objectives, is therefore very important [1, 24] to create an optimal learning mode for STT.
4.2. Realism Aspects of Simulation
There is much debate about the level of realism needed to create an optimal learning experience for participants. A certain level of realism is needed to persuade participants to perform actions and interact with their environment, as if they were in their own work place. However, to our experience there is a limit to the extent of realism needed to promote transfer of training. Physical aspects are important to activate participants [20], but performing actions in their “real-life” role and communication with team members seem to be more important once the scenario has started. Our results show that after participating in STT the importance of physical aspects decreases, while team aspects become more important. These findings are in line with earlier studies that suggest adjusting the realism level of the training to the learners’ needs [2]. This means that the realism level should be chosen thoroughly after determining learning goals.
4.3. Self-Efficacy of Resuscitation Skills
Overall there is a significant increase in self-efficacy regarding skills needed to assess and treat an acutely ill child after participating in STT. The results of our follow-up data show that the increase in self-efficacy lasts for at least one month, with a tendency to further increase in some skills. This is supported by another study on participation of nurses in simulation training [25], which suggests that nurses will have an increased sense of self-efficacy over a longer period of time when they have the possibility to apply new learned skills into practice. This point also came up during focus group sessions. Nurses made remarks about the relevance of team skills during non-emergency situations. According to them, skills learned during STT, such as closed loop communication and structured patient handovers, could be frequently applied during routine clinical practice. These insights promote transfer of skills into practice.
We were surprised to find that SE levels regarding team skills among nurses were relatively high in advance (self-efficacy score > 70 on VAS). During the focus group sessions participants came up with a possible explanation. Physicians receive more skills training and have more experience in treating acutely ill children. This is why they probably have a more realistic view on the difficulty of assessing children in these situations. Nurses, on the other hand, are handed many new tools to use in assessing these children and are very confident in applying this knowledge during an actual clinical situation, even after only one training session. Also there is a difference in responsibility and complexity of tasks between nurses and physicians during an emergency event. The main role of the nurses is to signal deterioration and to initiate an appropriate first response, whereas the physicians need to express leadership skills, gain overview of the situation, and make decisions on a working diagnosis and plan of action. These so-called nontechnical skills are very complex and self-efficacy regarding these skills is far more difficult to measure than in the case of specific well-defined resuscitation tasks [21].
5. Limitations of the Study
The study group consisted of neonatal intensive care nurses, pediatric care nurses, and physicians working as well in the pediatric unit as in the NICU. The nurses’ scores on attitude regarding STT and retention of self-efficacy regarding resuscitation skills seem to depend on their working environment and experience. Participating physicians work on different wards and have very different levels of working experience. This limits conclusions regarding specific learning needs for pediatric residents versus consultants. Scenarios took place using a METI Pediasi manikin for the pediatric teams and an enhanced Laerdal Babysi manikin for the neonatology teams. Although both manikins are high fidelity simulators with a similar level of realism, the use of two different models could bias the results on perceived physical realism within both groups.
For the qualitative part of the study we used only a small sample of participants ( out of 131). One of the two interviewers during the focus group sessions was also an STT instructor, which could have deepened the understanding of participants’ comments, but also participants may have identified benefits of STT to please the interviewer. To limit bias in interpretation during the analytical process this was done by an educationalist not otherwise involved in our training program. Furthermore STT mimics but does not equal clinical performance. Therefore results cannot automatically be transferred to daily practice. Future study could evaluate actual clinical performance and transfer of skills using video recording and checklist scoring of critical events.
6. Conclusion
We aimed to investigate how important prerequisites for transfer of skills are valued by participants before and after training and how they can be influenced to optimize STT. Intervention readiness and learners attitude can be positively influenced by proper preparation and adequate and safe debriefing afterwards. Perceived realism by participants during STT mostly depends on performing in their own professional role and training with real team members in their specific role. When training team skills one should invest in training with a complete team, rather than investing in the physical aspects of the manikin or environment. Self-efficacy regarding resuscitation skills (both technical and nontechnical) increases after STT. However, there is a significant difference regarding self-efficacy in advance, with nurses rating themselves significantly higher on technical and team skills than physicians, properly because of different responsibilities and previous experience. This makes the design and implementation of a multidisciplinary team training challenging. However learning from each other’s experience and responsibilities according to role is what makes this type of training valuable. Overall STT can be a very positive multidisciplinary learning experience, which creates the possibility of enhancing confidence, skills, and team performance within the clinical context.
Ethical Approval
The institutional Review Board of the Radboud University Medical Center decided that ethical approval of this study was not required according to Dutch Medical Ethics Law. Informed consent was obtained and anonymity of participants was guaranteed by the investigators.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Authors’ Contribution
E. H. A. J. Coolen, T. P. F. M. Klaassen, J. L. Loeffen, C. R. M. G. Fluit, and J. M. T. Draaisma designed and coordinated the study. T. P. F. M. Klaassen drafted, tested, and implemented the questionnaire. E. H. A. J. Coolen and T. P. F. M. Klaassen designed and carried out the focus group sessions. E. H. A. J. Coolen and T. P. F. M. Klaassen carried out data analysis and interpretation. E. H. A. J. Coolen drafted the paper. All authors read and approved the final paper.