Performances of low level hospital health caregivers after a neonatal resuscitation course

Background High fidelity simulation has been executed to allow the evaluation of technical and non-technical skills of health caregivers. Our objective was to assess technical and non-technical performances of low level hospitals health caregivers who attended a Neonatal Resuscitation course using high fidelity simulation in a standard-setting scenario. Methods Twenty-three volunteers were asked to manage a simple scenario (infant with secondary apnea) after the course. Technical and non-technical skills were assessed by using previously published scores. Performances were assessed during the scenario and after 2 months by filmed video recordings. Results Sixteen (69.5%) participants failed to pass the minimum required technical score. Staff experience and participation in previous courses were associated to higher score in technical and non-technical skills, while working in level I or II hospitals did not affect the scores. Previous experience in neonatal resuscitation requiring positive pressure ventilation was associated to better non-technical performance. Technical and non-technical scores were significantly correlated (r = 0.67, p = 0.0005). Delayed and direct evaluation of technical skills provided the same scores. Conclusions A neonatal resuscitation course, performed by using a high fidelity simulation manikin, had a limited impact on technical and non-technical skills of participants working in low level hospitals. Training programs should be tailored to the participants’ professional background and to the more relevant sessions.


Background
Approximately 10% of all newborns need some resuscitation procedures to start breathing, 3% requires positive pressure ventilation (PPV) and a small percentage (1%) requires more advanced interventions such as intubation, cardiopulmonary resuscitation and medications [1]. The Neonatal Resuscitation Program (NRP) is widely used for training of personnel involved in the care of the newborn at birth [2]. The skill assessment needs to be regularly done to monitor the staff training [3]. NRP courses have been showed to be effective in improving knowledge, manual skills and event management (eg. MegaCode) of participants [4,5]. The high fidelity simulation is a new training tool that seems to have potential advantages over the traditional teaching [6]. The higher emotional involvement of the participants and the possibility of debriefing after simulation, together with teamwork, are the strengths of this method [7,8]. A previous study, conducted in a level III hospital by using a high fidelity simulation manikin, showed a significant improvement on technical and non technical skills of participants [9]. It is unknown which could be the impact of a similar course, using the same evaluation method and the same setting, on health caregivers working in level I and II hospitals. The aim of the present study was to assess technical and non-technical performances of level I and II hospitals health caregivers who attended a NRP course by using high fidelity simulation in a standard-setting scenario. Furthermore, we assessed the duration of resuscitation procedures, the correlation between technical and nontechnical skills, the differences in scoring between direct and remote evaluation.

Participants
Twenty-three volunteers, 10 pediatricians and 13 pediatric nurses, were involved in the study. Of them, 11 were from level I (>36 weeks' gestation, low-risk pregnancies; normal nursery) and 12 from level II (>32 weeks' gestation pregnancies; neonatal semi-intensive care unit) hospitals of Campania region, Italy (Table 1). Written informed consent was obtained from each participant. The study was approved by the Ethics Committee of Santobono-Pausilipon Hospital, Naples.

Characteristics of the neonatal resuscitation course
The course was held at the Santobono-Pausilipon Hospital in Naples on December 2013 and it was taught by four national certified NRP instructors with more than 5-years experience in simulation. This course lasted two days and consisted of didactic sessions followed by practical handson skill stations, according to the principles of NRP [10]. Theoretical lessons included the following topics: physiology and pathophysiology of birth, initial steps of neonatal resuscitation, PPV (including endotracheal intubation and laryngeal mask), chest compressions, medications. After each class, a 10-question multiple-choice test focused on that specific issue was performed by each participant and then it was collectively corrected. Detailed resuscitation maneuvers (initial stabilization, PPV delivered with face-mask or laryngeal mask and self-inflating bag or T-piece resuscitator, endotracheal intubation, chest compression, cannulation of the umbilical vein and administration of drugs) were trained in practical handson skill station. Each participant executed the procedure three times. The learning level of participants was evaluated by using a scenario (infant with secondary apnea needing PPV) at the end of the course.

Simulation
A high-fidelity simulator (SimBaby; Laerdal Medical AS, Stavanger, Norway) was used to perform resuscitation procedures. A high fidelity simulation expert, placed in a hidden location, changed the settings of the manikin in response to resuscitation procedures. Vital signs (heart rate and SpO 2 ) were visible on a monitor placed in front of the operators. A facilitating nurse was always present in the room to help the participants only after a clear help request. Before starting the test, each participant was left for 10 min in the simulation setting to become familiar with the simulator and with all the tools. Then, the clinical case was presented: "You are called to the delivery room for a term pregnant woman with prolonged labor. The fetus became bradycardic just before birth, with a slow labor progression. Amniotic fluid was clear. At birth the infant presented apnea, depressed reflexes and tone, thus he was placed under the infant warmer". Participants were expected to prepare and check the equipment and to start resuscitation performing the first 30 s steps: newborn assessment, correct placement under heat radiant source, suction only if necessary, drying, stimulation of the feet and/or of the back. Furthermore, they should assess the heart rate (HR) with a stethoscope and place the pulse-oximeter on the right hand [11]. The infant was still presenting pale, with bad tone, apnea and low reflexes, HR of 30 beats per minute (bpm). The correct action was therefore starting an effective PPV for the next 30 s with an initial FiO 2 of 0.21. The infant would show adequate chest movements only if the ventilation was correctly performed. The second evaluation showed a HR of 90 bpm, with SpO2 = 65%. Thus, the correct action should have been to continue the PPV and, if well performed, would have lead the simulator to vigorously cry after 90 s from the beginning of the scenario, with HR of 140 bpm, valid breath and SpO 2 of 75%. The correct action should have been to stop ventilation and to monitor the infant after the resuscitation procedures.

Scoring and evaluation
The modified method of Rovamo and colleagues, consisting of a 22-items checklist, was used to assess technical skills of participants [9]. We focused on the first 60 s after Data expressed as number of subjects birth, the so called "Golden Minute", to score resuscitation skills; therefore all items about intubation, chest compressions and medications were excluded from the checklist. Each of the actions was graded as 1 = yes or 0 = no. The detailed check-list and the principles of grading are shown in Table 2. The cut-off point to pass the technical test was determined as 85% of correct items (i.e., 19 or more correct items), based on a previously published study [9]. Non-technical skills were evaluated by a 9-item checklist ( Table 3) based on a previously published study, using a Likert scale from 1 to 5 [9].
An external observer, who was blind to the profession (physician or nurse) of participants, directly scored the procedures during the scenario. In addition, DVD filmed video recording performances were evaluated after two months by another expert in resuscitation training [12,13].

Statistical analysis
Continuous data were expressed as median and interquartile range (IQR). Nonparametric combination (NPC) of dependent permutation test methodology was used to perform statistical analysis [14]. This approach is based on a conditional testing procedure which is less demanding in terms of underlying assumptions than parametric tests and is appropriate for small sample size. The correlation between technical and non-technical

Results
The hospital level of care, working and simulation experience of participants are shown in Table 1 (Table 5). Participants with longer working experience and those who attended previous courses had a higher technical score (p = 0.03 and p = 0.004, respectively, Table 5). Participants who had experience in PPV and those who attended previous courses were faster in performing resuscitation procedures corresponding to items 7 to 14 (p = 0.02 and p = 0.007, respectively; Table 5). In addition, participation in previous courses was associated with short execution time of items 1-6 (p = 0.03; Table 5).

Non-technical score
Overall, median non-technical score was 38 (IQR 34-42) over a maximum of 45 points (Table 4). Participants with longer working experience had a higher non-technical score than those with less experience (p = 0.04, Table 5). Previous experience in neonatal resuscitation requiring PPV and the participation in previous courses were also associated to a better non-technical performance (p = 0.0002 and p = 0.007, respectively; Table 5). Technical and non-technical scores were significantly correlated (Spearman's rho 0.67, p = 0.0005). The evaluation of technical skills using DVD filmed video provided the same scores of the direct evaluation for each participant. Non-technical skills were not assessed using DVD filmed video.

Discussion
This study evaluated the effectiveness of a neonatal resuscitation course using high fidelity simulation on practical and organizational skills of health caregivers who are routinely involved in the management of newborns at birth. Our data showed that most participants (16 out of 23 subjects) did not reach the minimum score to pass the test on technical skills. Incorrect head positioning, delay in controlling mask leak and inadequate inflation pressures of the self-inflating bag were the main reasons for the failures. Based on the 2010 version of the Neonatal resuscitation guidelines, attendants needed more than recommended time to execute the steps of neonatal resuscitation: the median execution time for initiating PPV was 40 s rather than 30 s [2]. However, based on previous studies, the 2015 guidelines recommend to start PPV within 60 s after birth, that fits with our participants' performance [1]. Every effort, however, should be done by instructors to emphasize the importance of avoiding unnecessary delay in initiating ventilation [16]. The effectiveness of our course was similar to that found in a real-life neonatal resuscitation setting [17], but was lower to that reported in a previous study using the same assessment instrument, but in a different group of subjects [9]. In fact, participants in the Rovamo et al. study were from level III hospital, whereas our participants were from low level hospitals. In agreement with previous work, the experienced staff evaluated in this study reached higher scores in technical as well as non-technical skills than participants with less experience. Our data suggest that, more than the professional background per sè, experience of staff plays an important role in the quality of performance, while working in level I or II hospital does not seem to have a relevant impact. In Italy, most of births occurs in level I and II hospitals, with a limited number of births per centre per year and, consequently, with a very limited exposure to patients in  need of resuscitative maneuvers [18]. Due to limited exposure to real life neonatal resuscitations, training programs are crucial, but they should be tailored to the participants' professional experience. Courses dedicated to low level hospital staff should be structured in order to allow to low-experienced caregivers to effectively improve their skills. Since maneuvers related to PPV were the main reasons for failure, to reserve more time to PPV training session could be an effective instrument to reach this goal. A periodic staff retraining could be also of great importance to obtain better performances in simulated scenarios and in real life resuscitations. Many studies proved indeed the importance of the long-term follow-up to improve training programs [19,20]. Particularly, resuscitation knowledge and skills were demonstrated to deteriorate by 4 months after initial training [19]. Interestingly, pediatricians and nurses were equally successful in the technical-skills test (8 pediatricians and 8 nurses failed) despite the different level of education and knowledge. This aspect underlines the importance of direct and repeated experience, in real contexts or in training sessions. Since pediatricians and nurses working in low level hospitals are rarely involved in real-life neonatal resuscitation scenarios, it is not surprising that, although their different educational background, their performances in technical skills were quite similar.
The evaluation "in the field" requires the presence of a dedicated observer during the scenario who could "lose" some basic steps. Our results show that direct and delayed (by using the DVD filmed video recordings) evaluations provided the same score for each participant. The DVD video recording of the performance could also give the candidates a stronger feed-back about their own behavior during the procedure and could help to improve technical skills, as previously demonstrated [9,18]. Non-technical skills could not be evaluated using the DVD recording because the camera was focused on the single view of patient, not allowing to capture organizational aspects. The opportunity to review video recordings of simulated resuscitations could be used to underline the points of strength and weakness of the participants and could be an useful tool in retraining programs [18]. Organizational skills (nontechnical) including continuous evaluation of the patient, the uninterrupted action plan, vigilance and anticipation with adequate medical knowledge are absolutely important for the correct management of the patient, especially in emergency settings. During resuscitation procedures an effective communication is essential for a good teamwork and the identification of the leader is important to drive the overall procedure and to recognize and anticipate the tasks required. Furthermore, the leader has to periodically assess the clinical situation and its progress [21]. We found that working experience, previous direct administration of PPV to a neonate needing resuscitation and previous participation in a neonatal course were associated to a better performance in non-technical skills. We speculate that this was due to the different educational backgrounds and different tasks in everyday work. The direct correlation between technical and non-technical scores suggests that those who have better technical skills have also a better ability to effectively communicate and coordinate the team work. Our study has some limitations. We enrolled a limited number of participants working in level I and II hospitals from a single Italian region. In addition, the use of a manikin could be misleading for participants (e.g., time to obtain skin color changes) and could lead to loss of motivation. However, high fidelity simulation is the gold standard for education of those involved in emergency situations. Finally, in the present study participants were assessed immediately after the course. The long-term maintenance of the skills achieved during this training was beyond the scope of this study. Nevertheless, having a follow-up (4-6 months) evaluation of participants is essential to assess the real efficacy in improving both technical and non-technical skills using high fidelity simulation [22].

Conclusions
We assessed technical and non-technical performances of level I and II hospitals health caregivers after a neonatal resuscitation course by using high fidelity simulation in a standard-setting scenario. Our results show a limited effectiveness in comparison to a previous study involving level III hospital health caregivers. Experience, previous direct administration of PPV to a neonate needing resuscitation and previous participation in a neonatal course improved participants' performance. Technical and nontechnical scores were significantly correlated. There was no difference between "real time" and delayed (by using the DVD filmed video recordings) assessment of participants' technical skills.