- Open Access
Performances of low level hospital health caregivers after a neonatal resuscitation course
Italian Journal of Pediatricsvolume 42, Article number: 100 (2016)
High fidelity simulation has been executed to allow the evaluation of technical and non-technical skills of health caregivers. Our objective was to assess technical and non-technical performances of low level hospitals health caregivers who attended a Neonatal Resuscitation course using high fidelity simulation in a standard-setting scenario.
Twenty-three volunteers were asked to manage a simple scenario (infant with secondary apnea) after the course. Technical and non-technical skills were assessed by using previously published scores. Performances were assessed during the scenario and after 2 months by filmed video recordings.
Sixteen (69.5%) participants failed to pass the minimum required technical score. Staff experience and participation in previous courses were associated to higher score in technical and non-technical skills, while working in level I or II hospitals did not affect the scores. Previous experience in neonatal resuscitation requiring positive pressure ventilation was associated to better non-technical performance. Technical and non-technical scores were significantly correlated (r = 0.67, p = 0.0005). Delayed and direct evaluation of technical skills provided the same scores.
A neonatal resuscitation course, performed by using a high fidelity simulation manikin, had a limited impact on technical and non-technical skills of participants working in low level hospitals. Training programs should be tailored to the participants’ professional background and to the more relevant sessions.
Approximately 10% of all newborns need some resuscitation procedures to start breathing, 3% requires positive pressure ventilation (PPV) and a small percentage (1%) requires more advanced interventions such as intubation, cardiopulmonary resuscitation and medications . The Neonatal Resuscitation Program (NRP) is widely used for training of personnel involved in the care of the newborn at birth . The skill assessment needs to be regularly done to monitor the staff training . NRP courses have been showed to be effective in improving knowledge, manual skills and event management (eg. MegaCode) of participants [4, 5]. The high fidelity simulation is a new training tool that seems to have potential advantages over the traditional teaching . The higher emotional involvement of the participants and the possibility of debriefing after simulation, together with teamwork, are the strengths of this method [7, 8]. A previous study, conducted in a level III hospital by using a high fidelity simulation manikin, showed a significant improvement on technical and non technical skills of participants . It is unknown which could be the impact of a similar course, using the same evaluation method and the same setting, on health caregivers working in level I and II hospitals. The aim of the present study was to assess technical and non-technical performances of level I and II hospitals health caregivers who attended a NRP course by using high fidelity simulation in a standard-setting scenario. Furthermore, we assessed the duration of resuscitation procedures, the correlation between technical and non-technical skills, the differences in scoring between direct and remote evaluation.
Twenty-three volunteers, 10 pediatricians and 13 pediatric nurses, were involved in the study. Of them, 11 were from level I (>36 weeks’ gestation, low-risk pregnancies; normal nursery) and 12 from level II (>32 weeks’ gestation pregnancies; neonatal semi-intensive care unit) hospitals of Campania region, Italy (Table 1). Written informed consent was obtained from each participant. The study was approved by the Ethics Committee of Santobono-Pausilipon Hospital, Naples.
Characteristics of the neonatal resuscitation course
The course was held at the Santobono-Pausilipon Hospital in Naples on December 2013 and it was taught by four national certified NRP instructors with more than 5-years experience in simulation. This course lasted two days and consisted of didactic sessions followed by practical hands-on skill stations, according to the principles of NRP . Theoretical lessons included the following topics: physiology and pathophysiology of birth, initial steps of neonatal resuscitation, PPV (including endotracheal intubation and laryngeal mask), chest compressions, medications. After each class, a 10-question multiple-choice test focused on that specific issue was performed by each participant and then it was collectively corrected. Detailed resuscitation maneuvers (initial stabilization, PPV delivered with face-mask or laryngeal mask and self-inflating bag or T-piece resuscitator, endotracheal intubation, chest compression, cannulation of the umbilical vein and administration of drugs) were trained in practical hands-on skill station. Each participant executed the procedure three times. The learning level of participants was evaluated by using a scenario (infant with secondary apnea needing PPV) at the end of the course.
A high-fidelity simulator (SimBaby; Laerdal Medical AS, Stavanger, Norway) was used to perform resuscitation procedures. A high fidelity simulation expert, placed in a hidden location, changed the settings of the manikin in response to resuscitation procedures. Vital signs (heart rate and SpO2) were visible on a monitor placed in front of the operators. A facilitating nurse was always present in the room to help the participants only after a clear help request. Before starting the test, each participant was left for 10 min in the simulation setting to become familiar with the simulator and with all the tools. Then, the clinical case was presented: “You are called to the delivery room for a term pregnant woman with prolonged labor. The fetus became bradycardic just before birth, with a slow labor progression. Amniotic fluid was clear. At birth the infant presented apnea, depressed reflexes and tone, thus he was placed under the infant warmer”. Participants were expected to prepare and check the equipment and to start resuscitation performing the first 30 s steps: newborn assessment, correct placement under heat radiant source, suction only if necessary, drying, stimulation of the feet and/or of the back. Furthermore, they should assess the heart rate (HR) with a stethoscope and place the pulse-oximeter on the right hand . The infant was still presenting pale, with bad tone, apnea and low reflexes, HR of 30 beats per minute (bpm). The correct action was therefore starting an effective PPV for the next 30 s with an initial FiO2 of 0.21. The infant would show adequate chest movements only if the ventilation was correctly performed. The second evaluation showed a HR of 90 bpm, with SpO2 = 65%. Thus, the correct action should have been to continue the PPV and, if well performed, would have lead the simulator to vigorously cry after 90 s from the beginning of the scenario, with HR of 140 bpm, valid breath and SpO2 of 75%. The correct action should have been to stop ventilation and to monitor the infant after the resuscitation procedures.
Scoring and evaluation
The modified method of Rovamo and colleagues, consisting of a 22-items checklist, was used to assess technical skills of participants . We focused on the first 60 s after birth, the so called “Golden Minute”, to score resuscitation skills; therefore all items about intubation, chest compressions and medications were excluded from the checklist. Each of the actions was graded as 1 = yes or 0 = no. The detailed check-list and the principles of grading are shown in Table 2. The cut-off point to pass the technical test was determined as 85% of correct items (i.e., 19 or more correct items), based on a previously published study .
An external observer, who was blind to the profession (physician or nurse) of participants, directly scored the procedures during the scenario. In addition, DVD filmed video recording performances were evaluated after two months by another expert in resuscitation training [12, 13].
Continuous data were expressed as median and interquartile range (IQR). Nonparametric combination (NPC) of dependent permutation test methodology was used to perform statistical analysis . This approach is based on a conditional testing procedure which is less demanding in terms of underlying assumptions than parametric tests and is appropriate for small sample size. The correlation between technical and non-technical scores was assessed using the Spearman test. A p value <0.05 was considered significant. Statistical analysis was performed using R 2.12 language .
The hospital level of care, working and simulation experience of participants are shown in Table 1. Overall, median technical score was 18 (IQR 16–19) over 22 maximum points (Table 4). Sixteen participants (8 pediatricians and 8 pediatric nurses) scored less than 19 over 22 points, thus failing to pass the technical test. The worst performances were observed for head positioning, controlling mask leak and inflation pressures of the self-inflating bag. The median execution time was 40 s (IQR 30–45) for items 1 to 6 (first 30 s after birth) and 40 s (IQR 36–50) for items 7 to 14 (the next 30 s of the “Golden Minute”). Participants’ hospital level of care did not affect the performance (Table 5). Participants with longer working experience and those who attended previous courses had a higher technical score (p = 0.03 and p = 0.004, respectively, Table 5). Participants who had experience in PPV and those who attended previous courses were faster in performing resuscitation procedures corresponding to items 7 to 14 (p = 0.02 and p = 0.007, respectively; Table 5). In addition, participation in previous courses was associated with short execution time of items 1–6 (p = 0.03; Table 5).
Overall, median non-technical score was 38 (IQR 34–42) over a maximum of 45 points (Table 4). Participants with longer working experience had a higher non-technical score than those with less experience (p = 0.04, Table 5). Previous experience in neonatal resuscitation requiring PPV and the participation in previous courses were also associated to a better non-technical performance (p = 0.0002 and p = 0.007, respectively; Table 5).
Technical and non-technical scores were significantly correlated (Spearman’s rho 0.67, p = 0.0005). The evaluation of technical skills using DVD filmed video provided the same scores of the direct evaluation for each participant. Non-technical skills were not assessed using DVD filmed video.
This study evaluated the effectiveness of a neonatal resuscitation course using high fidelity simulation on practical and organizational skills of health caregivers who are routinely involved in the management of newborns at birth. Our data showed that most participants (16 out of 23 subjects) did not reach the minimum score to pass the test on technical skills. Incorrect head positioning, delay in controlling mask leak and inadequate inflation pressures of the self-inflating bag were the main reasons for the failures. Based on the 2010 version of the Neonatal resuscitation guidelines, attendants needed more than recommended time to execute the steps of neonatal resuscitation: the median execution time for initiating PPV was 40 s rather than 30 s . However, based on previous studies, the 2015 guidelines recommend to start PPV within 60 s after birth, that fits with our participants’ performance . Every effort, however, should be done by instructors to emphasize the importance of avoiding unnecessary delay in initiating ventilation . The effectiveness of our course was similar to that found in a real-life neonatal resuscitation setting , but was lower to that reported in a previous study using the same assessment instrument, but in a different group of subjects . In fact, participants in the Rovamo et al. study were from level III hospital, whereas our participants were from low level hospitals. In agreement with previous work, the experienced staff evaluated in this study reached higher scores in technical as well as non-technical skills than participants with less experience. Our data suggest that, more than the professional background per sè, experience of staff plays an important role in the quality of performance, while working in level I or II hospital does not seem to have a relevant impact. In Italy, most of births occurs in level I and II hospitals, with a limited number of births per centre per year and, consequently, with a very limited exposure to patients in need of resuscitative maneuvers . Due to limited exposure to real life neonatal resuscitations, training programs are crucial, but they should be tailored to the participants’ professional experience. Courses dedicated to low level hospital staff should be structured in order to allow to low-experienced caregivers to effectively improve their skills. Since maneuvers related to PPV were the main reasons for failure, to reserve more time to PPV training session could be an effective instrument to reach this goal. A periodic staff retraining could be also of great importance to obtain better performances in simulated scenarios and in real life resuscitations. Many studies proved indeed the importance of the long-term follow-up to improve training programs [19, 20]. Particularly, resuscitation knowledge and skills were demonstrated to deteriorate by 4 months after initial training .
Interestingly, pediatricians and nurses were equally successful in the technical-skills test (8 pediatricians and 8 nurses failed) despite the different level of education and knowledge. This aspect underlines the importance of direct and repeated experience, in real contexts or in training sessions. Since pediatricians and nurses working in low level hospitals are rarely involved in real-life neonatal resuscitation scenarios, it is not surprising that, although their different educational background, their performances in technical skills were quite similar.
The evaluation “in the field” requires the presence of a dedicated observer during the scenario who could “lose” some basic steps. Our results show that direct and delayed (by using the DVD filmed video recordings) evaluations provided the same score for each participant. The DVD video recording of the performance could also give the candidates a stronger feed-back about their own behavior during the procedure and could help to improve technical skills, as previously demonstrated [9, 18]. Non-technical skills could not be evaluated using the DVD recording because the camera was focused on the single view of patient, not allowing to capture organizational aspects. The opportunity to review video recordings of simulated resuscitations could be used to underline the points of strength and weakness of the participants and could be an useful tool in retraining programs . Organizational skills (non-technical) including continuous evaluation of the patient, the uninterrupted action plan, vigilance and anticipation with adequate medical knowledge are absolutely important for the correct management of the patient, especially in emergency settings. During resuscitation procedures an effective communication is essential for a good teamwork and the identification of the leader is important to drive the overall procedure and to recognize and anticipate the tasks required. Furthermore, the leader has to periodically assess the clinical situation and its progress . We found that working experience, previous direct administration of PPV to a neonate needing resuscitation and previous participation in a neonatal course were associated to a better performance in non-technical skills. We speculate that this was due to the different educational backgrounds and different tasks in everyday work. The direct correlation between technical and non-technical scores suggests that those who have better technical skills have also a better ability to effectively communicate and coordinate the team work.
Our study has some limitations. We enrolled a limited number of participants working in level I and II hospitals from a single Italian region. In addition, the use of a manikin could be misleading for participants (e.g., time to obtain skin color changes) and could lead to loss of motivation. However, high fidelity simulation is the gold standard for education of those involved in emergency situations. Finally, in the present study participants were assessed immediately after the course. The long-term maintenance of the skills achieved during this training was beyond the scope of this study. Nevertheless, having a follow-up (4–6 months) evaluation of participants is essential to assess the real efficacy in improving both technical and non-technical skills using high fidelity simulation .
We assessed technical and non-technical performances of level I and II hospitals health caregivers after a neonatal resuscitation course by using high fidelity simulation in a standard-setting scenario. Our results show a limited effectiveness in comparison to a previous study involving level III hospital health caregivers. Experience, previous direct administration of PPV to a neonate needing resuscitation and previous participation in a neonatal course improved participants’ performance. Technical and non-technical scores were significantly correlated. There was no difference between “real time” and delayed (by using the DVD filmed video recordings) assessment of participants’ technical skills.
Beats per minute
Neonatal resuscitation program
Positive pressure ventilation
Wyckoff MH, Aziz K, Escobedo MB, Kapadia VS, Kattwinkel J, Perlman JM, et al. Part 13: neonatal resuscitation: 2015 American heart association guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2015;132(18 Suppl 2):S543–60.
Kattwinkel J, Perlman JM, Aziz K, Colby C, Fairchild K, Gallagher J, et al. Neonatal resuscitation: 2010 American heart association guidelines of cardiopulmonary resuscitation and emergency cardiovascular care. Pediatrics. 2010;126(5):1400–13.
Mosley CM, Shaw BN. A longitudinal cohort study to investigate the retention of knowledge and skills following attendance on the newborn life support course. Arch Dis Child. 2013;98(8):582–6.
Parotto M, Doglioni N, Micaglio M, Zanardo V, Perilongo G, Trevisanuto D. Efficacy of the neonatal resuscitation program (NRP) course on knowledge retained by residents: comparison among pediatrics, anesthesia and gynecology. Resuscitation. 2010;81(12):1741–2.
Lockyer J, Singhal N, Fidler H, Weiner G, Aziz K, Curran V. The development and testing of a performance checklist to assess neonatal resuscitation megacode skill. Pediatrics. 2006;118(6):e1739–44.
Halameck LP. Simulation as a methology for assessing the performance of healthcare professionals working in the delivery room. Semin Fetal Neonatal Med. 2013;18(6):369–72.
Amin HJ, Aziz K, Halamek LP, Beran TN. Simulation-based learning combined with debriefing: trainers satisfaction with a new approach to training the trainers to teach neonatal resuscitation. BMC Res Notes. 2013;6:251.
Carne B, Kennedy M, Gray T. Review article: crisis resource management in emergency medicine. Emerg Med Australas. 2012;24(1):7–13.
Rovamo L, Mattila MM, Anderson S, Rosenberg P. Assessment of newborn resuscitation skills of physicians with a simulator manikin. Arch Dis Child Fetal Neonatal. 2011;96(5):F383–9.
Halamek LP. Educational perspectives: the genesis, adaptation, and evolution of the Neonatal Resuscitation Program. NeoReviews. 2008;9(4):e142–9.
Kamlin CO, Dawson JA, O’Donnell CP, Morley CJ, Donath SM, Sekhon J, et al. Accuracy of pulse oximetry measurement of heart rate of newborn infants in the delivery room. J Pediatr. 2008;152(6):756–60.
Carbine DN, Finer NN, Knodel E, Rich W. Video recording as a means of evaluating neonatal resuscitation performance. Pediatrics. 2000;106(4):654–8.
Cronin C, Cheang S, Hlynka D, Adair E, Roberts S. Videoconferencing can be used to assess neonatal resuscitation skills. Med Educ. 2001;35(11):1013–23.
Racioppi M, Salmaso L, Brombin C, Arboretti R, D’Agostino D, Colombo R, et al. The clinical use of statistical permutation test methodology: a tool for identifying predictive variables of outcome. Urol Int. 2015;94(3):262–9.
R Development Core Team. R. A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2011. http://www.r-project.org/. Accessed 28 Jul 2016.
Trevisanuto D, De Bernardo G, Res G, Sordino D, Doglioni N, Weiner G, et al. Time perception during neonatal resuscitation. J Pediatr. 2016. doi:10.1016/j.jpeds.2016.07.003.
Trevisanuto D, Bertuola F, Lanzoni P, Cavallin F, Matediana E, Manzungu OW, et al. Effect of a neonatal resuscitation course on healthcare providers’ performances assessed by video recording in a low-resource setting. PLoS One. 2015;10(12):e0144443.
Rosaria Boldrini, Miriam Di Cesare e Cristina Tamburini. Certificato di assistenza al parto (CeDAP). Analisi dell’evento nascita – Anno 2009. In: Pubblicazioni. Ministero della Salute. 2009. http://www.salute.gov.it/imgs/C_17_pubblicazioni_1731_allegato.pdf Accessed 10 Aug 2015.
Hernández-Padilla JM, Suthers F, Granero-Molina J, Fernández-Sola C. Effects of two retraining strategies on nursing students’ acquisition and retention of BLS/AED skills: a cluster randomised trial. Resuscitation. 2015;93:27–34.
Broster S, Cornwell L, Kaptoge S, Kelsall W. Review of resuscitation training amongst consultants and middle grade paediatricians. Resuscitation. 2007;74(3):495–9.
Kim J, Neilipovitz D, Cardinal P, Chiu M. A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies (abbreviated as “CRM simulator study IB”). Simul Healthc. 2009;4(1):6–16.
Donoghue A, Nishisaki A, Sutton R, Hales R, Boulet J. Reliability and validity of a scoring instruments for clinical performance during pediatric advanced life support simulation scenarios. Resuscitation. 2010;81(3):331–6.
Authors have not receive funding for this research.
Availability of data and materials
The data will not be shared to respect the privacy of the participants.
Giuseppe De Bernardo wrote the first draft, collected the data and revised the manuscript. Desiree Sordino collected the data and revised the manuscript. Francesco Cavallin designed the study, performed statistical analysis, and revised the manuscript. Veronica Mardegan, Nicoletta Doglioni and Maria Luisa Tataranno contributed to interpretation of data, revised the manuscript and made substantial scientific contributions. Daniele Trevisanuto conceived and designed the study. He revised it for important intellectual content. All authors approved the final manuscript as submitted.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
The study was approved by the Ethics Committee of Santobono-Pausilipon Hospital, Naples.