The introduction of a high-fidelity simulation program for training pediatric critical care personnel reduces the times to manage extracorporeal membrane oxygenation emergencies and improves teamwork

Introduction

In the last decade, ECMO has seen a “renaissance” in its use with an increase in the number of adult ECMO runs to treat severe acute respiratory failure, cardiogenic shock and refractory cardiac arrest (1,2). In contrast, the use of ECMO in neonates and children has seen a reduction due to the improvement of conventional medical therapy (3). Thus, neonatal and pediatric ECMO is considered a low-volume and high-risk procedure, compounded by the fact that technical emergencies (i.e., pump or oxygenator failure, etc.) are associated with a mortality rate of 25% (4-6) in the neonatal/pediatric age group. It is therefore paramount to maintain the experience and minimize any external iatrogenic factors that could negatively influence the patient’s outcome.

The initiation of an ECMO program is based on many human and logistic efforts since this procedure requires extensive multidisciplinary coordination as well as well-trained personnel. The “ideal ECMO Team” would then need an immediate “in-hospital” availability of human resources, such as a surgeon able to perform urgent cannulation, a perfusionist experienced in rapidly priming a circuit and an ECMO specialist able to set up baseline ECMO parameters to stabilize the patient. All of these issues necessitate a structured organization by a team leader to minimize confusion and positions overlap in a moment of significant time pressure (2,7).

The Extracorporeal Life Support Organization, an international society focused on ECMO research and education, proposed for the ECMO education didactic lectures with water-drills or animal labs (8,9) and recently introduced the use of high-fidelity simulation programs to improve technical and behavioral skills related to the ECMO management (10). Simulation-based training has emerged as a valuable tool in critical care education offering the opportunity to improve technical skills and reduce the risk of human errors (1,2,11). Previous studies (5,12-16) have reported the feasibility and benefits perceived by trainees in simulation-based ECMO educational programs, but limited data showed the positive impact of simulation to improve both technical and behavioral skills at bedside (17-19). To address this issue, we hypothesized that the introduction of an ECMO specific high-fidelity simulation program would reduce the response times to manage ECMO emergencies and improve behavioral skills.

Methods

This retrospective study was performed in the 6-beds general pediatric intensive care unit (PICU) of the Children’s Hospital Bambino Gesù, Rome, Italy, from the beginning of the year 2011 to the end of 2016. The study population is composed of the personnel working in PICU (12 nurses, 5 physicians, 4 perfusionists). The study was approved by the local Institutional Scientific Board of Bambino Gesù Children’s Hospital (within the Annual Research Program “Ricerca Corrente 2017”, ref number 201702Q003968). Since the study did not involve patient data, the institutional board waived the need for informed consent.

In our hospital, the respiratory ECMO runs are managed in the general PICU since 2010. Cardiac ECMO runs instead are managed in the cardiac PICU since 1988.

Since 2010, an ECMO course per year composed of 34 hours (24 hours of didactic lessons and 10 hours of water drills) was established to train physicians, nurses and perfusionists (Figure 1A). This ECMO curriculum included also a period of supervised bedside training of almost 32 hours for the “new” ECMO specialists. Re-trainings were performed with water drills and seminars every 3 months from 2011 to 2013.

Figure 1 (A) Comparison of the didactic activities in the period 2011–2013 and 2014–2014; (B) simulation room in the Children’s Hospital Bambino Gesù. ECMO, extracorporeal membrane oxygenation.

At the end of 2013, this educational curriculum was completely modified with the introduction of a high-fidelity simulation program as an alternative training tool for all team members including any new and all previously trained “experienced” personnel. This modification was due to the contemporary concern about the positive value as educational tool of high-fidelity simulation (1,2,15,16). The new ECMO course consisted of 32 hours of training per year. The personnel attended both the didactic lessons (16 hours) and the simulation scenarios (16 hours). The simulation scenarios covered all the topics introduced with the didactic lessons. After the course, the “new” ECMO specialists attended 32 hours of bedside training. Re-trainings were performed every 3 months only with high-fidelity simulations and were associated to seminars.

Since the introduction of high-fidelity simulations in our program, 12 simulation sessions where performed to re-train personnel till December 2016. At the end of 2013 we also had new PICU staff because of personnel retirements. Seven new nurses without previous ECMO experience (7 new nurses on a total of 12 nurses), 2 new physicians without previous ECMO experience (2 new physicians on a total of 5 physicians) and 2 new perfusionists with limited ECMO experience (2 new perfusionists on a total of 4 perfusionists) were included in our PICU.

All simulations were conducted using a simulation room equipped with same materials (ventilator, defibrillator, emergency medications) present in PICU. Basic vital signs and physiologic data including central venous pressure, arterial blood pressure, end-tidal carbon dioxide tracing and an electrocardiogram were displayed in real time on a bedside monitor in accordance with clinical scenario (Figure 1B). Simulations were built up with preprogrammed scenarios using the Laerdal SimBaby software (Laerdal Italia SRL). This approach was used to assure that all the trainees could receive the same learning experience during the simulation sessions. All the simulations were videotaped and discussed in a debriefing session with a simulation instructor. During the debriefing all the learning objectives of the scenario were reviewed and were reinforced positive team behaviors, as well as areas for improvement (11,12).

Measurements and definitions

We defined the pre-simulation period from January of 2011 to December of 2013 and the post-simulation period from January 2014 to December 2016 according to the introduction of the simulation program.

Emergency intervention time “at bedside” was defined as the time in minutes from the exact moment the call was placed to the pager reported on the clinical charts to the actual resolution of the ECMO emergency identified as the re-start of the ECMO flow reported in the perfusionist charts. The ECMO emergencies evaluated retrospectively were: pump failure, oxygenator change and air embolism management.

In the pre-simulation period, all personnel were novice learners whereas in the post-simulation period we had both novice and experienced personnel. However, “only” ECMO emergencies managed by novice personnel in the post-simulation period were included and evaluated in the study. The introduction at the end of 2013 of a different educational program with high-fidelity simulations associated to the renewal of the personnel allowed us to make a more homogeneous comparison of the two study periods including only novice learners.

The behavioral skills reported in Figure 2 were evaluated “in situ” for each person involved in the management of the emergency and scored using a 5-point Likert scale (12). Scores were reported on a separate medical registry, which belongs to the Health Directorate of our hospital. Subjects were rated on 10 different behavioral items on a scale from 1 (poor) to 5 (excellent) for a possible total score of 50 points. The “in situ” evaluation of the behavioral skills is a required and established practice of the Bambino Gesù Risk Management Program. All medical emergencies occurring in our PICU are scored immediately after their management and recorded in separate registry. This evaluation is performed by one of the two doctors in charge not involved in the management of the crises. The use of a very simple Likert scale associated to repeated courses of risk management for all the medical personnel working in PICU provided a homogeneous scoring system among doctors.

Figure 2 Behavioral skills evaluated using a 5-point Likert scale. ECMO, extracorporeal membrane oxygenation; RT, respiratory therapist.

Statistical analysis

We retrospectively collected demographic and clinical data. We compared the duration of emergency interventions and behavioral skill scores in the pre-simulation period (January 2011 to December 2013) to the post-simulation period (January 2014 to December 2016). Further analyses were made between the two groups for confounding variables that could potentially affect the results. These analyses evaluated the following variables: the primary diagnosis leading to ECMO, the type of ECMO (venoarterial vs. venovenous ECMO), the surgical technique (percutaneous vs. surgical) and daytime vs. night-time/weekend emergency management. Continuous data were shown as median and interquartile ranges [IQR1 and IQR3]. Categorical data were described as absolute frequencies and percentage. Mann-Whitney U test was used to analyse non-parametric variables, student t-test was used to analyse normally distributed variables. Normal distribution and homogeneity of variances were analysed using Shapiro-Wilk test and Bartlett test. Categorical differences were tested using the Yates’s chi-squared test or the Fisher exact method depending on the size of the observed frequencies. A P values of <0.05 was considered statistically significant. Statistical analyses were performed using Epi Info, version 7.1.5 (Centres for Disease Control and Prevention, Atlanta, GA, 2015) and “R” statistical software (version 3.2.5; R Foundation for Statistical Computing, 2016).

Results

During the study period ranging from January 2011 to December 2016, a total of 30 ECMO runs occurred in our institution. Twelve ECMO runs were performed during the pre-simulation period, while 18 in the post-simulation period. Of these 18 ECMO runs, only 14 were included in the evaluation because managed only by novice personnel.

Demographic and clinical characteristics of the patients are summarized in Table 1. A total of 27 ECMO emergencies in 30 patients were recorded from 2011 to 2016. No patients had more than one emergency. Ten emergencies occurred during the pre-simulation period and 17 occurred in the post-simulation period. Of these 17 emergencies, only 12, which were managed only by novice personnel, were included in the analysis.

Table 1 Characteristics of patients and ECMO runs stratified by time period. Data are presented as counts (percentage) or median (IQR 1–3), where not otherwise specified
Full table

When comparing the two periods (pre-simulation vs. post-simulation), we did not find any statistically significant difference in patient age, weight, gender, distribution of neonatal vs. paediatric cases, distribution of diseases treated, type of ECMO support (VA vs. VV), number of emergencies managed, mode of cannulation (percutaneous vs. surgical), night and weekend time management of the ECMO emergencies (Table 1). Also, the distribution of the emergencies treated (pump failure, air embolism, oxygenator failure) was similar among the two periods. There was no survival difference between the pre and post-simulation period for the ECMO runs evaluated in the study (50% vs. 57.1%) (P=0.72).

The median time to change an oxygenator was 5.30 (4.8–6.02) min during the pre-simulation period whereas in the post-simulation period was 3.90 min (3.50–4.15) (P=0.02). The median time to manage an air embolism emergency was 22 (20–23.50) min during the pre-simulation period whereas in the post-simulation period was 15 (13.75–16.50) min (P=0.048). Given the infrequent occurrence of pump failure (1 episode in each period), we did not make a comparison of pump replacement times.

Behavioural skills were evaluated using a “cumulative score” (defined as the median of all behavioral scores pre and post-simulation). In the pre-simulation period the median cumulative behavioural score was 40 (35–44.75) whereas it was 48 (44.5–49) in the post-simulation period (P<0.01).

Discussion

Extracorporeal life support is one of the most complex techniques available in the critical care setting, requiring both technical and non-technical skills to deal with unforeseen events and complications (16).

Currently ECMO education is not a formal part of any medical or surgical subspecialty’s training curriculum (17-19), therefore ECMO training is usually accomplished with courses performed with didactic lectures, water drills, animal labs and periods of supervised mentorship. These teaching strategies overemphasize cognitive skills, underemphasize technical skills and completely ignore behavioural skills (1,2,6). The Extracorporeal Life Support Organization (ELSO) has developed guidelines for training and continuing education of ECMO specialists, which outline institutional requirements for effective ECMO use (8,9). In the 5th edition of the ELSO Red Book, these guidelines have been re-visited introducing the role of high-fidelity simulations to train specialists for skills acquisition in routine and emergency situations (10).

Simulation has become a very common method of training medical and nursing personnel to learn specific technical skills and behavioural skills. Previous studies (6,11-16) have demonstrated the efficacy of high-fidelity simulation to improve teamwork in stressful situations (urgent cannulation, extracorporeal cardiopulmonary resuscitation (ECPR), troubleshooting regarding equipment malfunctions and other bedside emergencies). However, despite the accepted role of this method in other medical fields, there has been a limited evaluation of the impact of ECMO high-fidelity simulations at bedside (1,2,17-19).

The main objective of our study was to evaluate, using a retrospective approach, if the introduction of a high-fidelity simulation program improves the performance of novice personnel. This hypothesis was tested by comparing the duration of the emergency interventions and the measurement of the behavioural skills score between the two-study periods. When comparing pre-simulation vs. post-simulation period, we found a reduction in the times used to manage clinical crises including the oxygenator change due to rapid failure or the circuit change because of air embolism. The same positive impact of high-fidelity simulation was found when we evaluated the cumulative behavioural skills score between the two study periods.

Our ECMO program started in 2010 (pre-simulation period) and to train our personnel we used the traditional ECMO training proposed by the ELSO guidelines for training and continuing education (8,9), therefore our ECMO course (34 hours) was performed with lectures and water drills, and 32 hours of mentorship at bedside, while re-trainings were performed every 3 months with water drills and seminars. All team members evaluated in the pre-simulation period were novice learners without previous experience of respiratory ECMO. Only the cardiac surgeons and the perfusionists had a previous knowledge of cardiac ECMO.

At the end of 2013 there was a critical change in our working team due to the retirement of some PICU staff, therefore new physicians, nurses and perfusionists were enrolled in the “new” ECMO team. Contemporary to this personnel renewal, we also introduced high-fidelity simulations as a teaching tool in our ECMO program. The evaluation in the post-simulation period of “only” novice personnel allowed us to “reduce” the biases due to the natural improvement of technical and behavioural skills over time with bedside practice. However, in any educational intervention, it is almost impossible to state that other learning opportunities (learning from experienced colleagues, reading, self-practising, etc.) do not contribute to improve the skills of the students.

Other investigators have also reported the improvements of technical and behavioural skills with the utilization of high-fidelity ECMO simulation training. Anderson et al. (12) found that ECMO practitioners reduced errors in emergency scenarios after a simulation-based training, but not the times to critical actions, whereas Burkhart et al. (14) reported that simulation-based training improved the emergency circuit management among thoracic-surgery residents working with a post-cardiotomy ECMO model. Zakhary et al. (2) reported the improvement of times to critical action with simulation-based training. These results differed from the ones reported by Anderson et al. (12) because in the randomized controlled trial of Zakhary et al. (2) novice learners were included suggesting that the benefits of high-fidelity simulation may be reduced when evaluating an experienced group. Allan et al. (13) also reported the improvement of surgical skills evaluated as time elapsed to cannulation in 10 cardiothoracic surgery residents and fellows with the use of high-fidelity simulations.

There is paucity of data on the positive effect of high-fidelity simulation to bedside ECMO management. Burton et al. (17) described for the first time the clinical results of simulation-based training for ECMO clinical specialists. Brazzi et al. (18) reported a more important impact of high-fidelity simulation on the delivery care. They used didactic lectures and simulations to rapidly train 193 ECMO physicians in 14 centers in response to the 2009 H1N1 pandemic. After their training program, no survival difference was detected between the centers with previous ECMO experience and those without (71% vs. 64%, P=0.35). Su et al. (19) reported the “clinical” efficacy of a high-fidelity simulation program to train personnel to manage ECPR situations. She showed a significant reduction of ECPR deployment times (51 vs. 40 minutes, P=0.018) when comparing pre and post intervention periods.

Our study, performed with the same retrospective approach of Su et al. (19), showed, using a simulation-based training, a significant reduction of the times used to manage ECMO emergency crises and a significant improvement of behavioral skills score when comparing pre-simulation period vs. post-simulation period. Our study confirmed the positive impact of high-fidelity simulations on personnel performance regarding the management of ECMO at bedside; furthermore, for the best of our knowledge, this study was the first to evaluate the behavioral skills of the ECMO team in real clinical scenarios as opposed to a simulation environment.

Our study has also several limitations. The most important are: the retrospective nature and the small sample size evaluated. In our study the small sample size was due: firstly, to the reduction of the ECMO runs in the last years because of the improvement of the medical management of the neonatal and pediatric respiratory failure (3); secondly, to the fact that many ECMO runs that occurred in the post-simulation period were not included in the study because were not managed by novice learners. However, this aspect according to our opinion highlights the positive performances acquired by novice learners during the second study period using high-fidelity simulation. Even is this small sample size did not allowed to evaluate any improvement in patient outcomes between the two study periods, this was not an objective of the study.

We also acknowledge that high volume ECMO centers reported lower mortality both in adult and children when compared with low volume ECMO centers (20-22), however this aspect is no more actual when dealing with neonates and children because the ECMO runs are reducing (3). Thus, we strongly believe that a standardized ECMO education with high-fidelity simulation is mandatory in low volume ECMO centers to homogeneously train all personnel.

In conclusion, our study demonstrated the incremental improvement in behavioural skills and measured outcomes among novice learners with the introduction of a high-fidelity simulation program. We believe that further prospective and randomized studies should be performed to determine whether simulation training improves clinical outcomes and translates to reduced complication rates in patients supported with ECMO.

Acknowledgements

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The study was approved by the local Institutional Scientific Board of Bambino Gesù Children’s Hospital (within the Annual Research Program “Ricerca Corrente 2017”, ref number 201702Q003968).

References

1. Banfi C, Bendjelid K, Giraud R. High-fidelity simulation for extracorporeal membrane oxygenation training, utile or futile? J Thorac Dis 2017;9:4283-5. [Crossref] [PubMed]
2. Sakamoto S. Simulation-based training for handling extracorporeal membrane oxygenation emergencies. J Thorac Dis 2017;9:3649-51. [Crossref] [PubMed]
3. ELSO Registry Report, International Summary. Available online: http://www.elso.org/Registry/Statistics/InternationalSummary.aspx
4. Zakhary BM, Kam LM, Kaufman BS, et al. The Utility of High-Fidelity Simulation for Training Critical Care Fellows in the Management of Extracorporeal Membrane Oxygenation Emergencies: A Randomized Controlled Trial. Crit Care Med 2017;45:1367-73. [Crossref] [PubMed]
5. Fehr JJ, Shepard M, McBride ME, et al. Simulation-Based Assessment of ECMO Clinical Specialists. Simul Healthc 2016;11:194-9. [Crossref] [PubMed]
6. Anderson JM, Boyle KB, Murphy AA, et al. Simulating Extracorporeal Membrane Oxygenation emergencies to improve human performance. Part I: Methodologic and technologic innovations. Simul Healthc 2006;1:220-7. [Crossref] [PubMed]
7. Allan CK, Kleinman ME. Enhancing the power of simulation for complex clinical care. Pediatr Crit Care Med 2014;15:904-6. [Crossref] [PubMed]
8. ELSO Guidelines for ECMO Centers. Available online: http://www.elso.med.umich.edu/guide.html. Accessed March 2017.
9. ELSO Guidelines for Training and Continuing Education of ECMO Specialists. Available online: http://www.elso.med.umich.edu/guide.html. Accessed March 2017.
10. Ogino MT, Froehlich CD, Moore EA. Education and Training. Chapter 67. In: Extracorporeal Life Support: The ELSO Red Book. Brogan TV, Lequier L, Lorusso R, et al. Editors. 5th Edition. Ann Arbor: ELSO, 2017.
11. Chan SY, Figueroa M, Spentzas T, et al. Prospective assessment of novice learners in a simulation-based extracorporeal membrane oxygenation (ECMO) education program. Pediatr Cardiol 2013;34:543-52. [Crossref] [PubMed]
12. Anderson JM, Murphy AA, Boyle KB, et al. Simulating Extracorporeal Membrane Oxygenation emergencies to improve human performance. Part II: Assessment of technical and behavioral skills. Simul Healthc 2006;1:228-32. [Crossref] [PubMed]
13. Allan CK, Pigula F, Bacha EA, et al. An extracorporeal membrane oxygenation cannulation curriculum featuring a novel integrated skills trainer leads to improved performance among pediatric cardiac surgery trainees. Simul Healthc 2013;8:221-8. [Crossref] [PubMed]
14. Burkhart HM, Riley JB, Lynch JJ, et al. Simulation-based postcardiotomy extracorporeal membrane oxygenation crisis training for thoracic surgery residents. Ann Thorac Surg 2013;95:901-6. [Crossref] [PubMed]
15. Brum R, Rajani R, Gelandt E, et al. Simulation training for extracorporeal membrane oxygenation. Ann Card Anaesth 2015;18:185-90. [Crossref] [PubMed]
16. Sanchez-Glanville C, Brindle ME, Spence T, et al. Evaluating the introduction of extracorporeal life support technology to a tertiary-care pediatric institution: Smoothing the learning curve through interprofessional simulation training. J Pediatr Surg 2015;50:798-804. [Crossref] [PubMed]
17. Burton KS, Pendergrass TL, Byczkowski TL, et al. Impact of simulation-based extracorporeal membrane oxygenation training in the simulation laboratory and clinical environment. Simul Healthc 2011;6:284-91. [Crossref] [PubMed]
18. Brazzi L, Lissoni A, Panigada M, et al. Simulation-based training of extracorporeal membrane oxygenation during H1N1 influenza pandemic: the Italian experience. Simul Healthc 2012;7:32-4. [Crossref] [PubMed]
19. Su L, Spaeder MC, Jones MB, et al. Implementation of an extracorporeal cardiopulmonary resuscitation simulation program reduces extracorporeal cardiopulmonary resuscitation times in real patients. Pediatr Crit Care Med 2014;15:856-60. [Crossref] [PubMed]
20. Barbaro RP, Odetola FO, Kidwell KM, et al. Association of hospital-level volume of extracorporeal membrane oxygenation cases and mortality. Analysis of the extracorporeal life support organization registry. Am J Respir Crit Care Med 2015;191:894-901. [Crossref] [PubMed]
21. Freeman CL, Bennett TD, Casper TC, et al. Pediatric and neonatal extracorporeal membrane oxygenation: does center volume impact mortality? Crit Care Med 2014;42:512-9. [Crossref] [PubMed]
22. Karamlou T, Vafaeezadeh M, Parrish AM, et al. Increased extracorporeal membrane oxygenation center case volume is associated with improved extracorporeal membrane oxygenation survival among pediatric patients. J Thorac Cardiovasc Surg 2013;145:470-5. [Crossref] [PubMed]