Impact of simulation training on performance and outcomes of endobronchial ultrasound-guided transbronchial needle aspiration performed by trainees in a tertiary academic hospital

Introduction

Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is a minimally invasive procedure performed for the diagnosis, staging and restaging of mediastinal lymphadenopathy in lung cancer (1) and other lung conditions such as lymphoma, sarcoidosis and tuberculosis (TB) (2-4). It is an alternative to mediastinoscopy, which is considered for many years as a reference standard for sampling mediastinal lesions (5,6).

EBUS-TBNA has a high diagnostic yield in conditions such as primary lung cancer, TB, sarcoidosis and metastatic disease (5). The reported complication rates are low as described in a systemic review done by von Bartheld et al. (6). They reported that out of 9,199 patients who underwent EBUS, serious adverse events and adverse events were seen in 5 (0.05%) and 10 patients (0.11%) respectively. A nationwide survey in Japan showed a higher complication rate of 1.23% in 7345 patients (7) and the AQuIRE registry reported a complication rate of 1.44% in 1317 patients (8).

Whilst EBUS-TBNA has been associated with high diagnostic yield and low complication rates, the performance outcome may differ between experts and trainees who perform it under supervision. In a single-center retrospective analysis of 220 subjects undergoing EBUS-TBNA, it was found that procedure time was longer (16 vs. 13.7 min), amount of lidocaine use was higher (322.3 vs. 304.2 mg) and the diagnostic yield was lower (52.6% vs. 68.3%) when comparing trainees to staff physicians performing EBUS-TBNA (9). In another study of 607 procedures, it was reported that there was a higher complication rate in the trainee group (4.7% vs. 1.1%, P=0.076) (10).

In order to improve the diagnostic yield of EBUS procedures performed by trainees, simulation training, standardized curriculums and various assessment tools have been developed across different centres. A randomized controlled trial was done by Konge et al. in 2015 (11) to assess the effectiveness of different modalities of training. Sixteen respiratory physicians, without EBUS experience, were randomized to either virtual reality simulator training or traditional apprenticeship training on patients. The study showed that simulator training resulted in a higher score for anatomical orientation, as well as for technical skills. Nevertheless, data is lacking on whether simulation training had effect on clinical endpoints such as diagnostic yield and complication rates.

Prior to 2013, we used the “apprenticeship model” to teach our trainees how to perform EBUS-TBNA. In this model, when supervisors were confident in the skills of the pulmonary trainees in the performance of conventional bronchoscopy, they were allowed to perform EBUS-TBNA on patients with direct supervision. There was no formal EBUS-TBNA didactic curriculum and no hands-on training on models. In 2013, EBUS simulation training was implemented in our hospital.

Therefore, our aims were to investigate the impact of simulation training on the: (I) diagnostic yield of EBUS-TBNA performed by trainees; (II) EBUS-TBNA related complication rates; (III) scope damages and repair costs; and (IV) learning curves of trainees.

Methods

Retrospective medical record review of patients in our hospital’s EBUS-TBNA Registry was performed. EBUS-TBNA was first performed in our hospital in August 2008 and we had previously reported some of our findings and outcomes (12-15). Bronchoscopy simulation training was introduced in our hospital in 2013. The study period was thus chosen to be between 1^st August 2008 and 31^st December 2016 in order to have adequate number of procedures before and after simulation training commenced for comparison of the outcomes stated. Informed consent for the procedures was obtained from all patients. This study was approved by SingHealth CIRB (2008/458/B and 2011/350/C).

Participants and eligibility

All patients in the hospital EBUS-TBNA Registry were screened for eligibility. There were four groups of patients who were excluded in this study: no trainee participation in EBUS-TBNA, patients on clinical trials whose specimens were processed in an external laboratory and histology reports were not available in the electronic medical records, foreigners who were lost to follow up as the final diagnoses could not be confirmed and lastly, trainees who had done less than 10 EBUS procedures as they had too few procedures to determine competency. The minimum number of procedures was chosen as 10 because a systematic review (16) reported that the number needed to overcome the initial learning curve of EBUS varied from 10 to 100. Complications due to procedures performed simultaneously with EBUS-TBNA, such as transbronchial biopsy (TBLB), endobronchial biopsy (EndoBBx) or bronchoalveolar lavage (BAL) were excluded when the complications were reported to be after TBLB, EndoBBx or BAL were performed.

Information on independent variables such as patient demographics (gender, age), clinical characteristics (pre-test probability of underlying pathology, lymph node size, number of lymph nodes biopsied, site of lymph node and number of passes made) and physician experience (number of years of senior residency training) was collected. The dependent variables included diagnostic yield, complication rate, scope damage and repair cost.

Procedure

EBUS-TBNA was carried out under local anaesthesia with lignocaine 2% in 2 mL aliquots, and moderate sedation with midazolam and/or fentanyl. The BF-UC260FW (Olympus Corporation, Tokyo, Japan) bronchoscope was used with 22-gauge needles for tissue aspiration.

The EBUS simulation training consisted of a two-day course which included lectures on ultrasound basics and physics, mediastinal anatomy, lung cancer staging, EBUS-TBNA application and techniques, and hands-on simulation training using low fidelity airway models and high fidelity virtual reality simulators. The EBUS-TBNA simulator used was BRONCH Mentor^TM by 3D systems. It features authentic scope, tactile feedback and a flexible working setup to promote team training. It also has virtual patient cases modeled after real patients. The EBUS-STAT checklist was used as reference during the training (16). There were three trainees in each session of simulation training group.

A true positive result was defined as an EBUS-TBNA histological, cytological or microbiological sample positive for primary lung cancer, metastasis, lymphoma, TB or sarcoidosis. A true negative result was defined as an EBUS-TBNA sample negative for lung cancer, metastasis, lymphoma, TB or sarcoidosis and confirmed by either a follow up PET-CT, chest CT or surgical sample, which were unyielding for an alternative diagnosis within a year of the EBUS-TBNA procedure being performed. A false negative result was defined as an EBUS-TBNA sample that was negative, but turned out to be positive for cancer, inflammatory or infectious causes through other diagnostic methods or on clinical and radiological follow-up (17).

Complications were classified into either major or minor events according to the British Thoracic Society guideline for diagnostic flexible bronchoscopy in adults (18). Major complications were defined as severe bleeding, cardiac arrhythmia requiring treatment, seizures, myocardial infarction/pulmonary oedema, pneumothorax requiring aspiration/intercostal drain, over-sedation requiring ventilatory support or reversal, hospitalization, admission to ICU or death. Severe bleeding was defined as requiring placement of bronchus blocker or catheter, applying fibrin sealant, resuscitation, blood transfusion, admission to critical care unit or death (18). Moderate bleeding was defined as the use of adrenaline or cold saline to stop bleeding. Mild bleeding was defined as spontaneous stoppage of the bleeding. Damages to the EBUS bronchoscope were recorded. These damages were either discovered during the procedure or during re-processing.

Trainee participation was defined as the presence of a pulmonary fellow who performed various aspects of the procedure under the supervision of an attending physician. All trainees were not credentialed to perform flexible bronchoscopy independently at the time of performing EBUS. Our credentialing requirements are: a minimum of 100 flexible bronchoscopies under supervision and 20 per year to maintain the privilege. All trainees who entered our respiratory program after 2013 underwent EBUS-TBNA simulation training.

CUSUM analysis

CUSUM analysis was applied to produce a learning curve for the two groups of trainees. In this study, we used the format and parameters for CUSUM scoring used by Kemp et al. (17). A graph was obtained by plotting the CUSUM value on the y-axis against the attempts on the x-axis. The CUSUM value was a summation of the increments (for each failure) and decrements (for each success). We designated acceptable and unacceptable failure rate to be 10% (p₀) and 20% (p₁) respectively (17). The score for each success (s) was calculated using the equation:

P = ln(p₁/p₀), Q = ln[(1 – p₀)/(1 – p₁)], s = Q/(P + Q).

Hence,

p₀ =0.1, p₁ =0.2, s =0.15, 1–s =0.85.

A declining or stable trend in CUSUM score would represent an acceptable success rate while an increasing trend would represent a lower than expected success rate. H₀ represents the value between each acceptable decision interval while H₁ is the value between each unacceptable decision level. To determine H, a type 1 error (α) and type 2 error (β) are given a value of 0.1.

a = ln((1–β)/α), b = ln[(1 – α)/β], H0 = b/(P + Q), H1 = a/(P + Q).

Hence,

H₀ =2.71, H₁ =2.71.

Statistical analysis

Our primary outcome was to compare pre and post simulation training based on diagnostic yield, complication rate and scope damage. Diagnostic yield was defined as . Outcomes diagnostic yield, complication rate and scope damage where treated as binary data. We presented diagnostic yield, complication rate and scope damage as percentage with corresponding 95% confidence interval (95% CI). All categorical and continuous demographic and clinical variables were expressed in terms of frequency with proportions and mean ± standard deviation (SD) or median [interquartile range (IQR)], whichever applicable, respectively. We fit separate univariate and multivariate logistic regression model to find association between outcomes and pre and post simulation training along with other demographic and clinical independent variables. Variables with P value <0.2 in univariate logistic regression and clinically relevant variables were considered for multivariate regression. The final variables for multivariate models were chosen through forward, backward and stepwise regression methods. Associations between primary outcome and other co-variates were quantified as odds ratio (OR) along with 95% CI. Effect of clustering, which recognised that procedurists may have different outcomes because of systematic differences in processes of care, was also taken into account.

Our secondary outcome was learning curves of trainees (CUSUM). We performed CUSUM analysis to check the performance of training provided. A declining or stable trend in CUSUM score would represent an acceptable success rate (competency) while an increasing trend would represent a lower than expected success rate (non-competency). P value <0.05 was considered as statistical significance. All tests performed were 2 sided. Statistical analyses were performed using IBM SPSS statistics version 24.

We had 608 eligible trainees. Our primary objective was to find association between simulation training and diagnostic yield. To find a minimum clinically meaningful difference of 10% in diagnostic yield between pre and post simulation training, we needed to recruit 540 (270×2) trainees in pre and post simulation training. Above calculation was based on following parameters: diagnostic yield in pre and post simulation training as 75% and 85% respectively i.e., a 10% increment in post simulation training, power as 80%, alpha or type I error as 5% and Fisher’s exact test. We also tried to find associated risk factors of diagnostic yield. Peduzzi et al., Concato et al. and Vittinghoff et al. (19-21) recommended that multivariable logistic regression models should be used with at least 10 events per predictor variable. We had 20 clinically meaningful variables to account for in the multivariate model and hence we needed at least 10×20=200 events in the data. In our data, we had almost 500 positive diagnoses of EBUS. Hence our study was adequately powered to find difference between pre and post simulation training in diagnostic yield of EBUS and associated risk factors of diagnostic yield of EBUS.

Results

From 1^st Aug 2008 to 31^st Dec 2016, 1,153 EBUS-TBNA procedures were performed. Out of these 1,153 EBUS procedures, 479 had no trainee involvement, 8 were involved in clinical trials and had no histological report available in the electronic system, 10 were foreign patients who were lost to follow up and 48 procedures were performed by trainees with less than 10 procedures. A total of 608 patients were included in this study (Figure 1).

Figure 1 Flow diagram on patient enrolment.

The mean ± SD age of patients was 62.3±12.7 years, and 413 (67.9%) were male. There were 331 (54.4%) procedures performed by trainees who did not undergo simulation training [2008–2012] and 277 (45.6%) procedures performed by trainees who underwent simulation training [2013–2016].

The overall diagnostic yield with trainee involvement was 86.5% (95% CI: 83.5–89.1%). Diagnostic yield of EBUS-TBNA was 88.2% (95% CI: 84.1–91.4%) for trainees without simulation training, compared to 84.5% (95% CI: 79.6–88.4%) for trainees with simulation training (P=0.179). There were 121 patients with sampling of a central lung lesion that was adjacent to the major airways. The median size of the lymph nodes/lesion sampled was 20 mm (range, 4–80 mm). The median procedure time was 40 min (range, 5–120 min). The median midazolam dose was 4.0 mg (range, 1–20 mg) and the median fentanyl dose was 50 mcg (range, 0–200 mcg). There was no statistically significant difference in the duration of the procedure and median dose of sedatives (midazolam and fentanyl) (Table 1).

Table 1 Patient demographics and clinical characteristics with respect to trainees with and without simulation training
Full table

Pretest probability of underlying pathology, lymph node station, lymph node size and final diagnosis were significant (Table 2). Multivariate analysis for diagnostic yield showed that trainees without simulation training were 1.28 (95% CI: 0.68–2.39, P=0.438) times more likely to obtain a diagnostic EBUS-TBNA as compared to those with training. However, this difference was not significant (Table 3). Factors that were independently associated with diagnostic yield included size of lymph node more than 20mm and lymph node station 4R. The presence of rapid on-site evaluation (ROSE) did not affect diagnostic yield (P=0.739).

Table 2 Patient demographics and clinical characteristics with respect to diagnostic yield
Full table

Table 3 Predictors of a diagnostic EBUS-TBNA
Full table

The overall incidence of complications was 15.0% (95% CI: 12.3–18.1%). Major complications occurred in 14 patients (2.3%). The major complication rate for trainees without simulation training was 3.6% vs. 0.7% for trainees with simulation training (P=0.112). The minor complication rate for trainees without simulation training was 10.0% vs. 15.9% for trainees with simulation training (P=0.041). Hypoxaemia (10, 1.6%) was the most frequent major complication while bleeding (37, 6.1%) was the most frequent minor complication. There were no mortalities. A breakdown of complication rates is shown in Table 4.

Table 4 Major/minor complications and scope damage
Full table

Based on multivariate analysis for comparison of complications, the result showed that trainees with simulation training were 2.25 times more likely to experience either a major or minor complication as compared to trainees without (OR: 2.247, 95% CI: 1.297–3.891, P=0.004) (Table 5).

Table 5 Predictors of major and minor complications of EBUS-TBNA
Full table

Overall scope damage rate was 1.2% (95% CI: 1.0–3.3%). Scope damage for trainees without simulation training was 2.7% (n=9) vs. 0.7% (n=2) for trainees with simulation training (P=0.066). The total cost for scope repairs for trainees without simulation training was SGD 136,994 (average of SGD 413.88 per procedure) vs. SGD 50,634 (average of SGD 182.79 per procedure) for trainees with simulation training, with the mean reduction of cost being SGD 231.09 (95% CI: 178.40–640.60, P=0.268).

The institutional CUSUM graph comparing the trainees (Figure 2) showed different learning curve patterns. For the trainees without simulation training, there are three distinct intervals, namely, the interval between procedures 1 and 21 (learning interval), between procedures 22 and 147 (intervals of worsening and proficiency period) and after procedure 148 (proficiency interval). As for the trainees with simulation training, there are 2 distinct intervals, namely, the interval between procedures 1 and 177 (proficiency interval) and after procedure 178 (worsening interval).

Figure 2 CUSUM graph of institutional learning curve comparing trainees with and without simulation training.

The individual procedurist CUSUM graphs were analysed by different groups of trainees and are shown in Figures 3 and 4. Trainee 16 had an increasing learning curve up till procedure 14, after which the trainee attained competency. Trainee 26, 37 and 39 were still in a learning phase after 23, 29 and 24 procedures respectively.

Figure 3 CUSUM graph of individual learning curves of trainees without simulation training.

Figure 4 CUSUM graph of individual learning curves of trainees with simulation training.

Discussion

We sought to investigate the impact of structured training that included simulation training on EBUS-TBNA performed by trainees. Diagnostic yield did not increase after simulation training. There was an increase in overall complications from 13.6% to 16.6% (adjusted OR, 2.247; 95% CI, 1.297–3.891; P=0.004), decreased trend of scope damage rates from 2.7% to 0.7% (P=0.066) and decreased trend of cost of scope damage from SGD 413.88 per procedure to SGD 182.79 per procedure (P=0.268).

A systematic review on training and proficiency in EBUS-TBNA showed that there were 8 studies (6 observational and 2 randomized trials) that described the role of simulator based learning for the accomplishment of EBUS skills. Six of the 8 studies were conducted at the same centre (16). The results from 112 trainees showed that there was no difference in procedure time, percentage of correct identification of lymph nodes and time to intubation between the simulator-based training group and the comparator arm. However, the simulator-based training group had higher chances of successful lymph node biopsies (16). Another study showed that simulation training was more effective than traditional apprenticeship training in the initial part of the learning curve, as evidenced by a higher score based on an EBUS assessment tool (EBUSAT) (11). Our outcome measurement was different from the above studies. We measured actual clinical outcomes such as diagnostic yield and complication rates while other studies quantified the procedural skills of the trainees. While a dichotomous clinical outcome of diagnostic yield may not provide information on individual trainee’s anatomical knowledge and biopsy techniques, it was considered an objective and significant clinical endpoint as opposed to a surrogate endpoint and the definitive measurement of the effectiveness of medical education (9,17).

Although there was a decrease in diagnostic yield of EBUS-TBNA of 3.7% after simulation training, it was not statistically significant. This can be attributed to the ceiling effect, as the diagnostic yield of the trainees without simulation training was already high at 88.2%, comparable to other EBUS-TBNA studies (5,22-24). The diagnostic yield was likely limited to the procedure itself rather than the skills of the trainees.

An overall major and minor complication rate of 15.0% seemed significantly higher than international data (7,8) and this was likely due to methodological differences in definition of complications. For example, a nationwide survey conducted in Japan of 7345 cases of EBUS-TBNA reported a complication rate of 1.23% (7) and hemorrhage was defined as treatments required other than aspiration, compression and cold physiological saline injection. We included all cases of hemorrhage as defined by the British Thoracic Society guidelines (18), which included the ones excluded by the Japan study. The most significant increase in complications between the 2 groups of trainees was due to mild bleeding, 4 cases (1.2%) vs. 10 cases (3.6%). A subgroup analysis of the 10 patients showed that 5 were biopsies of lung masses, 2 had presence of endobronchial lesions, 2 had metastatic disease and 1 was on enoxaparin for pulmonary embolism, which was stopped one day before EBUS-TBNA (Table S1). Overall, trainees with simulation training had more minor complications compared to those without.

Table S1 Subgroup analysis of patients who had complication of mild bleeding performed by trainees
Full table

Our data with scope damage and repair mirrored that of previous studies which had concluded that proper education of physicians and technical personnel would reduce preventable damage, which ranged from 64% to 87% (25-27). There was a 55.8% decrease in cost per procedure after simulation training was implemented in our hospital. In 2001, Colt et al. reported that a simulation training program for novice bronchoscopists increased dexterity of trainees and accuracy of the examination performed by trainees on the simulator (28). Lunn et al. took a step further by comparing the cost of repairs after the introduction of simulation training. He reported that the yearly repair cost decreased by 84% after the introduction of simulation training (27). By undergoing simulation training, the trainee’s EBUS-TBNA procedural techniques can be corrected and refined by experienced procedurists, hence reducing scope damage and repair costs.

Although there was no statistical difference in the diagnostic yield, the CUSUM analysis provided further information about the learning curves of the trainees. The institutional CUSUM graph for trainees without simulation training was a typical graph that reflected an initial learning phase, intervals of worsening and proficiency period and finally progressed to a proficiency interval. The graph for trainees with simulation training was different as there was an initial proficiency interval followed by a worsening interval. This could be attributed to the three trainees who were still in the learning phase after performing more than 20 procedures each. Hence, there may be a need to modify the current training to address the worsening interval for trainees who underwent simulation training. The current training period is short and trainees may benefit from a multiple module based training session. In a systematic review, it was reported that simulation-based education should be based on: (I) a mastery learning approach, in which trainees must reach a specified level of proficiency before advancing to the next phase of instruction; and (II) directed self-regulated learning in a distributed training programme (29). This meant that simulation training should be conducted over a period of time, rather than a one-time training session. In addition, another study commented that dyad practice (training in pairs) was possible and may increase the utility of available simulators (30). Simulation based medical training typically occurs in environments that are relatively stress free compared to the applied context. Since training is most effective when it mimics the actual procedure, inclusion of common clinical stressors in a simulated EBUS-TBNA environment may be beneficial (31).

There were a few limitations in this study. Firstly, it was a single institution, observational study and the results may not be applicable to other institutions. However, the large number of procedures and trainees being evaluated in this study adds to the growing data on the role of simulation training in EBUS-TBNA (11,32-35). Secondly, the impact of patient co-morbidities and physiological fitness as measured by standardized scoring systems (e.g., ASA physical status) were not accounted for. This was due to limitations of the hospital endoscopy reporting system which did not capture this data at the time the registry was set up. Thirdly, we were unable to determine the extent of the involvement of the trainee in the EBUS-TBNA procedures as the procedures were performed in the real-world clinical setting. Different supervisors with different levels of experience in supervising EBUS-TBNA may have different thresholds for allowing trainees to carry on with the procedure. However, this reflects the true outcome in a real-world clinical setting and our study was powered to detect these differences. It should be emphasised that by using performance on real patients as an outcome parameter, this reflects the real effectiveness of medical education.

Although high-tech simulators are exciting, to date most advances in simulation have been made through low-fidelity and low-cost approaches. Our study demonstrated that simulation-based training did not alter the early part of the learning curve associated with EBUS-TBNA, the diagnostic yield, complication rates or scope damage and repair costs. On the contrary, minor complication rates increased amongst the trainees with simulation training. Our study has shown that simulation will not replace the importance of key clinical experiences and that the road to expertise might only be achieved through a combination of structured training and close supervision through apprenticeship. High fidelity simulators despite their high cost may not be the panacea to solving all the challenges with procedural training especially the particular case of EBUS-TBNA.

Conclusions

Simulation training in EBUS-TBNA did not impact the diagnostic yield, major complication rates, scope damage and repair costs. CUSUM analysis showed increasing learning curve in the trainees with simulation training after an initial competency period. The value of simulation training in EBUS-TBNA remains uncertain.

Acknowledgements

We would also like to acknowledge the staff at Singapore General Hospital endoscopy centre and Dr. Angela Takano. This work was supported by Duke-National University of Singapore Medical Student Fellowship.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The study was approved by SingHealth CIRB (No. 2008/458/B and 2011/350/C) and written informed consent was obtained from all patients.

References

1. Jin XR, Ye M, Cai ZZ, et al. Standardized transbronchial needle aspiration procedure for intrathoracic lymph node staging of non-small cell lung cancer. J Thorac Dis 2015;7:S266-71. [PubMed]
2. Colt HG, Davoudi M, Murgu S. Scientific evidence and principles for the use of endobronchial ultrasound and transbronchial needle aspiration. Expert Rev Med Devices 2011;8:493-513. [Crossref] [PubMed]
3. Oki M, Saka H, Kitagawa C, et al. Real-time endobronchial ultrasound-guided transbronchial needle aspiration is useful for diagnosing sarcoidosis. Respirology 2007;12:863-8. [Crossref] [PubMed]
4. Korrungruang P, Oki M, Saka H, et al. Endobronchial ultrasound-guided transbronchial needle aspiration is useful as an initial procedure for the diagnosis of lymphoma. Respir Investig 2016;54:29-34. [Crossref] [PubMed]
5. Herth FJ, Eberhardt R, Vilmann P, et al. Real-time endobronchial ultrasound guided transbronchial needle aspiration for sampling mediastinal lymph nodes. Thorax 2006;61:795-8. [Crossref] [PubMed]
6. von Bartheld MB, van Breda A, Annema JT. Complication Rate of Endosonography (Endobronchial and Endoscopic Ultrasound): A Systematic Review. Respiration 2014;87:343-51. [Crossref] [PubMed]
7. Asano F, Aoe M, Ohsaki Y, et al. Complications associated with endobronchial ultrasound-guided transbronchial needle aspiration: a nationwide survey by the Japan Society for Respiratory Endoscopy. Respir Res 2013;14:50. [Crossref] [PubMed]
8. Eapen GA, Shah AM, Lei X, et al. Complications, Consequences, and Practice Patterns of Endobronchial Ultrasound-Guided Transbronchial Needle Aspiration: Results of the AQuIRE Registry. Chest 2013;143:1044-53. [Crossref] [PubMed]
9. Nguyen S, Ferland N, Beaudoin S, et al. Influence of trainee involvement on procedural characteristics for linear endobronchial ultrasound. Thoracic Cancer 2017;8:517-22. [Crossref] [PubMed]
10. Stather DR, Maceachern P, Chee A, et al. Trainee impact on advanced diagnostic bronchoscopy: An analysis of 607 consecutive procedures in an interventional pulmonary practice. Respirology 2013;18:179-84. [Crossref] [PubMed]
11. Konge L, Clementsen PF, Ringsted C, et al. Simulator training for endobronchial ultrasound: a randomised controlled trial. European Respiratory Journal 2015;46:1140-9. [Crossref] [PubMed]
12. Anantham D, Phua GC, Low SY, et al. Role of endobronchial ultrasound in the diagnosis of bronchogenic cysts. Diagn Ther Endosc 2011;2011. [Crossref] [PubMed]
13. Koh MS, Ong TH, Phua GC, et al. Feasibility of endobronchial ultrasound in mechanically ventilated patients. Ann Acad Med Singapore 2014;43:238-40. [PubMed]
14. Low SY, Koh MS, Ong TH, et al. Use of endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) in the diagnosis of granulomatous mediastinal lymphadenopathy. Ann Acad Med Singapore 2014;43:250-4. [PubMed]
15. Han AY, Tan AH, Koh MS. Utility of endobronchial ultrasound-guided transbronchial needle aspiration in diagnosis of intrathoracic lymphadenopathy in patients with human immunodeficiency virus infection. Biomed Res Int 2015;2015. [Crossref] [PubMed]
16. Sehgal IS, Dhooria S, Aggarwal AN, et al. Training and proficiency in endobronchial ultrasound-guided transbronchial needle aspiration: A systematic review. Respirology 2017;22:1547-57. [Crossref] [PubMed]
17. Kemp SV, El Batrawy SH, Harrison RN, et al. Learning curves for endobronchial ultrasound using cusum analysis. Thorax 2010;65:534-8. [Crossref] [PubMed]
18. Du Rand IA, Blaikley J, Booton R, et al. British Thoracic Society guideline for diagnostic flexible bronchoscopy in adults: accredited by NICE. Thorax 2013;68 Suppl 1:i1-i44. [Crossref] [PubMed]
19. Peduzzi P, Concato J, Feinstein AR, et al. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 1995;48:1503-10. [Crossref] [PubMed]
20. Concato J, Peduzzi P, Holford TR, et al. Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol 1995;48:1495-501. [Crossref] [PubMed]
21. Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol 2007;165:710-8. [Crossref] [PubMed]
22. Yasufuku K, Chiyo M, Koh E, et al. Endobronchial ultrasound guided transbronchial needle aspiration for staging of lung cancer. Lung Cancer 2005;50:347-54. [Crossref] [PubMed]
23. Navani N, Molyneaux PL, Breen RA, et al. Utility of endobronchial ultrasound-guided transbronchial needle aspiration in patients with tuberculous intrathoracic lymphadenopathy: a multicentre study. Thorax 2011;66:889-93. [Crossref] [PubMed]
24. Agarwal R, Srinivasan A, Aggarwal AN, et al. Efficacy and safety of convex probe EBUS-TBNA in sarcoidosis: a systematic review and meta-analysis. Respir Med 2012;106:883-92. [Crossref] [PubMed]
25. Mehta AC, Curtis PS, Scalzitti ML, et al. The High Price off Bronchoscopy. CHEST 98:448-54. [Crossref] [PubMed]
26. Kirkpatrick MB, Smith JR, Hoffman PJ, et al. Bronchoscope damage and repair costs: results of a regional postal survey. Respir Care 1992;37:1256-9. [PubMed]
27. Lunn W, Garland R, Gryniuk L, et al. Reducing Maintenance and Repair Costs in an Interventional Pulmonology Program. Chest 2005;127:1382-7. [PubMed]
28. Colt HG, Crawford SW, Galbraith O 3rd. Virtual Reality Bronchoscopy Simulation. Chest 2001;120:1333-9. [Crossref] [PubMed]
29. Naur TMH, Nilsson PM, Pietersen PI, et al. Simulation-Based Training in Flexible Bronchoscopy and Endobronchial Ultrasound-Guided Transbronchial Needle Aspiration (EBUS-TBNA): A Systematic Review. Respiration 2017;93:355-62. [Crossref] [PubMed]
30. Nilsson PM, Naur TMH, Clementsen PF, et al. Simulation in bronchoscopy: current and future perspectives. Advances in Medical Education and Practice 2017;8:755-60. [Crossref] [PubMed]
31. Andreatta PB, Hillard M, Krain LP. The impact of stress factors in simulation-based laparoscopic training. Surgery 2010;147:631-9. [Crossref] [PubMed]
32. Stather DR, MacEachern P, Chee A, et al. Evaluation of clinical endobronchial ultrasound skills following clinical versus simulation training. Respirology 2012;17:291-9. [Crossref] [PubMed]
33. Stather DR, Maceachern P, Rimmer K, et al. Assessment and learning curve evaluation of endobronchial ultrasound skills following simulation and clinical training. Respirology 2011;16:698-704. [Crossref] [PubMed]
34. Stather DR, Maceachern P, Rimmer K, et al. Validation of an endobronchial ultrasound simulator: differentiating operator skill level. Respiration 2011;81:325-32. [Crossref] [PubMed]
35. Stather DR, MacEachern P, Chee A, et al. Wet laboratory versus computer simulation for learning endobronchial ultrasound: a randomized trial. Can Respir J 2012;19:325-30. [Crossref] [PubMed]