Accuracy of rapid on-site evaluation of endobronchial ultrasound guided transbronchial needle aspirates by respiratory registrars in training and medical scientists compared to specialist pathologists—an initial pilot study

Introduction

Endobronchial ultrasound transbronchial needle aspiration (EBUS-TBNA) is a well-established procedure for the diagnosis and staging of lung and other cancers (1). It is now a component of many guidelines worldwide (2-4). Rapid on-site evaluation (ROSE) of TBNA samples has been shown in previous studies to reduce the rate of non-diagnostic sampling and repeat procedures (5-7). It is also helpful to the respiratory physician in establishing the best sites for biopsy to improve cell block sample sufficiency for ancillary testing, and during mediastinal staging procedures so that over sampling is not necessary. If an N3 node is confirmed positive for malignancy at ROSE, N2 nodes do not require sampling as establishing their status as benign or malignant will not affect staging and subsequent treatment decisions. A paper by Nakajima et al. found ROSE during EBUS did not identify any false positive results but did have a 5.7% false negative rate where diagnosis was obtained at cell block (5). Cell block therefore remains an important component of EBUS-TBNA procedure. EBUS-TBNA cell blocks are also usually sufficient for the molecular genetic tests that are now increasingly required for appropriate lung cancer management with rates of 72–95.5% in literature and 79% at our site (8-12).

In many Australian centres, ROSE requires a specialist pathologist to attend each procedure with associated time and cost implications. In other institutions, medical scientists or pathology registrars perform this role in lieu of the pathologist. However, many sites do not have ready access to a pathologist, pathology registrar or medical scientist. We therefore aimed to evaluate whether respiratory registrars in training at our institution could adequately perform this role after a short period of training as an alternative to pathologists or medical scientists. Some respiratory physicians overseas perform ROSE. This may be because they undertake more established pathology training as part of their medical student education and advanced training programs than is common in Australia. One Italian study found that a single respiratory physician after a period of textbook study was 81% concordant with the reference pathologist but there are no studies evaluating training in ROSE for respiratory trainees (13)

Methods

This study was performed in the Thoracic Procedure Suite at The Royal Adelaide Hospital with ethics approval (there are two numbers for this study with the Royal Adelaide Hospital Ethics Committee: HREC reference number: REC/13/RAH/329; CALHN reference number: R20130803). EBUS-TBNA procedures were performed or supervised by the same thoracic physician for all procedures. Patients planned for EBUS procedures were approached to consent to take part in the study.

A medical scientist, two respiratory registrars and a pathologist performed ROSE on the first two TBNA samples from each recruited patient. Only the first two slides were assessed so as not to prolong procedures due to the multiple reviewers. Each study participant gave consent to have their ROSE performance evaluated. There were three medical scientists available to attend ROSE procedures during this study. These scientists have each completed a 3-year tertiary science degree and completed two years of on the job training required to sit the national exam held by the Australian Society of Cytology to gain the Cytotechnologist certificate which is a requirement at our institution to attend ROSE. As part of the training for this certificate, medical scientists attend 50 supervised ROSE cases. The two respiratory registrars had had limited experience in cytology during their undergraduate university medical degrees and no further training in internship or specialist training. They were in their second and third year of advanced respiratory physician training, which means they had been working as doctors in internship, basic physician training and advanced respiratory training for five years. They received four 1-hour long teaching sessions spread over four weeks from a specialist pathologist. This consisted of the pathologist reviewing the needle aspiration samples at our institution from the previous week under a microscope with separate viewing ports for the two registrars. About 15 slides per training session were reviewed.

Procedures were conducted under light sedation. The Olympus linear EBUS scope UC 180-F was used for all procedures with either a 21- or 22-gauge needle. TBNA smears were prepared by two bronchoscopy suite staff according to departmental protocol. Aspirated material obtained at TBNA was split and smeared between two slides. The first slide was placed into Ethanol 95% solution for later review at the laboratory and the second slide was made available for ROSE.

Slides were prepared using a Romanowsky-type stain (Hemacolour^® Merck, Germany) protocol in the procedure suite by the medical scientist or pathologist to standardise slide preparation so that the results would not be affected by inexperienced slide preparation by the respiratory registrars. Study participants were then asked to determine if each sample was sufficient or insufficient. A specific diagnosis was not required. A sufficient sample was defined as a slide demonstrating lymphocytes in a concentration indicative of lymph node or diagnostic material such as a granuloma or malignant cells. Insufficient samples would not contain any diagnostic material or lymphocytes, but usually only blood or respiratory epithelium. Each ROSE reviewer was blinded to other reviewers’ assessment with the specialist pathologist’s assessment recorded as the gold standard. The time taken to perform ROSE was not recorded but participants were asked to record their response prior to reviewing the subsequent sample as per real world practice. After the first two passes had been reviewed by the study participants and their assessment recorded, the pathologist advised the respiratory physician if sufficient sampling had occurred. If it had not, the procedure would continue as usual with the pathologist alone performing ROSE and indicating to the respiratory physician when sufficient sampling had occurred.

As is standard practice at our institution, after each EBUS-TBNA smear, the needle was flushed with 50 mL of air into Hank’s solution to form a cell block. If an additional EBUS-TBNA was obtained before the pathologist performing ROSE indicated adequate sampling to the respiratory physician, that EBUS-TBNA would also be flushed directly into Hank’s solution to improve cell block yield, rather than preparing further slides for ROSE.

Other parameters recorded during this study were number of needle passes per patient and final diagnosis. Cohen’s Kappa Statistic was used to assess interrater variability; medical scientist performance versus pathologist and respiratory registrar performance versus pathologist.

Results

Twenty-five patients were recruited for this initial pilot study thereby obtaining fifty TBNA samples for statistical analysis. Twenty-three cases had lymphadenopathy. In two cases, the patients did not have enlarged lymph nodes but had a moderately FDG avid lymph node at PET scan and required biopsy prior consideration of surgical resection. One of these cases found benign lymphoid tissue, which was again confirmed during lymph node biopsy at lobectomy. The other case confirmed squamous cell carcinoma and the patient did not proceed to surgical resection.

Specialist pathologist review determined that 16/50 (32%) TBNA were sufficient and 34/50 (68%) insufficient. Medical scientists were 90% concordant with the pathologist. This kappa coefficient found good concordance between the pathologist and medical scientists at 0.774 with a 95% CI, 0.587–0.961. The respiratory registrars were 78% (K =0.568, 95% CI, 0.338–0.798) and 72% (K =0.448; 95% CI, 0.222–0.674) concordant, respectively. These kappa coefficients reflect moderate concordance between the registrars and the pathologist. These results are summarised in Table 1. The respiratory registrars were more likely to be concordant with the pathologist for sufficient TBNA slides than insufficient slides. Registrar 1 was concordant with pathologist on 81% (13/16) of sufficient slides and 76% (26/34) of insufficient slides. Registrar 2 was concordant with the pathologist on 88% (14/16) of sufficient slides and 65% (22/34) of insufficient slides. Medical scientists were more likely to be concordant with the pathologist for both sufficient (88%, 14/16) and insufficient (91%, 31/34) slides than the respiratory registrars.

Table 1 Concordance of study participants with pathologist opinion
Full table

Although 12/25 (48%) of patients had had adequate sampling within the first two linear EBUS-TBNA slides, adequacy was established in 21/25 (84%) patients during their procedure. Most patients that did not have adequate smears within the first two passes (8/13) had adequate smears from subsequent passes. The mean number of passes was 4.9 (range, 3–7). One case had inadequate slide cytology but cells suspicious for malignancy were identified on cell block. This information was clinically sufficient for that patient to proceed to lobectomy where NSCLC adenocarcinoma with N1 nodal disease was diagnosed. The most frequently biopsied lymph node was station 7 (20/50 cases). Other lymph nodes biopsies included R4 (12/50), L4 (4/50), R10 (8/50), and L10 (6/50).

Four patients had insufficient results from their procedure. These patients had follow-up that established a diagnosis. Two patients had a repeat linear EBUS-TBNA procedure that was diagnostic of a benign condition. One patient had a diagnostic endobronchial biopsy during the same procedure as the linear EBUS-TBNA. One patient was monitored with serial CT scans with improvement in their lymphadenopathy observed over time. Malignancy was found in 14/25 (56%) of patients and a benign process in 11/25 (44%) of patients. On follow-up of benign cases, no false negative results were found. Of the eleven cases with benign results during this study, eight had stable/or resolved lymphadenopathy on subsequent CT scan. Two cases had confirmed benign lymph nodes when biopsied at time of surgical lung resection for early stage carcinoid and NSCLC tumours. One case was lost to follow-up as their follow-up was continued at another hospital. Table 2 lists details of diagnoses from linear EBUS TBNA obtained during the study.

Table 2 Diagnosis from linear EBUS-TBNA procedure
Full table

Registrar 1 was more concordant with pathologist for benign cases (19/22, 86% for benign cases, 20/28, 71% for malignant cases.) Registrar 2 was more concordant with the pathologist for malignant cases (24/48, 86% for malignant cases, 12/22, 56% for benign cases). The medical scientists were concordant with the pathologist in 25/28 (89%) for malignant cases and 20/33 (91%) of benign cases.

Discussion

ROSE of EBUS-TBNA biopsies is a useful tool in ensuring sample sufficiency and improves diagnostic accuracy (5-7). It also reduces the number of passes taken per patient, thereby potentially reducing procedure time and complication risk for the patient (5-7). Fewer passes per patient also improves laboratory resource utilisation (14). Studies evaluating cost-effectiveness of ROSE have emphasised that cost-effectiveness will vary between institutions dependant on local costs of equipment and staffing, but ROSE is most likely to be cost-effective by avoiding repeat procedures if there are high fixed costs per procedure, a low per-pass adequacy rate and a short time per needle pass (15). If the repeat procedure is mediastinoscopy rather than repeat EBUS, the savings are greater (15,16).

Not all sites have access to an onsite pathology service, or it may not be practical for a pathologist to be available to perform ROSE. Tele-cytopathology is used in some centres and studies have found it to be comparable to conventional ROSE microscopy at EBUS (17-19). There are, however, some practical difficulties with tele-cytopathology that may have limited its more widespread implementation (20). Setting aside technology considerations, if static tele-cytopathology is used, someone on-site will need to be able operate a microscope and be relied upon not to miss important cytology findings. If dynamic video microscopy is used, the pathologist can remotely direct a person operating the microscope to find the region of interest, but that person will still need to be proficient at operating a microscope (20).

This study found that medical scientists have good concordance with specialist pathologists in assessing linear EBUS-TBNA sufficiency. Similar results have been found in other studies evaluating medical scientists’ performance of ROSE on radiology-guided FNA biopsies, endoscopic ultrasound guided FNA of pancreatic lesions and thyroid FNA (21-23). This study confirms that medical scientists can confirm adequate sampling at ROSE but, at many sites, they do not advise the respiratory physician of a likely diagnosis before a pathologist reviews the case. In rare urgent cases it can be preferable to have an immediate provisional diagnosis and for these cases an onsite pathologist is preferable.

Four hours of training was insufficient for respiratory registrars to become as accurate as medical scientists in assessing linear EBUS slides for biopsy sufficiency but registrars are likely to improve with further experience and/or training sessions. It is not clear from this study how much further training would be required, but it would be difficult for respiratory registrars to match the medical scientists’ fifty pathologist supervised training cases in addition to existing respiratory training requirements. Respiratory registrars also do not have the background science training of medical scientists. Respiratory registrars were more likely to assess slides as sufficient when they were not, than to accurately identify insufficient slides. This is especially problematic as respiratory registrars may therefore be prone to ceasing procedures prematurely before adequate sampling has occurred and risk non-diagnostic procedures. The respiratory registrars in the study did not stain the slides and training as there was concern that poor slide preparation would jeopardise slide interpretation. Training in this would also be required in order to perform ROSE without scientist assistance.

Conclusions

ROSE is an important part of the EBUS procedure to ensure sample sufficiency and improve diagnostic accuracy. Medical scientist review of ROSE samples is not significantly different to a specialist pathologist and is an acceptable alternative at sites where this is available. Respiratory registrars are not an alternative to a pathologist for ROSE after a short period of training. More intensive training would be required than undertaken in this study, which would be difficult to facilitate during a busy respiratory training program.

Acknowledgements

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: This study was performed in the Thoracic Procedure Suite at The Royal Adelaide Hospital with ethics approval (There are two numbers for this study with the Royal Adelaide Hospital Ethics Committee: HREC reference number: REC/13/RAH/329; CALHN reference number: R20130803). EBUS-TBNA procedures were performed or supervised by the same thoracic physician for all procedures. Patients planned for EBUS procedures were approached to consent to take part in the study.

References

1. Gu P, Zhao YZ, Jiang LY, et al. Endobronchial ultrasound-guided transbronchial needle aspiration for staging of lung cancer: a systematic review and meta-analysis. Eur J Cancer 2009;45:1389-96. [Crossref] [PubMed]
2. Rivera MP, Mehta AC, American College of Chest P. Initial diagnosis of lung cancer: ACCP evidence-based clinical practice guidelines (2nd edition). Chest 2007;132:131S-48S.
3. Vansteenkiste J, De Ruysscher D, Eberhardt WE, et al. Early and locally advanced non-small-cell lung cancer (NSCLC): ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol 2013;24 Suppl 6:vi89-98. [Crossref] [PubMed]
4. Vilmann P, Clementsen PF, Colella S, et al. Combined endobronchial and oesophageal endosonography for the diagnosis and staging of lung cancer. Eur Respir J 2015;46:40-60. [Crossref] [PubMed]
5. Nakajima T, Yasufuku K, Saegusa F, et al. Rapid on-site cytologic evaluation during endobronchial ultrasound-guided transbronchial needle aspiration for nodal staging in patients with lung cancer. Ann Thorac Surg 2013;95:1695-9. [Crossref] [PubMed]
6. Oki M, Saka H, Kitagawa C, et al. Rapid on-site cytologic evaluation during endobronchial ultrasound-guided transbronchial needle aspiration for diagnosing lung cancer: a randomized study. Respiration 2013;85:486-92. [Crossref] [PubMed]
7. Trisolini R, Cancellieri A, Tinelli C, et al. Randomized Trial of Endobronchial Ultrasound-Guided Transbronchial Needle Aspiration with and without Rapid on-Site Evaluation for Lung Cancer Genotyping. Chest 2015;148:1430-7. [Crossref] [PubMed]
8. Schuurbiers OC, Looijen-Salamon MG, Ligtenberg MJ, et al. A brief retrospective report on the feasibility of epidermal growth factor receptor and KRAS mutation analysis in transesophageal ultrasound- and endobronchial ultrasound-guided fine needle cytological aspirates. J Thorac Oncol 2010;5:1664-7. [Crossref] [PubMed]
9. Garcia-Olive I, Monso E, Andreo F, et al. Endobronchial ultrasound-guided transbronchial needle aspiration for identifying EGFR mutations. Eur Respir J 2010;35:391-5. [Crossref] [PubMed]
10. Nakajima T, Yasufuku K, Suzuki M, et al. Assessment of epidermal growth factor receptor mutation by endobronchial ultrasound-guided transbronchial needle aspiration. Chest 2007;132:597-602. [Crossref] [PubMed]
11. Santis G, Angell R, Nickless G, et al. Screening for EGFR and KRAS mutations in endobronchial ultrasound derived transbronchial needle aspirates in non-small cell lung cancer using COLD-PCR. PLoS One 2011;6. [Crossref] [PubMed]
12. Hopkins E, Moffat D, Parkinson I, et al. Cell block samples from endobronchial ultrasound transbronchial needle aspiration provide sufficient material for ancillary testing in lung cancer-a quaternary referral centre experience. J Thorac Dis 2016;8:2544-50. [Crossref] [PubMed]
13. Bonifazi M, Sediari M, Ferretti M, et al. The role of the pulmonologist in rapid on-site cytologic evaluation of transbronchial needle aspiration: a prospective study. Chest 2014;145:60-5. [Crossref] [PubMed]
14. Collins BT, Chen AC, Wang JF, et al. Improved laboratory resource utilization and patient care with the use of rapid on-site evaluation for endobronchial ultrasound fine-needle aspiration biopsy. Cancer Cytopathol 2013;121:544-51. [Crossref] [PubMed]
15. Schmidt RL, Walker BS, Cohen MB. When Is Rapid On-Site Evaluation Cost-Effective for Fine-Needle Aspiration Biopsy? PLoS One 2015;10. [Crossref] [PubMed]
16. Bruno P, Ricci A, Esposito MC, et al. Efficacy and cost effectiveness of rapid on site examination (ROSE) in management of patients with mediastinal lymphadenopathies. Eur Rev Med Pharmacol Sci 2013;17:1517-22. [PubMed]
17. Marotti JD, Johncox V, Ng D, et al. Implementation of telecytology for immediate assessment of endoscopic ultrasound-guided fine-needle aspirations compared to conventional on-site evaluation: analysis of 240 consecutive cases. Acta Cytol 2012;56:548-53. [Crossref] [PubMed]
18. Bott MJ, James B, Collins BT, et al. A Prospective Clinical Trial of Telecytopathology for Rapid Interpretation of Specimens Obtained During Endobronchial Ultrasound-Fine Needle Aspiration. Ann Thorac Surg 2015;100:201-5; discussion 205-6. [Crossref] [PubMed]
19. Khurana KK, Kovalovsky A, Wang D, et al. Feasibility of dynamic telecytopathology for rapid on-site evaluation of endobronchial ultrasound-guided transbronchial fine needle aspiration. Telemed J E Health 2013;19:265-71. [Crossref] [PubMed]
20. Pantanowitz L, Wiley CA, Demetris A, et al. Experience with multimodality telepathology at the University of Pittsburgh Medical Center. J Pathol Inform 2012;3:45. [Crossref] [PubMed]
21. Burlingame OO, Kesse KO, Silverman SG, et al. On-site adequacy evaluations performed by cytotechnologists: correlation with final interpretations of 5241 image-guided fine-needle aspiration biopsies. Cancer Cytopathol 2012;120:177-84. [Crossref] [PubMed]
22. Olson MT, Ali SZ. Cytotechnologist on-site evaluation of pancreas fine needle aspiration adequacy: comparison with cytopathologists and correlation with the final interpretation. Acta Cytol 2012;56:340-6. [Crossref] [PubMed]
23. Olson MT, Tatsas AD, Ali SZ. Cytotechnologist-attended on-site adequacy evaluation of thyroid fine-needle aspiration: comparison with cytopathologists and correlation with the final interpretation. Am J Clin Pathol 2012;138:90-5. [Crossref] [PubMed]