Learning curve for electromagnetic navigation bronchoscopy-guided microwave ablation: a cumulative sum analysis

Introduction

Lung cancer has put a heavy burden on public health worldwide for its high morbidity and mortality (1). With the development of imaging technology, the detection rate of pulmonary nodules, a common imaging manifestation of lung cancer, has been increasing (2). Electromagnetic navigation bronchoscopy (ENB) transbronchial microwave ablation (MWA), a novel technique that is capable of guiding MWA catheter to the target lesion through the airway by tracking electromagnetic signals, has showed its potential in the management of pulmonary nodules (3-6). Though the safety, feasibility, and effectiveness of ENB-guided MWA in treating pulmonary nodules have been demonstrated in previous studies, the learning curve of this technique remains unclear (4-6).

Learning curve analysis is essential for evaluating the difficulty of learning a new technique and the capability of the operator by quantitatively analyzing the rate of skill or knowledge acquisition within a certain period of time and detecting the change point of the ability. This method has been widely used in medical field and has been applied to date. Cumulative sum (CUSUM) method is one of the most commonly used methods for learning curve analysis, which has been applied to the learning of endobronchial ultrasound in evaluating novice operators (7,8). In brief, CUSUM method bases on a standard target level, by calculating the cumulative difference in sequence, the sequence statistics are generated, and the learning curves can be plotted accordingly (9). By applying CUSUM method, learning curve of operation time (OT), success rate, and complication rate of ENB-guided MWA for pulmonary nodules can be sketched, and the learning pattern of this technique can be analyzed. In this study, we aimed to analyze the learning curve of ENB-guided MWA for pulmonary nodules employing CUSUM method, to evaluate the difficulty of learning this technique and provide reference for clinical practice. We present this article in accordance with the STROBE reporting checklist (available at https://jtd.amegroups.com/article/view/10.21037/jtd-2024-2162/rc).

Methods

Study design

This study was designed as a retrospective study to analyze the learning curve of ENB-guided MWA for pulmonary nodules employing CUSUM method. The flowchart of the study is shown in Figure 1. Two experienced thoracic surgeons who were skilled at bronchoscopy but never learned electromagnetic navigation system were included in this study. During the learning process, they were instructed by a skilled operator who had performed more than 300 cases of ENB-guided MWA for pulmonary nodules. When following situations occurred, the operation was determined as failure: (I) the operator self-assessed that the navigation catheter could not reach the target lesion. The skilled operator would take over the operation; (II) the operator self-assessed that the navigation catheter had reached the target lesion, but the skilled operator assessed that the catheter had not reached the target lesion. The skilled operator would take over the operation; (III) the operator self-assessed that the navigation catheter had reached the target lesion, the skilled operator confirmed it, and ablation was conducted, but the follow-up computed tomography (CT) images showed that the lesion was not ablated successfully. Accordingly, when the operator completed the operation and the follow-up showed that the lesion was ablated successfully, the operation was considered as success. Ablation success was defined as the complete distortion in shape of original nodule and continuous reduction in size of ablation lesion in CT images for at least six months, which requires for CT surveillance during follow-up (10). Considering the potential association between ablation failure and nodule progression, the nodules with failed ablation would be excised via video-assisted thoracic surgery (VATS) typically. However, if patient’s health status did not permit VATS, alternatives, such as active surveillance and stereotactic body radiation therapy, might be employed as deemed appropriate (11). The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Institutional Review Board of Tongji Medical College of Huazhong University of Science and Technology (TJ-IRB202502073) and individual consent for this retrospective analysis was waived.

Figure 1 The flowchart of study. ENB, electromagnetic navigation bronchoscopy; MWA, microwave ablation.

ENB-guided MWA procedure

Before the initiation of treatment, a multidisciplinary team consultation involving thoracic surgery, pulmonology, oncology, and radiology should be conducted for the patient. The patient’s preferences should be fully taken into account, and the treatment plan should be selected based on the principle of maximizing the patient’s benefit.

Patient was placed in the supine position on operating table with electromagnetic positioning plate equipped. General anesthesia was conducted and laryngeal mask airway (LMA) was built. Electromagnetic signal sensors were fixed on chest wall. A preoperative cone-beam computed tomography (CBCT) would be conducted through CBCT system (Cios Spin; Siemens, Erlangen, Germany). Then, ENB-guided MWA procedure began through the ENB system (superDimension Navigation System, version 7.0; Medtronic, Minneapolis, the United States). When the navigation catheter was guided to the target lesion following pre-planned optimal operating path, a second CBCT would be conducted to confirm the position of the navigation catheter. If CBCT images showed that the position of catheter deviated, the navigation catheter would be adjusted until reached accurately. Then, the navigation catheter would be replaced with an ablation catheter and began MWA using a microwave generator (MTC-3; VISON MEDICAL, Nanjing, China) with a microwave frequency of 2,450±50 MHz and a power of 40 to 80 W for 2–4 minutes per site according to the tumor size, shape, and the distance to the pleura. CBCT would be conducted to check the ablation zone. Multiple ablations might be conducted if pre-planned or the ablation zone was not large enough to cover the nodules. After the ablation, the patient would be revived from anesthesia and transferred to the ward, or receive surgery as scheduled.

Data collection

The study retrospectively collected the clinical data of patients who underwent ENB-guided MWA for pulmonary nodules in the Department of Thoracic Surgery, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology from January 2023 to August 2024. The data extracted included the basic information, present history, past history, imaging data, operation records, etc. The information of the operators, the sequence of cases, the operation time, the success or failure of the operation, and the occurrence of complications were also recorded. The inclusion criteria were as follows: (I) pulmonary nodules were confirmed by chest CT; (II) pulmonary nodules were confirmed of or with high possibility of malignancy; (III) patients with multiple pulmonary nodules, enlarged pulmonary nodules or metachronous second primary pulmonary nodules after previous lung surgery, or resectable lung cancer but poor cardio-pulmonary function who are not able to tolerate surgery; (IV) operation was conducted by the two specific operators. The exclusion criteria were: (I) nodules were confirmed metastases; (II) patients refused to participate in this study; (III) the clinical data were incomplete; (IV) patients were lost to follow-up.

CUSUM method and parameter setting

For the analysis of OT, the cumulative sum of OT is calculated (12). The formula employed is as follows: CUSUMOT=∑i=1n(xi−μ), where CUSUM_OT is the cumulative sum of OT, x_i is the OT of the i-th case, and µ is the mean OT, which is set as the mean OT of the skilled operator.

For the analysis of success rate, the statistic CUSUM_Scoreis calculated (13). The CUSUM_Score is the cumulative sum of the increment for each failure and the decrement for each success, which is calculated as follows:

CUSUMScore=i×(1−s)−j×s[1]

where i is the number of failures, j is the number of successes, s is the decrement for each success, and 1−s is the increment for each success. The calculation of s is based on the following formulas:

S=QP+Q[2]

P=lnp1p0[3]

Q=ln1−p01−p1[4]

where p0 is the pre-set acceptable failure rate, and p1 is the pre-set unacceptable failure rate. In this study, p0 and p1 were set according to the successful rate of the skilled operator. To detect the change point, the lower boundary limit h0 and the upper boundary limit h1 need to be calculated. The formulas are as follows:

h0=bP+Q[5]

h1=aP+Q[6]

a=ln1−βα[7]

b=ln1−αβ[8]

where α is the first type error rate, and β is the second type error rate. According to the previous literature, α and β were set as 0.1 (13). Since α and β are equal, h0 and h1 are also equal, that is, h0=h1=h. The significance of h is to judge whether the success rate of operator has significantly changed. The CUSUM score analysis is based on a null hypothesis: there is no significant difference between the actual failure rate of the operator and the acceptable true failure rate. On the CUSUM learning curve, a set of parallel lines are drawn with h as the interval. The learning curve drawn may cross these parallel lines. When the curve is between the adjacent parallel lines, it is not able to judge whether the success rate has significantly changed; when the curve continues to rise and crosses the upper parallel line, the null hypothesis is rejected, and it can be considered that the actual failure rate is higher than the acceptable true failure rate; conversely, when the curve continues to decline and crosses the lower parallel line, the null hypothesis is accepted, and it can be considered that the actual failure rate is not significantly different from the acceptable true failure rate. Based on this, the change point can be detected to judge whether the operator is qualified. Similarly, this method can be extended to the occurrence rate of complications (14). In this study, since there were no death cases and few cases transferred to other treatments, the CUSUM analysis of adjusted risk was not conducted (15). The parameter setting of the CUSUM analysis method used in this study is shown in Table 1.

Statistical analysis

Median (range) was used to describe continuous variables, and the count (percentage) was used to describe categorical variables. The Kruskal-Wallis test, Chi-squared test, Yate’s adjusted Chi-squared test, or Fisher’s exact test were employed when appropriate to compare the baseline data of patients and nodules treated by the two operators. A two-sided P value of less than 0.05 was considered statistically significant. R 4.4.1 was used for all statistical analyses.

Results

Baseline data of patients and nodules

A total of 144 patients were treated by the two operators, with 68 patients treated by operator A and 76 patients treated by operator B. The baseline data of patients and nodules treated by the two operators including sex, age, height, weight, smoking history, comorbidities, Eastern Cooperative Oncology Group (ECOG) performance status scale, nodule size, nodule position, nodule density classification, and distance to pleura are shown in Table 2. By comparing the baseline data, no significant statistical difference was found between the two operators.

Operation outcomes

The operation outcomes of the two operators are shown in Table 3. For Operator A, the median operation time was 60 minutes (range, 30 to 235 minutes). The success rate was 85.3%. 10 cases failed, among them, 4 cases were terminated by himself because he self-assessed that the navigation catheter could not reach the target lesion on his own; 5 cases were assessed by the skilled operator as not accurately reaching the target lesion, and 1 case was found to be off-target during follow-up. The complications included 3 cases of infection, and 2 cases of intrapulmonary hemorrhage. For Operator B, the median operation time was 53 minutes (range, 30 to 228 minutes). The success rate was 84.2%. 12 cases failed, including 6 cases terminated by himself, 3 cases assessed as failure by the skilled operator, and 3 cases found to be off-target postoperatively. No intergroup differences were found in the operation outcomes and complications between the two operators.

Learning curve analysis

Figure 2A shows the relationship between operation time and the number of cases. The operation time of the two operators showed a downward trend with the increase of the number of cases, and tended to stabilize after approximate 35 cases. Though after the stabilization, the operation time still fluctuated and presented with several peaks, indicating that the operation time was too long in these cases. Figure 2B shows the relationship between the cumulative sum of operation time and the number of cases. The change points of the two operators were at the 40th and 33rd cases, respectively. After the change points, the curve showed a downward trend, indicating that the operation time required by the operators was less than the preset average time.

Figure 2 Learning curve of operation time. (A) Line chart of operation time versus number of cases. (B) Line chart of cumulative sum of operation time versus number of cases. At the 40th and 33rd cases for operator A and B respectively, the curve reversed from an uptrend to a downtrend.

Figure 3A shows the relationship between the CUSUM score of operation success and the number of cases. Initially, the learning curves of the two operators showed an upward trend and exceeded h, indicating that the failure rate of the two operators was higher than the true failure rate during this period. The first change points of Operator A and Operator B were at the 22nd and 13th cases, respectively. After the first change points, the curves showed a downward trend, indicating that the operation success rate of the operators gradually increased. At the second change point of 48th and 55th cases, the learning curves of Operator A and Operator B crossed h, indicating that the operation success rate of the two operators was not significantly different from the acceptable true failure rate at this time.

Figure 3 Learning curve of operation success rate and complication rate. (A) Line chart of cumulative sum score of success versus number of cases. At the 22nd and 13th cases for operator A and B respectively, the curve reversed from an uptrend to a downtrend. At the 48th and 55th cases for operator A and B respectively, the curves cross h. (B) Line chart of cumulative sum score of complications versus number of cases. At the 67th case, the curve for operator A cross −h. −h, h, 2 h represent the boundary limits helping to judge whether the success rates or complication rates of operators have significantly changed.

Figure 3B shows the relationship between the CUSUM score of complications and the number of cases. The learning curves of the two operators fluctuated downward. The learning curves of the two operators did not exceed h. The learning curve of Operator A crossed −h at the 67th case, indicating that the complication rate of the patients treated by Operator A was not significantly different from the acceptable complication rate during the entire learning process. For Operator B, the learning curve showed a downward trend but did not cross −h.

Discussion

Previous publications have preliminarily demonstrated the safety, feasibility, and efficacy of ENB-guided MWA for pulmonary nodules (3-6). In this study, we further evaluated the learning curve of this technique, analyzed the change points of the two operators in terms of operation time, operation success rate, and complication rate, and explored the learning pattern of this technique.

Operation time indicates the proficiency of operating the ENB system. With the increase of the number of cases, the operation time of the two operators showed a trend of downward initially and stabilized after change point. At the preliminary stage (about the initial 21 cases), the learning curves of the two operators coincided with each other. At this period, operators might require a longer theoretical operation time due to the unfamiliarity with the technique. Considering the impact of prolonged operation time on patients and the teaching purpose at the beginning of the learning process, the instructor would timely intervene, which affected the operation time and led to the similarity of the operation time of the two operators. After the instructor reduced the intervention, the curves of the two operators began to separate, indicating that the two operators had different mastery of the technique. Between the reduction of instruction and the change point, the operation time of the two operators showed few peaks, which represented the process of dealing with difficulties without intervention of instructor, which is necessary in the learning process. The change points of the two operators were different, at the 33rd and 40th cases, respectively, but the number of cases required was relatively small, less than the number of cases required to master some conventional thoracic surgery operations, indicating that this technique was relatively easy to learn (9,16). After the change points, although the operation time was relatively stable, there were still several large fluctuation points. These fluctuations may be due to the operators encountering some special situations during the operation, such as some special lung subsegments with large bronchial angles, which makes the catheter hard to enter due to lack of accessibility. Since this situation may occur for experienced operators, it cannot be concluded that the capability of operators declined. Recently, Shape-sensing robotic bronchoscopy shows its superiority in transbronchial approach for its excellent accessibility (17). By combining, technical success of MWA should be significantly improved and number of procedures needed to achieve proficiency should be reduced with better stability.

The analysis on operation success rate indicates the ability of transbronchial MWA, including the capability of properly judging the position of the catheter and selecting the ablation power and duration. The first change point where the trend of line chart reversed from upward to downward is the 22nd case for Operator A and the 13th case for Operator B. This change point indicates that the increase rate of successful cases has exceeded the increase rate of failed cases. For ENB-guided MWA, this change indicates that the operators have been gradually capable of correctly judging whether the catheter has reached the target lesion and selecting the ablation power and duration reasonably. However, due to the high failure rate at early learning period, a certain number of cases are required to reach a success rate close to the true failure rate. Operator A and Operator B required 48 and 55 cases to cross h, where the second change point lay, respectively, with a certain gap between them. The gap formed because Operator B had a higher failure rate in the early learning stage, and even after the curve showed a downward trend, there were still several failed cases, indicating poor stability. After the second change point, the operation success rate of the two operators was not significantly different from the acceptable true failure rate, which means that the two operators have mastered the operation in terms of operation success rate.

For the learning curve of complication rate, the curve of the two operators fluctuated in a downward trend. The learning curves for both operators did not exceed h, and the learning curve of Operator A crossed −h at the 67th case, indicating that the complication rate of the patients treated by Operator A was not significantly different from the acceptable complication rate during the entire learning process. For Operator B, the learning curve did not cross −h. Considering the downward trend, it can be predicted that the complication rate of Operator B would cross −h with the increase of cases. This result further demonstrates the safety of ENB-guided MWA for treating pulmonary nodules. Even for beginners, the operation can be performed safely under proper instruction, indicating its potential for popularization.

Shi et al. conducted a learning curve analysis of ENB-guided preoperative localization of pulmonary nodules (18). Compared with our study, the two studies showed some consistency and differences. In terms of operation time, Shi et al. showed that the cumulative operation time began to decrease after the 47th case, which was close to the result of our study. Although ENB-guided MWA in our clinical center added additional steps such as CBCT employment and ablation operation compared with the localization operation in the study of Shi et al., these steps did not increase the difficulty of learning. However, these additional steps would increase the operation time, as shown in the results of the two studies. In terms of operation success rate, Shi et al. showed that the operator mastered the technique after the 31st case, while the change points of the two operators in our study were the 48th and 55th cases, respectively, with a large gap. This may be due to the difference in the accuracy requirements of the catheter position between localization and ablation operations. The accuracy requirement is lower for localization since localization only indicates the approximate position of the lesion, but ablation requires the catheter to reach the lesion as accurate as possible to make sure the ablation zone covers the lesion. Correspondingly, the difficulty will increase, and more cases are needed for training.

There are some limitations in this study. Firstly, the two operators included in this study were both experienced thoracic surgeons, while the learning curve of this technique for junior residents may be different, hence, the results cannot be generalized in residents. Secondly, the instruction for the two operators were not uniform but individualized, which may affect the results of the learning curve. Finally, this study was a single-center study with few operators included, and the conclusion cannot be generalized to other medical centers and operators. Further multi-center studies are needed to verify the results.

Conclusions

This study analyzed the learning curve of ENB-guided MWA for treating pulmonary nodules employing CUSUM method. The results showed that ENB-guided MWA was relatively easy to learn, and should be mastered within a few cases. The complication rate of the operators was not significantly different from the acceptable complication rate, indicating its safety. This result provides a reference for the popularization of ENB-guided MWA for treating pulmonary nodules.

Acknowledgments

None.

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at https://jtd.amegroups.com/article/view/10.21037/jtd-2024-2162/rc

Data Sharing Statement: Available at https://jtd.amegroups.com/article/view/10.21037/jtd-2024-2162/dss

Peer Review File: Available at https://jtd.amegroups.com/article/view/10.21037/jtd-2024-2162/prf

Funding: None.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jtd.amegroups.com/article/view/10.21037/jtd-2024-2162/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Institutional Review Board of Tongji Medical College of Huazhong University of Science and Technology (TJ-IRB202502073) and individual consent for this retrospective analysis was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Bray F, Laversanne M, Sung H, et al. Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2024;74:229-63. [Crossref] [PubMed]
2. Mazzone PJ, Lam L. Evaluating the Patient With a Pulmonary Nodule: A Review. JAMA 2022;327:264-73. [Crossref] [PubMed]
3. Bao F, Yu F, Wang R, et al. Electromagnetic bronchoscopy guided microwave ablation for early stage lung cancer presenting as ground glass nodule. Transl Lung Cancer Res 2021;10:3759-70. [Crossref] [PubMed]
4. Zeng C, Fu X, Yuan Z, et al. Application of electromagnetic navigation bronchoscopy-guided microwave ablation in multiple pulmonary nodules: a single-centre study. Eur J Cardiothorac Surg 2022;62:ezac071. [Crossref] [PubMed]
5. Qu R, Tu D, Hu S, et al. Electromagnetic Navigation Bronchoscopy-Guided Microwave Ablation Combined With Uniportal Video-Assisted Thoracoscopic Surgery for Multiple Ground Glass Opacities. Ann Thorac Surg 2022;113:1307-15. [Crossref] [PubMed]
6. Chan JWY, Lau RWH, Chang ATC, et al. Concomitant electromagnetic navigation transbronchial microwave ablation of multiple lung nodules is safe, time-saving, and cost-effective. JTCVS Tech 2023;22:265-72. [Crossref] [PubMed]
7. Kemp SV, El Batrawy SH, Harrison RN, et al. Learning curves for endobronchial ultrasound using cusum analysis. Thorax 2010;65:534-8. [Crossref] [PubMed]
8. Kheir F, Alokla K, Myers L, et al. Endobronchial Ultrasound-Transbronchial Needle Aspiration of Mediastinal and Hilar Lymphadenopathy Learning Curve. Am J Ther 2016;23:e1016-9. [Crossref] [PubMed]
9. Matsuura N, Igai H, Ohsawa F, et al. Learning curve for uniportal thoracoscopic pulmonary segmentectomy: how many procedures are required to acquire expertise? Transl Lung Cancer Res 2023;12:1466-76. [Crossref] [PubMed]
10. Anderson EM, Lees WR, Gillams AR. Early indicators of treatment success after percutaneous radiofrequency of pulmonary tumors. Cardiovasc Intervent Radiol 2009;32:478-83. [Crossref] [PubMed]
11. Yan P, Lyu X, Wang S, et al. Insufficient ablation promotes the metastasis of residual non-small cell lung cancer (NSCLC) cells via upregulating carboxypeptidase A4. Int J Hyperthermia 2021;38:1037-51. [Crossref] [PubMed]
12. Maayan O, Pajak A, Shahi P, et al. Percutaneous Transforaminal Endoscopic Discectomy Learning Curve: A CuSum Analysis. Spine (Phila Pa 1976) 2023;48:1508-16. [Crossref] [PubMed]
13. Toennesen LL, Vindum HH, Risom E, et al. Learning Curves for Electromagnetic Navigation Bronchoscopy Using CUSUM Analysis. J Bronchology Interv Pulmonol 2022;29:164-70. [Crossref] [PubMed]
14. Morató O, Poves I, Burdío F, et al. Evaluation of the learning curve for laparoscopic pancreatoduodenectomy by CUSUM analyses. Cohort study. Int J Surg 2020;80:61-7. [Crossref] [PubMed]
15. Gil PJ, Ruiz-Manzanera JJ, Ruiz de Angulo D, et al. Learning Curve for Laparoscopic Sleeve Gastrectomy: a Cumulative Summation (CUSUM) Analysis. Obes Surg 2022;32:2598-604. [Crossref] [PubMed]
16. Dimitrovska NT, Bao F, Yuan P, et al. Learning curve for two-port video-assisted thoracoscopic surgery lung segmentectomy. Interact Cardiovasc Thorac Surg 2022;34:402-7. [Crossref] [PubMed]
17. Abdelghani R, Espinoza D, Uribe JP, et al. Cone-beam computed tomography-guided shape-sensing robotic bronchoscopy vs. electromagnetic navigation bronchoscopy for pulmonary nodules. J Thorac Dis 2024;16:5529-38. [Crossref] [PubMed]
18. Shi J, He J, He J, et al. Electromagnetic navigation-guided preoperative localization: the learning curve analysis. J Thorac Dis 2021;13:4339-48. [Crossref] [PubMed]