Learning curve of mitral valve repair: cumulative sum failure analysis from single high-volume center

Introduction

Mitral valve (MV) repair has become the gold standard for treating degenerative mitral regurgitation (MR). Previous studies (1-3) have demonstrated its safety, repeatability, and durability. The early literature (3,4) suggested that a nearly 100% repair rate is achievable for MV prolapse at reference centers. However, we learned from a Society of Thoracic Surgeons report, that from 2011 to 2014 the repair rate for MR due to annular or degenerative disease was only 75% (5). Clearly, wide variance still exists among centers, and MV replacement may still be the procedure of choice for inexperienced surgeons and for low-volume centers.

Although the overall risk of recurrent MR is low for degenerative MR after MV repair, recurrent moderate/severe MR remains a serious problem due to its adverse outcomes (6). Furthermore, any remaining, even mild, MR at the end of the repair is a strong independent predictor for subsequent failure, suggesting that the surgeon’s performance is associated with long-term outcomes. Chikwe et al. reported that a surgeon’s performance is strongly affected by both their own capabilities and the center’s volume of such cases (7). Although the individual surgeon’s experience is a determinant of repair rates and long-term outcomes, this study suggested that low-volume surgeons’ performance could be improved in high-volume, high-repair-rate centers.

The spectrum of mitral disease in China is different from that in Western, developed countries. Whereas rheumatic mitral stenosis used to be the primary cause of mitral disease in China, the prevalence of degenerative mitral disease has increased significantly in recent years. As the use of MV repair was instituted relatively late in China, surgeons with little experience might be reluctant to adopt this operation because of the initial learning curve. They may argue that the learning curve and failure rate at the beginning can outweight potential benefits.

To date, studies focused on learning curves for MV repair are scarce. Hence, we hypothesized that examining the learning curve for MV repair would be helpful for understanding the trends of individual performance and learning processes, encouraging surgeons to perform more MV repairs and ultimately benefitting patients. Sequential probability cumulative sum (CUSUM) analysis has previously been used to evaluate surgical performance. The technique provides information on changing the success rate, mortality and morbidity during treatment and can be easily used to generate a learning curve to monitor individual surgeon’s performance. We therefore designed this study to evaluate the learning curves for MV repair of the whole institution as well as of individual surgeons. The purpose was to provide recommendations for education and quality control in this field.

We present the following article in accordance with the STROBE reporting checklist (available at http://dx.doi.org/10.21037/jtd-20-1960).

Methods

Patients

Profiles of all patients with degenerative MR who underwent MV surgery at our institution were analyzed. MV replacement was performed in some cases because of the lack of experience during the early stage of the availability of MV repair and the individual surgeon’s or patient’s wish. In other cases, MV replacement was performed when MV repair failed. In this study, any patient who received MV replacement was considered a failed case.

From January 2003 to December 2016, a total of 2,709 patients underwent MV surgery for degenerative MR. Among them, 2,287 received MV repair and 422 MV replacement. Surgeons who had performed fewer than 10 repair operations were excluded, as were surgeons who had retired or left our institution. Finally, 2,482 operations performed by 14 surgeons were included in the study. The trends of the surgery volumes and repair rates are shown in Figure 1. Patients characteristics are presented in Table 1.

Figure 1 Trends of mitral valve repair and mitral valve replacement as well as repair rate from 2003 to 2016.

Table 1 Patients characteristics
Full table

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the institutional review board of Zhongshan Hospital, Fudan University (No. 20141221). The informed consent was exempted by the board.

End points

The primary end point was MV replacement under any condition. The secondary end point was a composite end point consisting of major adverse events, including valve-related reoperation during hospital stay, stroke, new renal failure requiring dialysis, prolonged mechanical ventilation or tracheotomy, low-cardiac-output syndrome, septicemia, mediastinitis and/or in-hospital death. Patients who suffered one or more of these adverse events were counted as having one secondary end point. Institutional, individual and average learning curves for primary and secondary end points were analyzed. Additionally, individual and average learning curves for cardiopulmonary bypass (CPB) time and aortic cross-clamping time were analyzed.

Surgical technique

Full or partial median sternotomy was performed in 2,082 patients. Minimally invasive surgery via right thoracotomy (385 patients) and a robot-assisted approach (15 patients) have been performed at our institution since 2009. In patients who underwent MV surgery via the right thoracotomy approach, femoral arterial and venous cannulation were performed for CPB. Aortic clamping was accomplished with transthoracic Chitwood clamps. The left atrium was entered through the interatrial groove, and MV repair was performed using standardized techniques, as in sternotomy cases.

Intraoperatively, transesophageal echocardiography (TEE) was routinely used to evaluate the MV lesion, and proper repair techniques were developed based on a combination of surgical inspection and TEE results. The primary leaflet repair included quadrangular or triangular resection of the prolapsed leaflet, artificial chordal reconstruction using expanded polytetrafluoroethylene sutures, and commissural closure. An annuloplasty band or ring was implanted in most patients. Routine TEE was performed after removing the patient from CPB. Intraoperative revision was necessary in the setting of remaining moderate/severe MR. MV replacement was performed in patients with remaining moderate/severe MR or when systolic anterior motion was still present.

Patients with moderate/severe tricuspid regurgitation or a dilated tricuspid annulus (>40 mm) generally underwent additional tricuspid valvuloplasty. The Maze procedure was performed electively in patients with atrial fibrillation. Other concomitant procedures were performed accordingly, including coronary artery bypass grafting, aortic valve replacement, and aortic surgery. Details of the surgical techniques are summarized in Table 2.

Table 2 Operative and details
Full table

Statistical analysis

Continuous variables are expressed as means ± standard deviation, and categorical variables are expressed as counts and percentages.

Patients with primary or secondary end points were considered failed cases. The learning curves for the primary and secondary end point were each calculated for the whole institution and individual surgeons.

We used CUSUM failure analysis to generate learning curves. The use of CUSUM failure has been described in previous studies (8-10). Herein, CUSUM was defined as Sn=∑(Xi − X0), where Xi =1 for a failed case (MV replacement or major adverse events) and Xi =0 for a successful case. The target value X0 was set at 0.15 for repair rate and 0.05 for major adverse events based on our experience and that of previous studies. The CUSUM curve, together with 80% and 95% control boundaries were then calculated and drawn according to formulas presented in the Appendix 1. Additionally, boundary lines were calculated to mark the borders for statistical significance.

The crossing of an upper boundary indicated that the surgeon’s failure rate was at an unacceptably high level (P1=0.25 for first primary point). The upper 80% boundary was mildly alarming, whereas crossing the upper 95% line indicated that the surgeon’s performance required investigation. Crossing the lower 95% boundary indicated that the surgeon’s failure rate was below the accepted failure rate.

Results

Among the 2,482 patients, 377 failure cases with the first end point and 99 failure cases with the secondary end point were identified. In the 99 failed cases with the secondary end point, new renal failure requiring dialysis occurred in 20 patients, prolonged ventilation requiring tracheotomy in 35 patients, valve-related reoperation during hospital stay in 4 patients, stroke in 14 patients, low-cardiac-output syndrome in 15 patients, septicemia in 23 patients, mediastinitis in 4 patients, and in-hospital death in 31 patients.

Institutional learning curve

The learning curves of the entire institution for primary end point and secondary end point are shown in Figure 2. During the early years [2003–2011], the number of MV repair failure cases was high. However, beginning in 2011, the learning curve showed a continuously downward trend. As for major adverse events, the learning curve presented a continuously downward trend from the beginning.

Figure 2 Institution learning curve. From 2003 to 2011, the learning curve of repair rate continue to rise up with fluctuation with in small range. Turning point came up in 2011, afterwards the performance of whole institution persist to improve. The learning curve for major adverse events continue to decrease from 2003 to 2016.

The annual trend of major adverse events is shown in Figure 3 and repair rates in Figure 1. Since 2011, the replacement rate stabilized at approximately 10%. The composite rate of adverse events remained low over 14 years. We also found that the percentage of cases with concomitant procedures rose up continuously from 2003 to 2016 (shown in Figure 3).

Figure 3 Over 13 years, overall adverse events rates and incidence of major adverse events including stroke, prolonged ventilation, new renal replacement treatment and in-hospital mortality were extremely low. Rate of concomitant procedures rised up over 13 years.

Learning curves of repair rate for individual surgeons

The repair rate learning curves for the 14 individual surgeons varied (Figure 4). Three learning curve patterns were identified: (I) surgeons with outstanding performances (n=5); (II) those with normal performances (n=7); and (III) those who underperformed (n=2). Representative learning curves of each of three patterns are shown in Figure 5.

Figure 4 Learning curves of repair rate for 14 individual surgeons. The two learning curve marked by black arrow were drew on the right axis.

Figure 5 Typical learning curves of different surgeons. Surgeons with normal performance: surgeon C, D, E and F. Surgeons with outstanding performance: surgeon A, G, H and I. Surgeon with underperformance: surgeon B.

According to the individual learning curves of the normal-performance surgeons (C, D, E, and F), the number of operations needed to overcome the learning curve ranged from 100 to 200. Surgeon C began MV repair early at our institution. With the lack of institutional experience during the early phase, surgeon C’s learning curve (normal performance) fluctuated. Yet, his failure rate was within an acceptable range. Surgeons E and F performed their first MV repairs in 2006. The number of operations they needed to perform to pass the learning curve was approximately 100 to 125. We observed that when they completed the learning process after 110 operations, their learning curves began to rise again. After reexamining the surgical data, we found that these surgeons had both begun to repair complex MV lesions using a minimally invasive approach at that time.

Surgeon A had had some experiences with MV repair before 2003, and this surgeon had the greatest operation volume at our institution (>1,100 cases). After the initial learning process, their failure rate remained extremely low. In recent years, the experience accumulated by surgeon A was essential to the learning process of inexperienced surgeons in low-volume setting.

Surgeon B also began MV repair early at our institution. The learning curve of surgeon B (who underperformed) continues because of his decision to perform MV replacement in complex cases.

Surgeon I, who began the learning process in 2009, was an example of outstanding performance. The number of operations required to complete the learning process was <50. In recent years, surgeons G and H, who began the learning process in 2014, also achieved outstanding performances within 50 operations.

The repair rate was further assessed according to the individual’s surgical volume and the time at which the surgeon began the learning process (shown in Figure 6). Although surgeon A, who performed >1,100 cases, achieved a repair rate of 95%, having more cases was not necessarily associated with a higher repair rate for all surgeons. However, surgeons who began their learning process after 2011, tended to have a faster learning process and achieve a higher repair rate.

Figure 6 Repair rate of individual surgeons. (A) A larger individual surgery volume was not necessarily associated with higher repair rate. (B) Surgeons who began their learning process after 2011 generally achieved higher repair rate.

Average learning curves for major adverse events

The rate of major adverse events (prolonged ventilation requiring tracheotomy, acute kidney injury requiring renal replacement therapy, in-hospital death, stroke, etc.) remained low (<2%) and did not show a clear trend (shown in Figure 3).

Average and individual learning curves for CPB and aorta cross-clamping time

The average learning curves for the cardiopulmonary time and aortic cross-clamping time are shown in Figure 7. As experience grew, CPB time and aortic cross-clamping time decreased. After 200 cases, the differences were statistically significant (P<0.001). Individual learning curves for operative time were further analyzed. The individual learning curve for CPB and clamping time of five typical surgeons are presented in Figure 7. No clear individual learning curve for CPB or clamping time was observed.

Figure 7 Cardiopulmonary time and aorta cross clamp time decreased as the accumulation of experiences, and after 200 cases, significant decrease was observed (P<0.001). Scatter diagram of CPB and aorta cross clamp time for typical individual surgeon did not show obvious individual learning curve.

Discussion

MV repair has been the primary choice of treatment for degenerative MR. According to the American Heart Association/American College of Cardiology guideline for valvular heart disease, a more aggressive approach of early surgical intervention is recommended for primary MR as it can improve late outcomes (11,12). However, this early surgical strategy, should be restricted to experienced centers in which MV repair rates are higher than 95%. Although MV repair is a repeatable operation and many repair techniques have been shown to be effective and safe in previous studies (1,13-15), the overall repair rate is still <80%, according to the Society of Thoracic Surgeons database (5,16).

Although we have benefited from the advanced experience accumulated by large-scale centers in Western, developed countries achieving excellent outcomes of MV repair for degenerative MR, the overall repair rate in China is still low. According to previous studies, an increased mitral repair rate is associated with reduced mortality (17). Therefore, to improve the outcomes for patients with degenerative mitral disease, educating both surgeons and patients about MV repair is of critical importance.

This study included one of the largest retrospective Chinese population cohorts from a single high-volume center. Our institution is one of the largest centers for adult cardiac surgery in China as well as one of the pioneers in MV repair for degenerative MR. Although MV repair had been performed at our institution since before 2003, the number of cases was too small to obtain meaningful results. Additionally, surgeon A was the only one in this group who had performed MV repair before 2003, although rarely. More surgeons began to learn and perform this operation beginning in 2003, so the cohort presented in this study was initiated during that year. During that time, based on CUSUM failure analysis, we observed a significant difference between learning curves for repair rate and major adverse events, and a wide variation among individual learning curves for repair rate. Hence, the results of this large sample could provide important information for formulating a program to educate surgeons about MV repair as well as to improve and maintain the performance of inexperienced surgeons in this operation. As for the repair rate learning curve of the whole institution, we observed a long (8-year) period of increasing experience with this procedure. During this time, the failure rate continued to increase until a turning point in 2011. At that point, we found that the MV replacement rate decreased from 20% to 10% and remained at 10% for the following years. Yet for major adverse events, we observed that the institutional learning curve decreased continuously. The learning curve of major adverse events was not congruent with the repair rate learning curve. This may be explained by the fact that, although the overall adverse event rate changed over time, it remained low. Furthermore, the average learning curve for major adverse events also demonstrated that during the beginning phase of the learning process, the average major adverse events rate was controlled at a low level. This again demonstrated that MV repair is safe, even when it was initially implemented at our institution. With the help of a “heart team” including a cardiac surgeon, intensive care physician, cardiologist, and nurses, MV repair was safely performed by inexperienced surgeons in an experienced high-volume center. The next pressing question is how to increase the use of MV repair; this is the most challenging and necessary part of the learning process of the MV repair.

In a previous study, Chikwe et al. showed that inexperienced surgeons at institutions with experienced surgeons with high repair rates improve their repair rates (7). Thus, the repaire rate learning process for individual surgeons is strongly affected by institutional experience. In the current study, we also found that, surgeons who performed their first operation during the experience accumulation period prior to 2011 gained their experience more slowly than those who began their learning process more recently. During the early period, inappropriate techniques may have been used in certain cases, and a substantial number of operations were needed for surgeons to improve their technique. As standard techniques were gradually adopted at our institution, young surgeons could easily grasp them and avoid unnecessary MV replacements. Close and, standardized supervision from experienced surgeons can further prevent beginning from making severe and unnecessary mistakes. Hence, the number of operations required to finish the repair rate learning process has decreased in recent years.

Individual repair rate learning curves may be flexible. They are closely tied to the entire institution’s experience as well as the individual surgeon’s experience and treatment choices. For example, the repair rate learning curve for surgeon D became viable again after he started using minimally invasive MV repair for complex lesions. This trend suggests that even experienced surgeons may need to go through another learning process when new techniques are introduced. Therefore, young surgeons should have a certain amount of related experience before performing a new technique. For example, before performing minimally invasive MV repair, young surgeons should accumulate substantial experience with MV replacement using a minimally invasive approach. The willingness of the surgeon to change his or her operative preference strongly affects the individual’s learning curve. We found that the learning curve of surgeon B continued to rise from the beginning. This surgeon, though experienced, was already senior when MV repair was commonly practiced at our institution. Therefore, surgeon B tended to choose MV replacement, with which he was familiar, for complex cases.

Regarding CPB and aortic cross-clamping time, we observed a significant decrease in the average operative time after performing 200 MV repairs. However, the individual learning process for operative time did not exhibit an obvious learning curve. In a previous study on learning minimally invasive MV surgery, Holzhey and colleagues did not find an obvious learning curve for operative time (18). At our institution, only three surgeons had performed more than 200 MV repairs since 2003, therefore the average CPB and clamping time after 200 cases only reflected the experience of these three surgeons. Hence, more data is needed before drawing conclusions regarding a learning curve for operative time. Nevertheless, we believe that CPB and aortic cross-clamping time generally do not strongly affect patients’ outcomes if they are not too long. In the current study, the average aortic cross-clamping time during the initial learning process was restricted to approximately 60 minutes. Thus, inexperienced surgeons should choose less complex cases during their initial learning process to, maintain the safe operative times.

Limitations

The study has some limitations. This retrospective study has the inherent weakness of retrospective analysis. Besides, there was wide variation among operation volume for each surgeon in this study, and several surgeons had just began their learning process. Additionally, the study was based on the experience of a single center, so the conclusions might not be applicable to other centers. The learning process for surgeons in low-volume institutions might be longer. However, abundant online and offline learning opportunities could provide these surgeons valuable resources to shorten the learning curve. Therefore, future follow-up might be needed to further assess the learning curve for MV repair.

Conclusions

A repair rate learning curve for MV repair procedures exists for both institutions and individual surgeons. The number of cases required to overcome the repair rate learning curve is substantial (50–200 in this study), with marked variation among individual surgeons. At experienced high-volume centers, MV repair is safe even during the initial learning process. The information we gathered in this study may facilitate future training protocols and maintain MV repair performance, especially at institutions where MV repair is being introduced.

Acknowledgments

We thank the language editing service provided by AJE.

Funding: This work was supported by the Talent Programme Foundation for the Excellent Backbone of Zhongshan Hospital [2019ZSGG07].

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at http://dx.doi.org/10.21037/jtd-20-1960

Data Sharing Statement: Available at http://dx.doi.org/10.21037/jtd-20-1960

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/jtd-20-1960). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the institutional review board of Zhongshan Hospital, Fudan University (No. 20141221). The informed consent was exempted by the board.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Braunberger E, Deloche A, Berrebi A, et al. Very long-term results (more than 20 years) of valve repair with carpentier's techniques in nonrheumatic mitral valve insufficiency. Circulation 2001;104:I8-11. [Crossref] [PubMed]
2. Castillo JG, Anyanwu AC, Fuster V, et al. A near 100% repair rate for mitral valve prolapse is achievable in a reference center: implications for future guidelines. J Thorac Cardiovasc Surg 2012;144:308-12. [Crossref] [PubMed]
3. David TE, Armstrong S, McCrindle BW, et al. Late outcomes of mitral valve repair for mitral regurgitation due to degenerative disease. Circulation 2013;127:1485-92. [Crossref] [PubMed]
4. Goldstone AB, Cohen JE, Howard JL, et al. A "Repair-All" Strategy for Degenerative Mitral Valve Disease Safely Minimizes Unnecessary Replacement. Ann Thorac Surg 2015;99:1983-90. [Crossref] [PubMed]
5. Badhwar V, Rankin JS, He X, et al. The Society of Thoracic Surgeons Mitral Repair/Replacement Composite Score: A Report of The Society of Thoracic Surgeons Quality Measurement Task Force. Ann Thorac Surg 2016;101:2265-71. [Crossref] [PubMed]
6. Suri RM, Clavel MA, Schaff HV, et al. Effect of Recurrent Mitral Regurgitation Following Degenerative Mitral Valve Repair: Long-Term Analysis of Competing Outcomes. J Am Coll Cardiol 2016;67:488-98. [Crossref] [PubMed]
7. Chikwe J, Toyoda N, Anyanwu AC, et al. Relation of Mitral Valve Surgery Volume to Repair Rate, Durability, and Survival. J Am Coll Cardiol 2017. [Epub ahead of print]. [Crossref] [PubMed]
8. Novick RJ, Stitt LW. The learning curve of an academic cardiac surgeon: use of the CUSUM method. J Card Surg 1999;14:312-20. [Crossref] [PubMed]
9. Novick RJ, Fox SA, Stitt LW, et al. Cumulative sum failure analysis of a policy change from on-pump to off-pump coronary artery bypass grafting. Ann Thorac Surg 2001;72:S1016-21. [Crossref] [PubMed]
10. Novick RJ, Fox SA, Stitt LW, et al. Assessing the learning curve in off-pump coronary artery surgery via CUSUM failure analysis. Ann Thorac Surg 2002;73:S358-62. [Crossref] [PubMed]
11. Nishimura RA, Otto CM, Bonow RO, et al. 2014 AHA/ACC guideline for the management of patients with valvular heart disease: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 2014;63:e57-185. [Crossref] [PubMed]
12. Nishimura RA, Otto CM, Bonow RO, et al. 2017 AHA/ACC Focused Update of the 2014 AHA/ACC Guideline for the Management of Patients With Valvular Heart Disease: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation 2017;135:e1159-95. [Crossref] [PubMed]
13. Salvador L, Mirone S, Bianchini R, et al. A 20-year experience with mitral valve repair with artificial chordae in 608 patients. J Thorac Cardiovasc Surg 2008;135:1280-7. [Crossref] [PubMed]
14. De Bonis M, Lapenna E, Lorusso R, et al. Very long-term results (up to 17 years) with the double-orifice mitral valve repair combined with ring annuloplasty for degenerative mitral regurgitation. J Thorac Cardiovasc Surg 2012;144:1019-24. [Crossref] [PubMed]
15. Lawrie GM, Earle EA, Earle N. Intermediate-term results of a nonresectional dynamic repair technique in 662 patients with mitral valve prolapse and mitral regurgitation. J Thorac Cardiovasc Surg 2011;141:368-76. [Crossref] [PubMed]
16. Gammie JS, Sheng S, Griffith BP, et al. Trends in mitral valve surgery in the United States: results from the Society of Thoracic Surgeons Adult Cardiac Surgery Database. Ann Thorac Surg 2009;87:1431-7. [Crossref] [PubMed]
17. Vassileva CM, McNeely C, Spertus J, et al. Hospital volume, mitral repair rates, and mortality in mitral valve surgery in the elderly: an analysis of US hospitals treating Medicare fee-for-service patients. J Thorac Cardiovasc Surg 2015;149:762-8. [Crossref] [PubMed]
18. Holzhey DM, Seeburger J, Misfeld M, et al. Learning minimally invasive mitral valve surgery: a cumulative sum sequential probability analysis of 3895 operations from a single high-volume center. Circulation 2013;128:483-91. [Crossref] [PubMed]