Diagnostic performance of coronary computed tomography angiography versus exercise electrocardiography for coronary artery disease: a systematic review and meta-analysis
Introduction
Coronary artery disease (CAD) is the leading cause of morbidity and mortality in both developed and developing countries (1). Coronary atherosclerosis involves a prolonged asymptomatic developmental phase, with its clinical manifestations often resulting in angina pectoris, acute myocardial infarction or cardiac death. In addition to invasive coronary angiography (ICA), which is the reference standard for assessing anatomical stenosis severity, a variety of non-invasive testing methods have been advocated recently to provide an anatomic and/or functional evaluation of coronary artery. Available methods include exercise/stress electrocardiography (ExECG), single proton emission computed tomography (SPECT), myocardial perfusion imaging (MPI), coronary computed tomography angiography (CCTA) and coronary computed tomography with fractional flow reserve (FFRCT). Despite these facts, assessment of the presence of CAD remains challenging.
Among these diagnostic methods, ExECG is a well-established and inexpensive procedure to evaluate intermediate risk patients with angina pectoris (2). However, ExECG has relatively limited diagnostic performance in patients with silent CAD (3). As a new non-invasive alternative test, CCTA has high diagnostic performance to rule out CAD (4,5). Moreover, CCTA can be used in patients with equivocal stress test or unable to exercise stress test (6). But this method also suffers a number of limitations, such as a progressive loss of sensitivity and specificity as the pretest probability of disease decreases (7).
To date, several studies have compared the effectiveness of CCTA with that of ExECG for the diagnosis of CAD (8-14). But there was controversy about the specificity of two arms (12,13). Additionally, a major limitation of these investigations was their reliance, by necessity, on observational studies due to the limited data in each single study. Therefore, we performed a meta-analysis to compare the diagnostic performance of CCTA and ExECG for CAD based on a larger data, which indicates a more specific comparison about the value of anatomic and functional evaluation in clinical decisions.
Methods
Literature search
To identify relevant articles eligible for the meta-analysis, we searched PubMed and Embase databases up to May 22, 2015 using the following search terms: coronary computed tomography angiography, CCTA, stress ECG, exercise ECG, ExECG, non-invasive coronary angiography CT or exercise testing. To reduce the impact of individual differences in the maximum degree, we only selected the articles compared CCTA with ExECG and limited to articles published in English. We additionally searched the references of all articles retrieved. All relevant articles identified through the search were scanned on the basis of title and abstract. The articles which clearly didn’t meet the inclusion criteria were rejected in the initial screening. The potentially associated articles were read in their entirety to assess their appropriateness for inclusion in the analysis.
Inclusion and exclusion criteria
All studies had to meet the following inclusion criteria: (I) studies that determined the comparison of CCTA and ExECG; (II) patients with the symptoms of stable angina, atypical chest pain or silent ischaemia; (III) sensitivity and specificity results in the diagnosis of CAD were reported; (IV) significant coronary stenosis was defined as at least ≥50% luminal obstruction on ICA; (V) prospective or retrospective studies. The exclusion criteria were study type being a review, case report, commentary or outcome without raw data.
Data extraction
Two authors extracted the data from each article independently to increase objectivity using a standardized data extraction form including study characteristics (study design, total patient number, mean age ± SD, male/female ratio, pretest probability, CT-imaging technique, lumen diameter reduction, treatment) and true positive (TP), false positive (FP), true negative (TN) and false negative (FN) results. During data extraction, we performed the quality assessment of included studies using an updated quality assessment tool ‘‘Quality Assessment of Diagnostic Accuracy Studies-2’’ (QUADAS-2) guidelines. The QUADAS-2 tool consists of four key domains that discuss patient selection, index test, reference standard and flow of patients. We selected seven items to assess risk of bias and applicability which was shown in Table S1. The answer to each item was “yes”, “no” or “unclear” (“yes” indicates low risk of bias, “no” indicates high risk of bias, “unclear” indicates unclear risk of bias). If a study was judged as “low” on all domains relating to bias or applicability, then it was appropriate to have an overall judgment of “low risk of bias” or “low concern regarding applicability” for that study. If a study was judged “high” or “unclear” in one or more domains, then it might be judged as “at risk of bias” or “concerns regarding applicability” (15).
Full table
Data analysis
We first did the spearman correlation analysis and the ROC plane plot to confirm whether there was heterogeneity. If ROC plane appeared “shoulder-arm shape” or spearman correlation analysis showed P<0.05, there was heterogeneity caused by threshold effect in the statistics. In addition, the likelihood ratio (I2) index and Cochran Q test were used to quantify heterogeneity of the included studies (I2 >25% or PQ<0.05 indicated heterogeneity among studies). If there was heterogeneity among studies, the random-effect model was used for the meta-analysis; otherwise, the fixed-effect model was chosen.
The pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR) and 95% confidence interval (CI), summary receiver-operating characteristic curve (SROC), area under curve (AUC) were calculated in this meta-analysis. If the sensitivity and specificity were more close to 100%, the results would have more diagnostic value. Furthermore, positive predictive value (PPV) and negative predictive value (NPV) were calculated, which could provide additional evidence. These effort sizes were used to compare the diagnostic accuracy of CCTA and ExECG for CAD.
In addition to main (overall) analysis, which evaluated all available data, subgroup analyses were also performed by risk of bias of included studies and characteristic of disease (stable or unstable angina). We also performed the meta-regression for age, gender and diabetes mellitus. The potential presence of publication bias was evaluated using Deeks’ funnel plots (16).
Statistical analysis was performed with Stata statistical software (Version 12.0, Stata Corp LP, College Station, TX, USA), Meta-Disc software (Version 1.4, Madrid, Spain) and Review Manager (RevMan) (Version 5.0, Copenhagen, Nordic Cochrane Centre, The Cochrane Collaboration, 2010). When the P value was less than 0.05, the difference was considered statistically significant.
Results
Literature evaluation and study characteristics
The article search results were shown in Figure 1. The number of search records was 235. After initial evaluation, 226 search records were removed and the remaining nine articles were further evaluated by reading the full text. Two studies did not provide the raw data. Thus, only seven articles were appropriate for the meta-analysis (All articles could be found in PubMed or Embase) (8-14). One study was retrospective (10) and six studies were prospective (8,9,11-14). Table 1 showed the general characteristics of the seven studies. In total, 1,242 patients undergoing CCTA and 1,122 patients undergoing ExECG were included. Among them, data from 804 patients undergoing CCTA and 672 patients undergoing ExECG were eligible for the analysis after excluding the equivocal results, which could affect the system evaluation. The pretest probability was varied from low to high and we evaluated the risk factors of all studies. But the significant stenosis standard of all studies was different (50% or 70%), indicating that there was partially potential verification bias. All patients received a sublingual dose of nitroglycerin and patients in six studies were administered with beta blockers (atenolol, metoprolol) before the CCTA scan. All studies processed the related data of age, gender and diabetes mellitus.
Full table
Five articles were evaluated as low risk of bias, and two articles were deemed unclear risk of bias. Table S1 showed the quality assessment of all included studies based on the QUADAS-2. Absolute incidences of clinical outcomes for each of the studies were reported in Table S2.
Full table
Heterogeneity
The spearman correlation coefficients for CCTA and ExECG were −0.60 (P=0.285) and −0.50 (P=0.391), respectively, which indicated there was no heterogeneity caused by threshold effect in each arm. The results were confirmed by the performance of ROC plane plots, in which no pattern of “shoulder-arm” was observed.
The heterogeneity caused by other factors was assessed by I2 index to choose the appropriate calculation model. In CCTA arm, the I2 of sensitivity, specificity, PLR, NLR and DOR were 9.9% (P=0.353), 93.8% (P=0.000), 94.2% (P=0.000), 0.0% (P=0.803), and 45.4% (P=0.089), respectively. Therefore, the random-effect model was used for calculating pooled specificity, PLR and DOR, and the fixed-effect model was used for calculating pooled sensitivity and NLR. In ExECG arm, the I2 of sensitivity, specificity, PLR, NLR and DOR were 68.7% (P=0.004), 94.4% (P=0.000), 90.4% (P=0.000), 75.3% (P=0.001), and 80.4% (P=0.000), respectively. So, the random-effect model was used for calculating all the effect sizes.
Diagnostic accuracy of CCTA and ExECG for CAD
Figure 2 showed the results of the sensitivity and specificity of CCTA in the diagnosis of CAD using the combined data from the included studies. The pooled sensitivity and specificity of CCTA were 98% (95% CIs: 95–99%) and 84% (95% CIs: 81–87%), respectively. The overall PLR and NLR were 5.62 (95% CIs: 2.49–12.68) and 0.05 (95% CIs: 0.03–0.10), respectively. The pooled DOR of CCTA was 110.24 (95% CIs: 35.07–346.55) (Figure 3). The data showed that the SROC curve of CCTA was positioned near the desirable upper left corner and the Q-value was 0.972; while the AUC area was 0.100±0.005 (Figure 4). We counted the PPV and NPV of CCTA of each study, which was shown in Table S2. Six studies showed higher PPV and NPV in CCTA arm, which was consistent with effect sizes.
The pooled sensitivity and specificity of ExECG were 66% (95% CIs: 59–72%) and 75% (95% CIs: 71–79%), respectively (Figure 2). The overall PLR and NLR were 2.74 (95% CIs: 1.35–5.55) and 0.45 (95% CIs: 0.27–0.76), respectively. The pooled DOR was 6.28 (95% CIs: 2.06–19.13) (Figure 3). Compared with CCTA arm, the SROC curve for ExECG was not positioned near the desirable upper left corner, the Q-value was 0.712 and the AUC area was 0.773±0.064, indicating that the diagnostic accuracy of ExECG was lower than that of CCTA arm (Figure 4). We also calculated the PPV and NPV of ExECG of each study, and the results were no difference to CCTA arm’s (Table S2).
Subgroup analysis
Two studies included patients with stable angina and five studies included patients with unstable angina. Thus we analyzed the diagnostic accuracy of CCTA and ExECG for CAD in these two subgroups. In the stable angina group, the pooled sensitivity and specificity of CCTA were 100% (95% CIs: 90–100%) and 95% (95 CIs: 91–96%); the pooled sensitivity and specificity of ExECG were 77% (95% CIs: 50–93%) and 91% (95 CIs: 87–94%), respectively. In the unstable angina group, the pooled sensitivity and specificity of CCTA were 97% (95% CIs: 95–99%) and 68% (95 CIs: 61–75%); the pooled sensitivity and specificity of ExECG were 64% (95% CIs: 58–71%) and 52% (95 CIs: 45–60%), respectively. These results indicated that CCTA was better than ExECG in both stable and unstable angina subgroups.
We also found the diagnostic performance of CCTA was better than that of ExECG regardless of the different significant stenosis standard (50% or 70%) (Table S3). We did not find the sources of heterogeneity by getting rid of the two studies which deemed unclear risk of bias.
Full table
Additionally, we performed the subgroup analysis of low risk of bias and unclear risk of bias articles, but we did not find any statistically significant results.
Meta-regression
We performed the meta-regression for age, gender and diabetes mellitus (Figure S1). We found that sensitivity of CCTA was inversely related to the age and diabetes, while directly related to the male/female ratio. Similarly, ExECG arm also got the same result. The adjusted R2 of ExECG was 0.94 and the P>F was 0.01. The adjusted R2 of CCTA was 0.64, but the P>F was 0.12. So we further performed CCTA’s meta-regression for age, gender and diabetes mellitus, respectively. The results showed that only the P value of diabetes group was less than 0.05 (Figure S2).
Publication bias
The potential presence of publication bias was explored by Deeks’ funnel plot. The shape of the funnel plot for CCTA and ExECG did not reveal any evidence of obvious asymmetry. The Deeks’ test did not show potential publication bias (P=0.17) (Figure 5).
Discussion
Exercise testing is widely used as the convenient detecting tool in the diagnosis of CAD for patients with angina pectoris. The test is non-invasive, not involving radiation exposure, and is simple to perform. Patients with stable or unstable angina, especially those with major cardiovascular risk factors, prefer to undergo stress testing (17,18). Objectively, it can reveal the functional performance of the coronary artery. CCTA can be used to rule out CAD in anatomic performance, particularly in patients with low to intermediate risk of CAD. In an outpatient population, CCTA has emerged as an accurate and rapid tool for the exclusion of CAD (19,20). Several studies compared the diagnostic performance of ExECG versus CCTA to evaluate the clinical value of functional and anatomic methods (21).
In our study, the sensitivity and specificity of CCTA were significantly higher than those of ExECG, indicating that it was the first choice to use the CCTA to detect CAD. The PLR and NLR of CCTA were also significantly higher than ExECG arm, which was similar to the DOR estimation. AUC area was considered the critical standard in judging diagnostic performance, and there was a difference between the CCTA and ExECG. According to the results of subgroup analysis, the diagnostic accuracy of CCTA was superior to ExECG both in stable and unstable angina subgroups. According to the results of meta-regression, we considered that ExECG has higher sensitivity in low-age groups and in patients without diabetes. Moreover, the sensitivity of ExECG will be increased, along with the increase of male/female ratio. We also found CCTA had higher sensitvity in patients without diabetes.
We also confirmed that the diagnostic performance of CCTA, which was better than ExECG regardless of the different significant stenosis standard (50% or 70%). The results indicated the results of the analysis were not influenced by factors such as the selection of significant stenosis standard. But we did not find any statistically significant results between low risk of bias and unclear risk of bias articles.
To provide additional comprehensive evidence of the conclusion, we also calculated the PPV and NPV (Table S2). It changed with the prevalence of CAD and it was helpful to clinical physicians. But in this meta-analysis, we did not put emphasis on these effect sizes due to it can’t be used as a diagnostic test evaluation index.
A comparative effectiveness report conducted a systematic review of the accuracy of different non-invasive technologies including CCTA and ExECG, for diagnosing CAD in women with symptoms suspicious of CAD. But for CCTA, the number of male patients was substantially higher than that of female patients. Thus our results increased the proportion of women. In some extent, we improved the total female ratio to make up the shortage of the meta-analysis. Another previous meta-analysis found CCTA reduced costs of care and the time to diagnosis in the emergency department, while rates of direct discharge were lower than standard care. However, the meta-analysis had some limitations, such as the absence of long-term follow-up and economic analysis of all studies. Thus, we must view the results objectively (22).
Except for CCTA and ExECG, there are several comparable diagnostic methods. A previous meta-analysis reported the diagnostic accuracy and posttest outcomes of ExECG and SPECT compared with CCTA in patients with suspected stable CAD (23). Patients with and without previously knowing CAD were considered eligible for inclusion in the meta-analysis, which were different from our analysis. Because ExECG was used more frequently than SPECT, we put the emphasis on the comparison between ExECG and CCTA using larger-size populations. Another diagnostic method is FFRCT, which reduces the CCTA alone narrow degree classification of false positive rate and avoids the fractional flow reserve’s invasive defect. FFRCT <0.80 was considered diagnostic of lesion specific ischaemia. A report said that low-density non-calcified plaque (LD-NCP) and FFRCT yielded diagnostic improvement over stenosis assessment with AUCs increasing from 0.71 by stenosis 50% to 0.79 and 0.90 when adding LD-NCP ≥30 mm3 and LD-NCP ≥30 mm3 + FFRCT ≤0.80, respectively (24). However, whether CCTA image can display the actual vascular elasticity still needs to be explored. Not only that, a complete FFRCT analysis usually takes 5 hours, the defect will limit its clinical application (25).
Nevertheless, we can’t ignore the limitations of this meta-analysis: (I) we did not evaluate the cost and length of stay of the two arms due to limited available data. Among the studies included, two studies also included the analysis of cost and length of stay. One study showed the CCTA-based evaluation was less expensive than ExECG, while the other study pointed out the CCTA was more expensive than exercise test; (II) even though the total female radio was improved, but it still did not meet the best proportion (1:1), the low proportion of women in the present meta-analysis might have contributed to the higher specificity of ExECG, because exercise testing is known to have lower diagnostic performance on women than on men (7,26); (III) in ExECG arm, all of the effect sizes were highly heterogeneous, which might affect the pooled effect sizes.
Conclusions
CCTA in the diagnosis for CAD has higher sensitivity and specificity than ExECG evaluation, which may offer a better solution for the clinical problem of the diagnosis for CAD. It is worth mentioning that ExECG does not measure stenosis degree but functional CAD. The meta-analysis highlights the strength of CCTA, thus we should put more emphasis on the diagnostic performance of CCTA and do more comparison between anatomic and functional evaluation.
Acknowledgements
Funding: This study was supported by the National Natural Science Foundation of China (81570401, 81170136, 81300103, 81571934), Taishan Scholar Program of Shandong Province (ts20130911), Specialized Research Fund for the Doctoral Program of Higher Education (20130131110048), Key Technology Research and Development Program of Science and Technology of Shandong Province (2014kjhm0102), Department of Science and Technology of Shandong Province (2014GSF11811), and the Fundamental Research Funds of Shandong University (2014QLKY04).
Footnote
Conflicts of Interest: The authors have no conflicts of interest to declare.
References
- Gibbons RJ, Jones DW, Gardner TJ, et al. The American Heart Association's 2008 Statement of Principles for Healthcare Reform. Circulation 2008;118:2209-18. [Crossref] [PubMed]
- Gibbons RJ, Balady GJ, Bricker JT, et al. ACC/AHA 2002 guideline update for exercise testing: summary article. A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Update the 1997 Exercise Testing Guidelines). J Am Coll Cardiol 2002;40:1531-40. [Crossref] [PubMed]
- Gianrossi R, Detrano R, Mulvihill D, et al. Exercise-induced ST depression in the diagnosis of coronary artery disease. A meta-analysis. Circulation 1989;80:87-98. [Crossref] [PubMed]
- Meijboom WB, Meijs MF, Schuijf JD, et al. Diagnostic accuracy of 64-slice computed tomography coronary angiography: a prospective, multicenter, multivendor study. J Am Coll Cardiol 2008;52:2135-44. [Crossref] [PubMed]
- Abdulla J, Abildstrom SZ, Gotzsche O, et al. 64-multislice detector computed tomography coronary angiography as potential alternative to conventional coronary angiography: a systematic review and meta-analysis. Eur Heart J 2007;28:3042-50. [Crossref] [PubMed]
- Hendel RC, Patel MR, Kramer CM, et al. ACCF/ACR/SCCT/SCMR/ASNC/NASCI/SCAI/SIR 2006 appropriateness criteria for cardiac computed tomography and cardiac magnetic resonance imaging: a report of the American College of Cardiology Foundation Quality Strategic Directions Committee Appropriateness Criteria Working Group, American College of Radiology, Society of Cardiovascular Computed Tomography, Society for Cardiovascular Magnetic Resonance, American Society of Nuclear Cardiology, North American Society for Cardiac Imaging, Society for Cardiovascular Angiography and Interventions, and Society of Interventional Radiology. J Am Coll Cardiol 2006;48:1475-97. [Crossref] [PubMed]
- Dewey M, Dübel HP, Schink T, et al. Head-to-head comparison of multislice computed tomography and exercise electrocardiography for diagnosis of coronary artery disease. Eur Heart J 2007;28:2485-90. [Crossref] [PubMed]
- Cademartiri F, La Grutta L, Palumbo A, et al. Computed tomography coronary angiography vs. stress ECG in patients with stable angina. Radiol Med 2009;114:513-23. [Crossref] [PubMed]
- Hamilton-Craig C, Fifoot A, Hansen M, et al. Diagnostic performance and cost of CT angiography versus stress ECG--a randomized prospective study of suspected acute coronary syndrome chest pain in the emergency department (CT-COMPARE). Int J Cardiol 2014;177:867-73. [Crossref] [PubMed]
- Maffei E, Palumbo A, Martini C, et al. Stress-ECG vs. CT coronary angiography for the diagnosis of coronary artery disease: a "real-world" experience. Radiol Med 2010;115:354-67. [Crossref] [PubMed]
- Mollet NR, Cademartiri F, Van Mieghem C, et al. Adjunctive value of CT coronary angiography in the diagnostic work-up of patients with typical angina pectoris. Eur Heart J 2007;28:1872-8. [Crossref] [PubMed]
- Nagori M, Narain VS, Saran RK, et al. Efficacy of multi-detector coronary computed tomography angiography in comparison with exercise electrocardiogram in the triage of patients of low risk acute chest pain. Indian Heart J 2014;66:435-42. [Crossref] [PubMed]
- Nieman K, Galema T, Weustink A, et al. Computed tomography versus exercise electrocardiography in patients with stable chest complaints: real-world experiences from a fast-track chest pain clinic. Heart 2009;95:1669-75. [Crossref] [PubMed]
- Ovrehus KA, Jensen JK, Mickley HF, et al. Comparison of usefulness of exercise testing versus coronary computed tomographic angiography for evaluation of patients suspected of having coronary artery disease. Am J Cardiol 2010;105:773-9. [Crossref] [PubMed]
- Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529-36. [Crossref] [PubMed]
- Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol 2005;58:882-93. [Crossref] [PubMed]
- Abrams J. Clinical practice. Chronic stable angina. N Engl J Med 2005;352:2524-33. [Crossref] [PubMed]
- Greenland P, Gaziano JM. Clinical practice. Selecting asymptomatic patients for coronary computed tomography or electrocardiographic exercise testing. N Engl J Med 2003;349:465-73. [Crossref] [PubMed]
- Arbab-Zadeh A, Miller JM, Rochitte CE, et al. Diagnostic accuracy of computed tomography coronary angiography according to pre-test probability of coronary artery disease and severity of coronary arterial calcification. The CORE-64 (Coronary Artery Evaluation Using 64-Row Multidetector Computed Tomography Angiography) International Multicenter Study. J Am Coll Cardiol 2012;59:379-87. [Crossref] [PubMed]
- Budoff MJ, Dowe D, Jollis JG, et al. Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicenter ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Invasive Coronary Angiography) trial. J Am Coll Cardiol 2008;52:1724-32. [Crossref] [PubMed]
- Pundziute G, Schuijf JD, van Werkhoven JM, et al. Head-to-head comparison between bicycle exercise testing and coronary calcium score and coronary stenoses on multislice computed tomography. Coron Artery Dis 2009;20:281-7. [Crossref] [PubMed]
- D'Ascenzo F, Cerrato E, Biondi-Zoccai G, et al. Coronary computed tomographic angiography for detection of coronary artery disease in patients presenting to the emergency department with chest pain: a meta-analysis of randomized clinical trials. Eur Heart J Cardiovasc Imaging 2013;14:782-9. [Crossref] [PubMed]
- Nielsen LH, Ortner N, Nørgaard BL, et al. The diagnostic accuracy and outcomes after coronary computed tomography angiography vs. conventional functional testing in patients with stable angina pectoris: a systematic review and meta-analysis. Eur Heart J Cardiovasc Imaging 2014;15:961-71. [Crossref] [PubMed]
- Gaur S, Øvrehus KA, Dey D, et al. Coronary plaque quantification and fractional flow reserve by coronary computed tomography angiography identify ischaemia-causing lesions. Eur Heart J 2016;37:1220-7. [Crossref] [PubMed]
- Pontone G, Andreini D, Baggiano A, et al. Functional relevance of coronary artery disease by cardiac magnetic resonance and cardiac computed tomography: myocardial perfusion and fractional flow reserve. Biomed Res Int 2015;2015:297696.
- Kwok Y, Kim C, Grady D, et al. Meta-analysis of exercise testing to detect coronary artery disease in women. Am J Cardiol 1999;83:660-6. [Crossref] [PubMed]