Retrospective analysis of the sensitivity of reporting of thoracic computed tomography for pleural malignancy: an Australian multi-centre study

Introduction

Background

Malignant pleural effusions (MPEs) are common and increasing in incidence, with a current global incidence of approximately 70 per 100,000 (1). They affect up to 15% of all patients with cancer and can arise from either a primary cancer of the pleural space, such as malignant pleural mesothelioma (MPM), or more commonly, via metastatic spread from a primary malignancy, such as lung and breast cancer (2). The presence of cancer in this space heralds the development of incurable disease and is associated with a poor prognosis and often debilitating breathlessness (3).

Rationale and knowledge gap

Contrast-enhanced computed tomography (CE-CT) is considered the current gold-standard imaging modality for the pleura (3). It is a key component of contemporary diagnostic algorithms, and the 2023 British Thoracic Society (BTS) guidelines recommend a contrast CT chest in all patients with a suspected pleural malignancy (4). However, its utility in this algorithm has not been formally evaluated or validated. Consequently, the role of CT in the diagnostic assessment of MPE is identified as a priority area for future research by the BTS pleural guideline development group (4).

In 1990 Leung et al. established that the CT features suggestive of malignant disease of the pleura were; nodular pleural thickening, mediastinal pleural thickening, parietal pleural thickening >1 cm, and circumferential pleural thickening (5). Using one or more of these criteria CT evaluation gave an overall specificity of 82% and sensitivity of 72% for MPE. The first large scale assessments of CT chest in the assessment of MPE were when Hallifax et al. and Tsim et al. assessed CT reports with pre-specified terminology similar to those described in Table 1 (6,7). Despite using more permissive criteria to define a report indicative of MPE, they both found a reduced degree of specificity and sensitivity than first described by Leung et al. (5). Both studies had higher proportions of MPM then described in global population data which limits the generalisability of this data. These results have led to recommendations that the absence of features of MPE on CT should not discourage invasive diagnostic sampling (8).

A recent systematic review by Reuter et al. was unable to include CT sensitivity for MPE in a planned meta-analysis due to an inadequate amount of available data (9). A sensitive non-invasive test to rule out MPE would drastically change the current unilateral pleural effusion diagnostic pathway. At present contrast-enhanced CT chest is used as an adjunct alongside pleural analysis but it appears to lack sufficient sensitivity to avoid subsequent pleural sampling and potential thoracoscopy when negative or equivocal for features of malignancy.

Objective

Given the current paucity of data for CT use in MPE worldwide, we aimed to assess the current use of CE-CT in a large Australian health network and examine the sensitivity of CT reporting for MPE in a population closer resembling that found in clinical practice worldwide. We anticipated a lower rate of MPM within presenting MPE patients in our population compared to prior published data which we hypothesised would impact the diagnostic performance of CE-CT in this context. We present this article in accordance with the STROBE reporting checklist (available at https://jtd.amegroups.com/article/view/10.21037/jtd-24-1797/rc).

Methods

Study population

The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. This project was approved by the Hunter New England Local Health District Department of Research Ethics and recognised as a negligible risk research activity (Au202204-08). Informed consent was not required as the data were anonymised and will only be published in aggregate. All participating institutions were informed and agreed the study. Diagnosis-related group (DRG) codes were used to identify patients who had undergone pleural intervention or had been diagnosed with malignant pleural disease at two tertiary centres (John Hunter Hospital, Calvary Mater Hospital) in Newcastle, Australia, in the 24 months to 31/12/2020. The catchment area of these centres includes a population of 962,390 in a region of 131,785 square kilometres (10). Codes included were diagnosis of “secondary malignant neoplasm of pleura”, “mesothelioma of pleura”, “malignant pleural effusion” and procedure codes for “insertion of intercostal catheter for drainage”, “diagnostic thoracocentesis” and “therapeutic thoracocentesis”. These criteria were chosen to enhance the inclusion of all-cause unilateral pleural effusions and minimise the chance of missing confirmed MPE within our population.

A total of 1,025 unique medical records were identified. In addition, local procedural databases were assessed for inclusion which yielded a further 97 unique medical records. These 1,122 cases were then reviewed by the authors and all confirmed cases of pleural malignancy that first presented within the 24 months to 31/12/2020 were included. We included only patients with both a gold standard histocytological diagnosis of pleural malignancy and those whom a CT scan report prior to the diagnostic sampling procedure was available. Patients were required to have had available CT imaging within 3 months of the first pleural intervention. There were 22 cases identified that could not be included due to the absence of a preceding CT scan. Cases with a prior diagnosis of pleural malignancy and those that had pleural intervention for investigation/management prior to CT were excluded. The inclusion and exclusion criteria are included in Table 2. This process is summarised in Figure 1.

Figure 1 Study flow diagram. CT, computed tomography.

Data analysis

CT reports were assessed by two independent reviewers from two disciplines (respiratory and radiology) based on pre-specified agreed criteria drawn from existing research (shown in Table 1) (6,7). Any disagreement between reviewers in the independent assessments was resolved by discussion to reach consensus. Other data collected included population demographics, CT contrast phase, underlying histopathological diagnosis and findings of Leung features of MPE in the CT report (5).

Statistical analysis

All histocytologically confirmed cases were considered as the true positive rate of pleural malignancy in our population. We compared CT reports with the (later obtained) histopathological diagnosis to determine the sensitivity of CT reporting suggestive of MPE in the evaluation of pleural effusion. We then assessed for differences in sensitivity based on CT contrast phase, and the effect of using only pleural specific criteria as defined by Leung et al. (5). Confidence intervals (CIs) were calculated using the Wilson Score Interval, a widely accepted formula for calculation of binomial CIs (11).

Results

Population demographics

A total of 101 patients were included. Of those with a final histocytological diagnosis of pleural malignancy, 22 (18%) were excluded due to lack of CT within 3 months of diagnostic biopsy/fluid test or prior to significant pleural intervention. Minimum follow up was 24 months.

Of included cases, 52% (52/101) cases were male. The median age was 72 years (38–89 years). Of these cases, 64% (64/101) had a significant smoking history, only 14% (14/101) had significant asbestos exposure reported, and 52% (52/101) had a current diagnosis of a non-pleural malignancy. Most cases (87%; 87/101) had a unilateral effusion at the time of presentation whilst the remaining 13% (14/101) had bilateral effusions.

CT protocol and histocytological diagnoses

Patients underwent CT scanning using a variety of machines at a number of public and private radiology practices. The machine types used included Toshiba Aquilion PRIME, Siemens SOMATOM Force, SOMATOM Definition AS+. There were 50 unique reporting radiologists. About 67% (68/101) of included cases were late arterial phase CT scans, 25% (25/101) were CT pulmonary angiogram (CTPA), and there were 6 non-contrast CT scans and 2 delayed venous phase CT scans.

Data for histocytological diagnosis of the pleural effusions are shown in Table 3. The most common cause was non-small cell lung cancer (NSCLC) making up 42% (43/101) with the second-most common being MPM with 16% (16/101). Most scans (74%; 74/101) had a co-existing pulmonary nodule and/or lymphadenopathy on CT.

Sensitivity of CT reporting for pleural malignancy

Seventy-one patients had reports indicating MPE on the basis of the pre-hoc defined criteria (Table 2), providing an overall sensitivity of 70% (95% CI: 61–78%). Arterial phase scans reported MPE in 50/68 patients (sensitivity 73%, 95% CI: 62–83%) and CTPA reports were positive in 15/25 patients (sensitivity 60%, 95% CI: 41–77%) which was not significantly different (χ² P=0.21). Of the two delayed venous phase scans, both reported MPE.

Analysis of the reports for presence of radiological (Leung) criteria demonstrated that when only relying on detection of these specific findings for malignant pleural disease, the sensitivity of reported scans reduced to 42% (95% CI: 32–51%). When applying these specific pleural features to arterial phase contrast CT reports the sensitivity was 49% (95% CI: 37–60%) and CTPA report sensitivity overall was 16% (95% CI: 6–35%) which was significantly inferior (χ² P<0.05). Both delayed venous phase CT scans had reported specific pleural findings consistent with MPE. These results are detailed in Table 4.

Discussion

Key findings

The overall sensitivity of CT reporting for MPE in a population of individuals with a first presentation pleural effusion in our cohort was 70%. This is similar to previous studies from the UK (6,7) (Table 5) despite the predominant use of arterial phase contrast enhancement in our population. Delayed venous phase enhancement for pleural evaluation has been recommended since the BTS statement on malignant mesothelioma in 2007 (12). The main proposed benefit of venous delayed contrast enhancement is improved pleural assessment (13) (demonstrated in Figure 2). When relying only on reporting of specific features of pleural malignancy as established by Leung (5), the overall sensitivity of CT reporting in our cohort fell to 42%. This finding was comparable to the decreased sensitivity of CTPA described by Tsim and colleagues (7).

Figure 2 Comparison of nodular pleural thickening on chest CT with and without delayed venous contrast. CT, computed tomography.

Strengths and limitations

To our knowledge this is the first assessment of CT chest performance in the assessment of MPE in an Australian population and the first outside of Western Europe. Although retrospective, the study comprehensively identified patients with pleural malignancy using a combination of DRG codes and audit of relevant respiratory and cardiothoracic procedural lists. We included only those with histocytopathological confirmation of MPE, representing the gold-standard for diagnosis. We ensured our specified duration of study had greater than 24 months of further clinical documentation available to reduce the number of cases missed due to a delayed histopathological diagnosis.

The centres involved in this study are the only tertiary centres for a catchment population of 962,390. As such, they deliver the majority of cancer care and pleural intervention for this population. Furthermore, the catchment area comprises metropolitan, regional and rural areas. These findings are likely to be generalisable, particularly within the Australian context. The lower rate of MPM in this cohort also provides a greater degree of external validity compared to prior results of the large United Kingdom cohorts which exhibited higher rates of MPM then seen in the majority of the world (7). In the future we expect progressive reduction in rates of MPM on the basis of declining global incidence rates in the context of government strategies for reduction of asbestos exposure (14).

This study is limited by its retrospective nature and the inherent risk of missing cases compared to a study that prospectively enrolled those presenting with new pleural effusion. Our study is limited to confirmed MPE cases, and as such we haven’t captured the non-MPE population which would have allowed calculation of specificity, negative and positive predictive values for CT chest in MPE. We also were unable to include 22 confirmed MPE cases (19%) due to a lack of appropriately timed CT imaging as per our inclusion criteria.

The low rates of delayed venous phase contrast are likely to have reduced the sensitivity and specificity demonstrated in this study (7). However, this does highlight the difficulty of standardising CT protocols in real-world clinical practice, particularly in countries like Australia where a large proportion of image acquisition and reporting is performed outside the public healthcare system. Although the patients in this study were within two tertiary centres, the CT image acquisition and reporting was done in a variety of public and private radiology settings with 50 unique reporting radiologists. It is likely that these practitioners, along with the relevant radiographers, had varied experience and expertise in the imaging of malignant pleural disease. This is at odds with previous studies in the literature which have drawn on a small number of specialist thoracic radiologists at expert centres and is both a limitation and strength of our study design. The diversity of radiologists and referring clinicians in this cohort would contribute to heterogeneity in the clinical information that has been made available to the reporting radiologist which also impacts the reporting and detection of subtle pleural abnormalities on scans included in this study. It is important to note that the overall sensitivity of CT reporting for MPE shown in this study partially reflects the native sensitivity of CT as an imaging modality, but more accurately demonstrates the accuracy of real-life CT reporting with a mixed cohort of radiologists (the majority of whom are not specialist thoracic radiologists).

Comparison with similar research

In 1990 Leung et al. established four specific CT findings which were indicative of MPE in a single centre retrospective study of 74 patients with diffuse pleural abnormality on CT (5). Using one or more of these criteria in CT evaluation gave an overall specificity of 82% and sensitivity of 72% for MPE. None of these signs were reliable in differentiating MPM from pleural metastatic spread.

The first large scale assessment of CT in this context was a retrospective review of patients who underwent thoracoscopy in two tertiary centres in the United Kingdom by Hallifax et al. in 2015 (6). This study included 370 patients with a high rate of MPM, which accounted for 109/202 (54%) of those with MPE. Hallifax and colleagues created a criteria to judge whether a CT report indicated a pleural malignancy based on phrases used by the radiologist rather than the strict inclusion of Leung features (6). This criteria used terms similar to those in Table 1 and did not require mention of specific radiological findings but rather an impression of malignancy as the cause for the pleural effusion. Even with a more permissive criteria, the specificity of CT was 78% and the sensitivity was 68%. However, as this study’s population was limited to those undergoing thoracoscopy at a set of expert centres; it is reasonable to assume that this is a small subset of those undergoing CT for the assessment of undiagnosed pleural effusion. This was an important finding as it was the first large scale study to re-evaluate the sensitivity of CT for MPE and it found a significantly lower sensitivity which reduced the clinical utility of a benign appearing CT chest in the work-up for potential MPE.

This work was followed by a retrospective study of 315 participants of the DIAPHRAGM study cohort in 2017 (7). The DIAPHGRAM study was designed to evaluate the diagnostic performance of fibulin-3 and SOMAscan blood biomarkers for MPM. This trial prospectively recruited first presentation pleural effusions in multiple centres in the UK and Ireland. Its analysis of the diagnostic utility of CT for MPE used a similar criteria to that described by Hallifax et al. (6) and showed a specificity of 80% and a sensitivity of 58%. It also described the significant variation in CT contrast phase used and that the sensitivity of CTPA reporting was approximately 27% compared to 61% in delayed venous phase CT. This emphasised the importance of contrast modality for effective pleural assessment but also showed that in a setting closer to real-world clinical practice CT chest performed even worse in evaluation of MPE.

Explanation of findings

The main factor we propose explaining the preserved sensitivity of CT for MPE in our study despite sub-optimal contrast phase compared to the prior UK studies is the difference in the underlying disease prevalence. Our included population had a substantially lower rate of MPM (16%) compared to 30% and 54% in the Tsim and Hallifax studies respectively (Table 5) (6,7). The higher proportion of secondary pleural malignancy resulted in a high rate of extra-pleural signs suggesting malignancy. 41% (42/101) of the included scans showed pulmonary or extra-thoracic masses and 47% (48/101) had significant lymphadenopathy. This resulted in an increased clinical suspicion for MPE despite the absence of specific pleural signs for malignancy. This is consistent with the significant decline in performance of CT chest reporting in our cohort when solely relying on Leung features to classify our reports. 12/16 (75%) of our MPM cases had a positive CT scan with identified Leung features for pleural malignancy (Table 3). A higher proportion of MPM would likely increase the sensitivity of CT when assessing solely for Leung features. Conversely the increased proportion of secondary pleural malignancy results in increased rates of extra-pleural signs that are not as reliant on the contrast phase of the CT.

It is also important to acknowledge the contribution of information bias which is a factor unrelated to the performance of CT which will impact radiologist reporting for MPE. Clinical histories on imaging request forms and access to old scans or those in an alternative modality will impact on the reporting for MPE independent of the underlying ability of the reporter to discern specific pleural features of malignancy on the scan.

Conclusions

This is, to our knowledge, the first evaluation of CT chest sensitivity in MPE assessment in Australian centres. This study adds to the existing literature demonstrating that CT has inadequate sensitivity to rule out malignant pleural disease in those presenting with a pleural effusion and reinforces the need for early diagnostic pleural intervention when pleural malignancy is suspected. Our data highlights the heterogeneity in real-world radiological practice with infrequent use of venous phase contrast across a health network despite established data supporting its use when evaluating suspected malignant pleural disease. It also provides an assessment of CT performance in the evaluation of MPE in a population with a prevalence of MPM more representative of clinical practice worldwide. The role of CT chest in the diagnostic algorithm for MPE remains unclear. Further research to evaluate the utility of CT in the planning of pleural intervention, the value of sequential CT scans and the optimal role for CT within the pleural diagnostic pathway are urgently needed.

Acknowledgments

An abstract of this study was presented at The Australia & New Zealand Society of Respiratory Science and The Thoracic Society of Australia and New Zealand (ANZSRS/TSANZ) Annual Scientific Meeting for Leaders in Lung Health & Respiratory Science 2023.

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at https://jtd.amegroups.com/article/view/10.21037/jtd-24-1797/rc

Data Sharing Statement: Available at https://jtd.amegroups.com/article/view/10.21037/jtd-24-1797/dss

Peer Review File: Available at https://jtd.amegroups.com/article/view/10.21037/jtd-24-1797/prf

Funding: This work was supported by a PhD scholarship from the Australian Government Research Training Program (RTP) to V.G.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jtd.amegroups.com/article/view/10.21037/jtd-24-1797/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. This project was approved by the Hunter New England Local Health District Department of Research Ethics (under which both the John Hunter Hospital and Calvary Mater Hospital are governed) and recognised as a negligible risk research activity (Au202204-08). Informed consent was not required as the data were anonymised and will only be published in aggregate. All participating institutions were informed and agreed the study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Bodtger U, Hallifax RJ. Epidemiology: why is pleural disease becoming more common? In: Maskell NA, Laursen CB, Lee YCG, editors. Pleural Disease 2020;87:1-12. doi: 10.1183/2312508X.10022819.
2. Bibby AC, Dorn P, Psallidas I, et al. ERS/EACTS statement on the management of malignant pleural effusions. Eur Respir J 2018;52:1800349. [Crossref] [PubMed]
3. Psallidas I, Kalomenidis I, Porcel JM, et al. Malignant pleural effusion: from bench to bedside. Eur Respir Rev 2016;25:189-98. [Crossref] [PubMed]
4. Roberts ME, Rahman NM, Maskell NA, et al. British Thoracic Society Guideline for pleural disease. Thorax 2023;78:s1-s42. [Crossref] [PubMed]
5. Leung AN, Müller NL, Miller RR. CT in differential diagnosis of diffuse pleural disease. AJR Am J Roentgenol 1990;154:487-92. [Crossref] [PubMed]
6. Hallifax RJ, Haris M, Corcoran JP, et al. Role of CT in assessing pleural malignancy prior to thoracoscopy. Thorax 2015;70:192-3. [Crossref] [PubMed]
7. Tsim S, Stobo DB, Alexander L, et al. The diagnostic performance of routinely acquired and reported computed tomography imaging in patients presenting with suspected pleural malignancy. Lung Cancer 2017;103:38-43. [Crossref] [PubMed]
8. Addala DN, Kanellakis NI, Bedawi EO, et al. Malignant pleural effusion: Updates in diagnosis, management and current challenges. Front Oncol 2022;12:1053574. [Crossref] [PubMed]
9. Reuter S, Naur TMH, Clementsen PF, et al. The value of computed tomography in discriminating malignant from non-malignant causes of unresolved unilateral pleural effusions: a systematic review. Eur Clin Respir J 2019;6:1565803. [Crossref] [PubMed]
10. New South Wales Government. Hunter New England Local Health District. NSW Health. Accessed on June 7th, 2023. Available online: https://www.health.nsw.gov.au/lhd/Pages/hnelhd.aspx#
11. Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med 1998;17:857-72. [Crossref] [PubMed]
12. BTS statement on malignant mesothelioma in the UK, 2007. Thorax 2007;62:ii1-ii19. [Crossref] [PubMed]
13. Croft M, Lim W, Lavender N, et al. Optimising CT-chest protocols and the added value of venous-phase contrast timing; Observational case-control. J Med Imaging Radiat Oncol 2022;66:768-75. [Crossref] [PubMed]
14. Huang J, Chan SC, Pang WS, et al. Global Incidence, Risk Factors, and Temporal Trends of Mesothelioma: A Population-Based Study. J Thorac Oncol 2023;18:792-802. [Crossref] [PubMed]