Interobserver size measurement variability in part-solid lung adenocarcinoma using pre-operative computed tomography

Introduction

Lung cancer has been reported to be the most frequent cause of major cancer incidence and mortality worldwide, and adenocarcinoma has been reported to be the most common histologic subtype of lung cancer (1). For lung cancer patients with sub-solid nodules (SSNs), which refers to both pure ground-glass nodules (GGNs) and part-solid GGNs, accurate clinical staging is important for determining the extent of a surgical procedure pre-operatively (2). Pulmonary GGNs are defined as nodules showing hazy increased attenuation that does not obliterate underlying bronchial or vascular structures on thin-section CT images (3), and part-solid GGNs are defined as heterogeneous attenuations with an internal solid component completely obscuring the underlying lung parenchyma (4). The findings of a thin-section CT scan, namely, the presence or absence of a GGO component, have a strong effect on oncologic outcome and could be a compelling clinical descriptor affecting survival (5). The 8^th edition of the tumor-node-metastasis (TNM) staging of lung cancers states that the clinical T-stage should be based on the size of its solid portion within SSNs (2,6). Thus, with the growing importance of the solid component of lung carcinoma, if pre-operative imaging can predict the invasive component of adenocarcinomas with radiologically part-solid GGNs precisely, it is likely to have great clinical value in determining the surgical extent of resection.

In the current TNM classification for lung cancer, measurement of the solid size is used to determine tumor diameter. However, in daily practice, it is sometimes difficult to measure the solid component size; therefore, variability between observers may increase, particularly in part-solid nodules with ground-glass opacity (GGO).

In this study, we aimed to investigate the inter-observer size measurement variability in lung adenocarcinoma.

Methods

This study was approved by institutional ethics board of Shinshu University Hospital, Matsumoto, Japan (No. 3395), and the requirement for patient informed consent was waived.

From January to December 2016, 83 patients with lung cancer who had undergone surgery in Shinshu University Hospital were enrolled. Pre-operative computed tomography (CT) images with a 1.25–2.5 mm section thickness were evaluated. The CT images of 47 patients with part-solid type lung adenocarcinoma excluding pure GGNs and pure solid nodules were assessed. Six physicians, comprising five surgeons and one radiologist, undertook unidimensional measurements of the solid component size and the total size of the tumor using a lung window setting [window level, −550 Hounsfield units (HU); window width, 1,500 HU] with electronic calipers, and the axial view of pre-operative thin-section CT images of full resolution scans in 1.25-mm-thick sections was used. To improve accuracy and agreement in regard to size measurement of the tumors, the use of reconstructed sagittal and coronal planes of the CT images was not permitted. We used two types of CT scanners for the diagnosis of the tumor, namely, Light Speed VCT Vision and Discovery CT 750 HD (GE Healthcare, Freiburg, Germany). Figure 1 shows the measurement of solid size by the author (K Hamanaka) using electronic calipers.

Figure 1 The solid size measurement using electronic calipers.

Inter-observer variability

The difference among the observers between the maximum and minimum diameters measured in each case was calculated as the inter-observer variability. The mean and standard deviation (SD) from the inter-observer variability of 47 images was evaluated. Although SD represents dispersion of measurement variability, when the mean difference for comparing objects is large, it is affected by the mean value. To correct for differences of mean value of the object, the coefficient of variation (CV), calculated as the ratio of the SD to the mean (SD/mean), was also evaluated (7).

Trends in size measurements in relation to observer experience

The specialization and experience of six observers (A-F) were as follows: A, a radiologist with 22 years of experience; B, a thoracic surgeon with 21 years of experience; C, a thoracic surgeon with 15 years of experience; D, a thoracic surgeon with 10 years of experience; E, a thoracic surgeon with seven years of experience, and; F, a surgical resident with three years of experience. The counts of maximum and minimum measurement values for observers A to F were assessed to investigate trends in size measurements according to each observer’s experience.

Morphological characteristics of the tumor

Tumor images (n=47 patients) were subclassified into five groups according to morphological characteristics of the tumor. The number of patients in each group and the representative images showing the largest inter-observer variability (also showing the maximum and minimum measurement values and the pathological invasive size, as supplementary information) in each group are shown in Table 1.

Table 1 The computed tomography image patterns
Full table

Results

The difference between the maximum and minimum diameter measurement of the tumor size by six observers for all patients was plotted as inter-observer variability (Figure 2). The maximum and mean inter-observer variability and SD are also shown in Figure 2. The maximum diameter measurement and SD were larger in the measurement of the solid component size than the total size, respectively, and mean inter-observer variability was larger in solid size than total size measurements of tumor diameter, showing a statistically significant difference (P=0.0398).

Figure 2 Inter-observer variability was plotted to show the difference between the maximum and minimum measurement diameter of the solid size (blue dots) and the total size (red dots) by six observers for all patients. Maximum and mean values of inter-observer variability and SD are also shown. NA, not applicable; SD, standard deviation

The trend in size measurements according to an observer’s experience was assessed through counting the number of times a maximum and minimum measurement size was undertaken among the six observers. The frequency of measurement of the maximum and minimum values is shown in Figure 3, respectively. Observers C and D, who were moderately experienced thoracic surgeons (15 and 10 years of experience, respectively), measured the maximum diameter more frequently than the other observers when measuring the solid and total size of a tumor (Figure 3A). Observer A, an experienced radiologist, and observer B, the most experienced of the thoracic surgeons, measured the minimum diameter more frequently when measuring the solid size than the other observers (Figure 3B).

Figure 3 Figure shows the frequency of measurement counts of maximum (A) and minimum (B) value among six observers (A-F).

We assessed the relationship between the morphological characteristics of the tumors, shown in Table 1, and the inter-observer size measurement variability in terms of the solid component size and the total size of the tumors (Figures 4,5). The SD value for each group of morphologic characteristics concerning inter-observer variability in the measurement of solid size and total size are shown in Figure 4, respectively, and the mean values for each group are also shown below. The SD value for group V was the largest, as was the mean, which was presumed to affect the mean value. Therefore, the CVs of each group were calculated according to solid size and total size (Figure 5). The CV of the solid component size in group I was the largest value compared to the other groups, even with the smallest mean value (Figure 5A); however, there was not a similar tendency in the CV of the total size (Figure 5B).

Figure 4 Figure shows the standard deviation (SD) value for each morphologic characteristic group (Table 1) from inter-observer variability in measurement of solid size (A) and total size (B), with the mean values for each group shown below. SD, standard deviation.

Figure 5 Figure shows the coefficient of variation (CV) value for each of the morphologic characteristic groups (Table 1). Solid (A) and total (B) sizes are shown. CV, coefficient of variation.

Discussion

Adenocarcinoma is the most common histologic subtype of lung cancer (1). Advances in understanding of the pathology and radiology of lung adenocarcinoma have demonstrated important new prognostic features that have led to changes in classification and diagnostic criteria (8). Travis et al. (9) summarized the history of the histologic subclassification of lung adenocarcinoma by the World Health Organization (WHO), and explained the radiological and pathological correlations with bronchioloalveolar carcinoma (BAC). With the advent of CT screening for lung cancer, there has been enormous interest in the pathology and CT features of lung adenocarcinoma, and much has been learned about the strong correlation between histological features and radiological features from CT images. When there is a prominent BAC component, the nodules may appear to be mostly ground-glass on CT. In the 1999 WHO classification, emphasis was placed on the importance of the amount of BAC component. Travis et al. stated concerning a favorable prognosis in patients with adenocarcinoma who had a predominant BAC component in their review article (9).

More recently, new concepts involving adenocarcinoma in situ (AIS) and minimally invasive adenocarcinoma (MIA) have been introduced by the International Association for the Study of Lung Cancer (IASLC), the American Thoracic Society (ATS), and the European Respiratory Society (ERS) (1,10). A small solitary adenocarcinoma with either pure lepidic growth (AIS) or predominant lepidic growth with 5 mm invasion (MIA) is used to define patients who will have almost 100% disease-specific survival when they undergo complete resection. Therefore, if pre-operative imaging can predict the invasive component of adenocarcinomas precisely, this is likely to have great clinical value when determining the extent of surgical resection and a patient’s prognosis.

Improvements in imaging technology mean that detailed tumor measurements are increasingly available for use in clinical decision-making. Tumor measurement is commonly performed using a computer interface that allows radiologists to provide precise measurements. However, multiple factors can contribute to the variability of CT-based tumor measurement (11). With anatomic imaging modalities such as CT, tumor boundaries are particularly difficult to define for patients with atelectasis, pleural effusion, pneumonitis, or other normal tissue displacement (12). Lee et al. reported the correlation between the size of a solid component on thin-section CT and the invasive component on pathology in small lung adenocarcinoma manifesting as SSNs that included both pure GGNs and part-solid GGNs (10). One finding in this study was that three-dimensional measurements of the solid component size on CT showed a tendency to be larger than the invasive component size on pathology, whereas two-dimensional measurements tended to be similar to the size of the invasive component. One possible explanation for differences between CT and pathology measurements was that not only the invasive component but also the collapsed alveolar space, fibrosis, or mucin components may have presented as a solid portion on the CT images (7). Furthermore, variability in tumor segmentation is affected by other factors, including the specialty of the observer, professional experience, and the presence or absence of strict guidelines for measurement (13).

The methodology of size measurement for radiological solid portions and predictions for the presence of pathological invasive lesions have recently been investigated. Regarding the measurement of the solid component, the window setting method has been controversial until recently. The Fleischner Society recommended using the mediastinal window setting in 2013 (14). However, the updated literature in 2017 noted that “small solid or semi-solid components that represent early signs of invasive adeno-carcinoma may be rendered invisible with mediastinal window settings, and the current consensus is that such nodules are best evaluated subjectively using a lung window setting and an edge-enhancing (sharp) filter to judge the presence and extent of solid components” (15). To identify factors that could predict the presence of pathological invasion, Eguchi et al. evaluated 101 pulmonary GGNs. That study showed a combination of tumor size (cutoff, 11 mm) and CT attenuation (cutoff, –680 HU) were predictive factors for pathological invasiveness of pulmonary GGNs, yielding a sensitivity and specificity of 91.7% and 71.4%, respectively, with an area under the curve (AUC) of 0.82, which was higher than individual assessment of tumor size or CT attenuation (AUC, 0.75 and 0.77, respectively) (16).

There are various factors that make measurement of the tumor diameter of lung cancer showing part-solid nodules difficult. The solid portion can represent areas of fibroblastic proliferation, collapsed alveolar space, and mucin, as well as invasive components of the tumor (17). One important aspect of our study was to elucidate the relationship between tumor morphological characteristics and inter-observer size measurement variability. Morphological characteristics of lung cancer on CT images vary widely, and our results showing that inter-observer variability was greater in the type that was expected to be low grade, minimally invasive, or less invasive lung cancer suggests the difficulty in decision-making concerning the necessity of surgical resection and suitability of radical sublobar resection for lung cancer.

An interesting result shown in our study was that two experienced observers (the radiologist and the most experienced thoracic surgeon) measured the minimum tumor diameter more frequently than the other observers. This was possibly because they may have more strictly discriminated the adjacent blood vessels and bronchi from the tumor, and excluded the secondary change such as atelectasis, inflammatory change, and other density. Furthermore, one of the moderately experienced thoracic surgeons recorded maximum diameters more frequently than the other observers, suggesting that a large judgement bias may occur due to the characteristics of the physician measuring the tumor. The less-experienced thoracic surgeon and surgical resident performed relatively average measurements in relation to the other observers.

This study has some limitations. The patient numbers were small, and the observers were mainly thoracic surgeons, with only one radiologist included because the study had been planned by the division of thoracic surgery. However, thoracic surgeons routinely perform tumor diameter measurements of lung cancer using CT images, and also contrast the CT images with post-operative pathological findings, and this may have influenced the reliability of this study. Another limitation is that, since only the axial image was used to measure the tumor diameter in this study, it was impossible to confirm its accuracy from the measured maximum diameter of the tumor in the pathological specimen. However, the purpose of this study was to assess inter-observer variability within the radiological measurement, and we consider that we could accurately assess the variability through unifying the measurement methods.

Conclusions

Inter-observer size measurement variability of the solid component in part-solid lung adenocarcinoma was greater than those of the total size. Moreover, greater variability was shown in group I, indicating the difficulty of size measurement for low-grade malignant potential nodules such as adenocarcinoma in situ, minimally invasive adenocarcinoma, and early-stage invasive adenocarcinoma. It is important to recognize that there is a possibility of unavoidable size measurement variability when making decisions concerning surgical procedures for these diseases.

Acknowledgments

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was approved by institutional ethics board of Shinshu University Hospital, Matsumoto, Japan (No. 3395), and the requirement for patient informed consent was waived.

References

1. Travis WD, Brambilla E, Noguchi M, et al. International association for the study of lung cancer/American Thoracic Society/European Respiratory Society international multidisciplinary classification of lung adenocarcinoma. J Thorac Oncol 2011;6:244-85. [Crossref] [PubMed]
2. Ahn H, Won K, Hee K, et al. Effect of computed tomography window settings and reconstruction plane on 8th edition T-stage classification in patients with lung adenocarcinoma manifesting as a subsolid nodule. Eur J Radiol 2018;98:130-5.
3. Kim H, Park CM, Lee SM, et al. Pure and part-solid pulmonary ground-glass nodules: measurement variability of volume and mass in nodules with a solid portion less than or equal to 5 mm. Radiology 2013;269:585-93. [Crossref] [PubMed]
4. Zhou QJ, Zheng ZC, Zhu YQ, et al. Tumor invasiveness defined by IASLC/ATS/ERS classification of ground-glass nodules can be predicted by quantitative CT parameters. J Thorac Dis 2017;9:1190-200. [Crossref] [PubMed]
5. Hattori A, Matsunaga T, Takamochi K, et al. Neither maximum tumor size nor solid component size is prognostic in part-solid lung cancer: impact of tumor size should be applied exclusively to solid lung cancer. Ann Thorac Surg 2016;102:407-15. [Crossref] [PubMed]
6. Travis WD, Asamura H, Bankier AA, et al. The IASLC lung cancer staging project: proposals for coding T categories for subsolid nodules and assessment of tumor size in part-solid tumors in the forthcoming eighth edition of the TNM classification of lung cancer. J Thorac Oncol 2016;11:1204-23.
7. Brown CE. Coefficient of Variation. In: Applied Multivariate Statistics in Geohydrology and Related Sciences. Berlin, Heidelberg: Springer 1998;155-7.
8. Travis WD, Garg K, Franklin WA, et al. Bronchioloalveolar carcinoma and lung adenocarcinoma: the clinical importance and research relevance of the 2004 World Health Organization pathologic criteria. J Thorac Oncol 2006;1:S13-9. [Crossref] [PubMed]
9. Travis WD, Garg K, Franklin WA, et al. Evolving concepts in the pathology and computed tomography imaging of lung adenocarcinoma and bronchioloalveolar carcinoma. J Clin Oncol 2005;23:3279-87. [Crossref] [PubMed]
10. Lee KH, Goo JM, Park SJ, et al. Correlation between the size of the solid component on thin-section CT and the invasive component on pathology in small lung adenocarcinomas manifesting as ground-glass nodules. J Thorac Oncol 2014;9:74-82. [Crossref] [PubMed]
11. Oxnard GR, Zhao B, Sima CS, et al. Variability of lung tumor measurements on repeat computed tomography scans taken within 15 minutes. J Clin Oncol 2011;29:3114-9. [Crossref] [PubMed]
12. Caldwell CB, Mah K, Ung YC, et al. Observer variation in contouring gross tumor volume in patients with poorly defined non-small-cell lung tumors on CT: the impact of 18FDG-hybrid PET fusion. Int J Radiat Oncol Biol Phys 2001;51:923-31. [Crossref] [PubMed]
13. Altorjai G, Fotina I, Lütgendorf-Caucig C, et al. Cone-beam CT-based delineation of stereotactic lung targets: the influence of image modality and target size on interobserver variability. Int J Radiat Oncol Biol Phys 2012;82:265-72. [Crossref] [PubMed]
14. Naidich DP, Bankier AA, Herold CJ, et al. Recommendations for the management of subsolid pulmonary nodules detected. Radiology 2013;266:304-17. [Crossref] [PubMed]
15. MacMahon H, Naidich DP, Goo JM, et al. Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner Society 2017. Radiology 2017;284:228-43. [Crossref] [PubMed]
16. Eguchi T, Yoshizawa A, Kawakami S, et al. Tumor size and computed tomography attenuation of pulmonary pure ground-glass nodules are useful for predicting pathological invasiveness. PLoS One 2014;9:e97867. [Crossref] [PubMed]
17. Lee SM, Goo JM, Lee KH. CT findings of minimally invasive adenocarcinoma (MIA) of the lung and comparison of solid portion measurement methods at CT in 52 patients. Eur Radiol 2015;25:2318-25. [Crossref] [PubMed]