Insights into artificial intelligence and our intelligence—on the frontier of lung cancer screening

Background

Artificial intelligence (AI) was once a theoretical construct dreamt up by scholars and poets alike. Now we are faced with a reality in which machine learning is accelerating exponentially and the application of it is limited only by human ingenuity. Much like the invention of the motor car, the aeroplane or the internet, AI has the potential to reshape every industry and every life on earth. However, as we all know too well, the motorcar, the aeroplane and the internet did not come without undue harm. The adaptation of AI into medicine will be upon us as clinicians before we know it. The power and pitfalls of AI in medicine will undoubtedly become more apparent with time and until such time we are set to ponder the potential.

Coinciding with the introduction of AI, we are approaching the introduction lung cancer screening programs across the globe. Lung cancer remains the most common cancer and the most lethal cancer across the globe (1). Early iterations of lung cancer screening programs, using low dose computed tomography (CT) scans have demonstrated an overall decrease in all-cause mortality of 7% and a 20% decrease in lung cancer related mortality in screened populations (1). Traditionally, most lung cancers are diagnosed in late stages (III–IV), while screening programs are likely to shift the stage of diagnosis, with up to 85% of cancers being stage I in screening trials (2). Critical to the success of these screening programs is the ability of the system to identify and deal with these new diagnoses. Contemporary literature, prior to the introduction of screening even in developed countries such as Australia and New Zealand, demonstrate issues coping with the timely treatment of lung cancers (3). Delays to intervention or diagnosis of as little as 6 weeks are associated with pathological upstaging of tumours, increasing tumour recurrence and reduced survival (4). In Australia, approximately 15,000 lung cancers are diagnosed per year, a number set to increase with the introduction of screening and the pathological stage of which will decrease, so the need for timely interpretation of this increasing number of scans is set to rise dramatically. With the expected drastic changes these screening programs are set to instil there are already calls for changes in the contemporary practice regarding the management and diagnosis of lung cancer within our current multidisciplinary team (MDT) structure (5). With the reliance of CT scanning and high number of scans expected, there will undoubtedly be a role for AI in the re-structuring of this MDT.

Unsurprisingly, with the temporal relationship between the rise of AI in Medicine and the introduction of lung cancer screening we are starting to see the first generation of interactions between the two. The pros, cons and clinicians caught between lung cancer screening and AI are considered hereafter.

Automating medicine

The automation of any industry is always met with resistance by those that the automation serves to replace, and medicine is no exception. Replacement is a sentiment that already exists amongst clinicians, a survey performed in Portugal demonstrated that 52% of doctors believe that AI will replace some medical specialties in the future, while 88.2% disagreed that AI could replace clinicians completely. Despite the rhetoric of replacement, 76.5% believe that AI will revolutionize medicine and 73.3% believe it will improve it (6). The advanced computing power that AI provides in the scope of an ever increasingly digitised medical system has the potential to increase system efficiency dramatically and decrease the workload of doctors and other health care staff. AI will be able to aid in the interpretation of scans, streamline the delivery of basic cares, improve recording of procedures and monitoring of medications, the applications of AI are virtually endless. But are we comfortable leaving medicine to the machine? What are the parameters that need to be met to hand over the controls and what are the consequences?

Current use of CT scans in cancer screening

Thoracic CTs have long been the gold standard for lung cancer detection and will now form the basis of lung cancer screening. Capacity for an AI to rapidly compare a multitude of scans, assess for volumetric doubling and identifying suspicious patterns for a given lung nodule is profound and in time the rate and accuracy at which it performs this assessment will surpass that of the humble radiologist. Factor in the advent of lung cancer screening with CT scans that we are seeing across the globe, and the need for rapid assessment of a high volume of CT scans is paramount to the success of screening programs. Current estimates approximate 300 million CTs are performed yearly with this number increasing at a rate of 4% per year (7). Further to this, AI holds the capacity to analyse more simultaneous datapoints than a human can and will likely identify disease patterns otherwise not yet perceived by our current understanding (7). We have also demonstrated that AI is as good as, if not better, in detecting lung cancer on thoracic CTs compared to their human counterparts. Moreover, meta-analysis on the use of AI in lung cancer CT scan interpretation have shown that combined AI-Human interpretation had a sensitivity of 0.87 [95% confidence interval (CI): 0.82–0.90], a specificity of 0.87 (95% CI: 0.82–0.91), with a missed diagnosis rate of 13% and misdiagnosis rate of 12%, showing considerable promise to aid the diagnosis of lung cancer and strong potential to support clinicians in more efficient detection of cancer (8,9).

Sybil, a validated deep learning model, is a prime example of revolutionary AI in the lung cancer screening realm (9). Sybil is an AI model that has been developed by a collaboration of Massachusetts Institute of Technology (MIT) and Massachusetts General Hospital in Boston, trained on the National Lung Screening Trial (NLST) to predict individual patient risk out to 6 years based on CT image data interpretation only, without needing the addition of demographic data. It has been tested against two other major databases with promising results. The model achieved comparable results to the known outcome (diagnosed lung cancer) with an area under the curve for lung cancer prediction at 1 year at 0.9 (95% CI: 0.88–0.95) on NLST database, and 0.86 (95% CI: 0.82–0.90) and 0.94 (95% CI: 0.91–1.00) on other tested databases. The major limitations cited by the team include the fact that the model was trained on a largely homogenous ethnic group (Caucasian) and it is difficult to predict at this stage the ability to extrapolate this data into the more heterogenous real-world population. The team also agree that the integration of this tool into lung cancer screening will require “careful and transparent development” including critical, regular review (9). But then this ultimately leads to the question that as these models grow in the future independently, who will be skilled enough to provide these critical reviews and oversight?

So, from a service provision and innovation perspective, AI makes sense. With the contemporary practice of medicine being evidence based, transparent and trustworthy, the willingness for clinicians to adopt AI into practice as well as safety concerns need to be considered.

Human and machine learning: the similarity

Pattern based learning; currently in all areas of medicine, basic knowledge is taught in university, but the real practice and art of medicine comes from pattern-based learning (10). There is no substitute for experience and exposure volume to enhance pattern recognition. A concept well established in surgical literature and practice, is the idea that high volume leads to enhanced performance and better patient outcomes (11). A theme certainly recognised with respect to education and learning within radiology during the coronavirus disease 2019 (COVID-19) pandemic, where reduction in case volume was associated with lower attribution of learning and diagnostic accuracy (12). Whether it is high-performance surgery or image interpretation, the ability of a clinician to make an accurate diagnosis and perform an intervention is proportional to the volume of cases they have seen and the experience they draw on (12). AI is no different. The more scans it reads, the more efficient it is poised to become.

Pitfalls of handing over control to AI: transparency

Unlike other application for AI, in healthcare AI cannot develop inside a Blackbox, for ethical and clinical reasons. Transparency will be the key to trust and acceptance of these models to allow their integration into clinical practice and transparency in this sense means interpretability. By nature, and nurture, clinicians need to understand the inner workings of the AI model to assess its strengths and weaknesses to integrate its use into clinical practice in a safe way, to ensure continued beneficence and non-maleficence. A survey conducted by the American Association of Medicine demonstrated that 80% of clinicians surveyed had concerns about the transparency of AI algorithms as a barrier to its utility (13). An inability to understand the AI model poses a significant barrier to clinicians accepting its use in practice (6).

Pitfalls of handing over control to AI: evidence-based practice and quality control

Transparency aside, these models need to be evidence based. To ethically integrate these models into practice they need to be trialled, taught and studied with the same degree of scrutiny that clinicians apply to pharmacology and new technology. To do so they need to be validated as reliable and they need to be continually audited to ensure they stay as such. Given the concerns with trust, transparency and the development of an evidence base, human comparison will likely be the mainstay of validation for AI models and their integration into practice. Herein lies a problem with integration of AI, colleges of radiology around the world are quoting numbers of approximately 500 thoracic CTs per annum to keep up with minimum requirements for the safe recognition of lung nodules and cancer (5,14). If we allow AI to interpret out scans, how will the physicians of the future develop these wells of knowledge and exposure necessary to validate the AI model when that very model deprives them of the volume the need? When all screening studies are directed to AI for assessment, how will we train the doctors? And do we train them in isolation? How do we maintain peer-review practice when we no longer have human peers? But who will validate these results after we deskill all our radiologists as they no longer have the volume exposure be incidental or through intent. It is a matter of equipoise, allowing clinicians the adequate exposure to develop deep resources of pattern recognition and allowing the same for AI. Such examples are already emerging in breast cancer and mammography for screening with double-reading, one human and one AI (15). Co-existence not competition.

Recognising the limitations of AI learning algorithms in the context of the contemporary technology is also pivotal to the quality control of the integration of AI into medicine. Medicine is often described as equal parts science and art. The Sybil research group identified the limitations of their model that hinder the widespread adaptation of it (9). The research group raised concerns about the ability to extrapolate the algorithm outside of the demographic group reflective of the database it was built from, as it interprets scans without clinical context. Herein lies the need for radiologist. Until such time as an algorithm can interpret CT imaging and clinical context across simultaneously (and this time likely come) there will be a persistent need for clinician oversight of the system, through radiologists.

Until screening ability of AI surpasses the current accuracy of radiologists or the ideal balance between man and machine is unclear, it is unlikely AI will be incorporated into mass scale screening initiatives, despite possible improvements seen in overall workflow and monitoring (15).

Trust

A unifying concept that challenges the integration of AI in to contemporary practice; trust. We have discussed concerns that need to be addressed to build clinician’s trust in AI, including transparency, validation, and evidence-based practice. Perhaps more importantly, one must consider how our patients feel about having an algorithm for a doctor. We know, there are a multitude of factors that influence patients trust in AI-integration in medicine, from age, nationality, income, education to the individual’s combination of medical problems (14). Interestingly evidence to date has demonstrated patient trust in AI in medicine is inversely proportional to the risk of the disease process. Patients had a high degree of trust in AI integration in low-risk conditions compared to higher risk diseases (16). Patients interacting with AI in early studies have illustrated concerns with the ability to communicate with AI and fear the privacy of their information in these models (6,14,16). The path forward to a future where clinician’s and patients alike trust in AI’s seems tumultuous.

Accountability

At some point, AI interpretation will surpass our ability, it will identify disease patterns beyond our perception born out of its limitless computing potential and AI will likely surpass us long before we realise it has. Will we ever be able to let go and just trust AI? Especially in the knowledge that in health care, these decisions can often be life or death. A question that leads to our closing concern. Nothing on earth is perfect, whether its human or AI and at some point, the model will make a mistake. What do we do with this mistake? Who takes the blame and ultimate responsibility for this mistake? No-one holds an algorithm accountable. A sentiment patient in early AI interaction trials agree with (14). Yes, some of medicine can be automated, but the human experience of medicine and our role as clinicians in understanding this information, validating it, and communicating to patients as a fellow human with compassion will never be replaced.

In terms of legislation, the current governing bodies around the world are racing to generate legislation to keep up with the onslaught of AI applications being seen in healthcare. This rapid growth in AI applications in all facets of healthcare has spurred agencies such as the Therapeutic Goods Administration in Australia and the Food and Drug Administration in the USA, to develop guidelines and policies to manage this growth area and ensure its safe application.

Conclusions

AI is coming. It is coming to all facets of life. Its potential in healthcare is astronomical but comes with unique issues that need to be addressed before its widespread and exuberant introduction. It is the ultimate Catch 22, in order to build trust in AI models, they will need to be validated, at the end of the day by humans. At present, human’s and AI models are scoring the same in most cases when it comes to diagnostic screening, this is in the setting of humans still processing the sheer volume of CTs availed at present. But what happens when workflow, accuracy and cost effectiveness lead us to rely heavily, if not solely, on uninterpretable AI models? Then the humans we call on to validate the AI model, will have been deskilled by the very same model.

We talk of trust. How do we get patients and the general population to trust these AI models? We will validate the models by getting highly trained and skilled radiologists to review the results and we must ensure we allow our radiologist adequate exposure to do so.

It will ultimately need to be a balance, even a synergistic relationship between doctor and computer. It is unlikely that without transparency and understanding that AI, even with its extreme computing power, will be accepted over the humble clinical physician. AI may rise and may replace us in many ways, but there will always be a need for a doctor, because at the end of the day, who will take the responsibility?

Acknowledgments

Funding: None.

Footnote

Peer Review File: Available at https://jtd.amegroups.com/article/view/10.21037/jtd-24-1077/prf

Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at https://jtd.amegroups.com/article/view/10.21037/jtd-24-1077/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Ridge CA, McErlean AM, Ginsberg MS. Epidemiology of lung cancer. Semin Intervent Radiol 2013;30:93-8. [Crossref] [PubMed]
2. Wood DE, Eapen GA, Ettinger DS, et al. Lung Cancer Screening NCCN Guidelines® NCCN Lung Cancer Screening Panel Members. Journal of the National Comprehensive Cancer Network 2012;10:
3. Kirk F, Crathern K, Chang S, et al. The influence of the COVID-19 pandemic on lung cancer surgery in Queensland. ANZ J Surg 2023;93:1536-42. [Crossref] [PubMed]
4. Fligor SC, Tsikis ST, Wang S, et al. Time to surgery in thoracic cancers and prioritization during COVID-19: a systematic review. J Thorac Dis 2020;12:6640-54. [Crossref] [PubMed]
5. Kirk F, He C, Stroebel A. Lung cancer screening: The pulmonologist, the thoracic surgeon and working together in a team. Respirology 2023;28:515-7. [Crossref] [PubMed]
6. Pedro AR, Dias MB, Laranjo L, et al. Artificial intelligence in medicine: A comprehensive survey of medical doctor's perspectives in Portugal. PLoS One 2023;18:e0290613. [Crossref] [PubMed]
7. Borg M, Hilberg O, Andersen MB, et al. Increased use of computed tomography in Denmark: stage shift toward early stage lung cancer through incidental findings. Acta Oncol 2022;61:1256-62. [Crossref] [PubMed]
8. Liu M, Wu J, Wang N, et al. The value of artificial intelligence in the diagnosis of lung cancer: A systematic review and meta-analysis. PLoS One 2023;18:e0273445. [Crossref] [PubMed]
9. Mikhael PG, Wohlwend J, Yala A, et al. Sybil: A Validated Deep Learning Model to Predict Future Lung Cancer Risk From a Single Low-Dose Chest Computed Tomography. J Clin Oncol 2023;41:2191-200. [Crossref] [PubMed]
10. Loveday T, Wiggins M, Festa M, et al. Pattern Recognition as an Indicator of Diagnostic Expertise. In: Carmona PL, Sánchez JS, Fred ALN. editors. Pattern Recognition - Applications and Methods. Berlin: Springer Berlin Heidelberg; 2013:1-11. Available online: https://link.springer.com/10.1007/978-3-642-36530-0_1
11. Maruthappu M, Gilbert BJ, El-Harasis MA, et al. The influence of volume and experience on individual surgical performance: a systematic review. Ann Surg 2015;261:642-7. [Crossref] [PubMed]
12. Gabr AM, Li N, Schenning RC, et al. Diagnostic and Interventional Radiology Case Volume and Education in the Age of Pandemics: Impact Analysis and Potential Future Directions. Acad Radiol 2020;27:1481-8. [Crossref] [PubMed]
13. American Medical Association. November 2023 AMA Augmented Intelligence Research Physician sentiments around the use of AI in health care: motivations, opportunities, risks, and use cases. 1 Background and Objectives. 2023 [cited 2024 Jul 25]. Available online: https://www.ama-assn.org/system/files/physician-ai-sentiment-report.pdf
14. Esmaeilzadeh P, Mirzaei T, Dharanikota S. Patients' Perceptions Toward Human-Artificial Intelligence Interaction in Health Care: Experimental Study. J Med Internet Res 2021;23:e25856. [Crossref] [PubMed]
15. Marinovich ML, Wylie E, Lotter W, et al. Artificial intelligence (AI) for breast cancer screening: BreastScreen population-based cohort study of cancer detection. EBioMedicine 2023;90:104498. [Crossref] [PubMed]
16. Xu J. Overtrust of Robots in High-Risk Scenarios 2018 [cited 2024 Sep 9]. Available online: https://www.aies-conference.com/2018/contents/papers/student/1st_AIES-DC_paper_8.pdf