Estimating the quality of YouTube videos on pulmonary lobectomy

Introduction

Currently, lung cancer is the leading cause of cancer death in the world (1), and, with the development of low-dose helical computed tomography (LDCT), the number of patients with early stage lung cancer is increasing (2). Anatomic lobectomy is the standard treatment strategy for early stage lung cancer (3).

Conventionally, residents and medical students learn about pulmonary lobectomy by textbook or by watching live operations performed by experienced thoracic surgeons. More advanced trainees can perform lobectomy under supervised guidance with direct teaching. Now, with the development of information technology, medical education in the United States (US) is becoming increasingly digital (4). For many students with laptops, tablets, and smart phones, internet access and relevant content has become pervasive and is replacing the use of traditional textbooks (5). YouTube is a valuable and popular source of original videos uploaded by individuals and includes valuable medical resources. Current residents and medical students are the first generation of physicians who have had access to the internet throughout their higher education, and who have had the ability to learn about and review medical techniques and information on YouTube. Although it is unclear how often residents and medical students turn to YouTube video content for medical information, it is likely a convenient and efficient method for medical study.

Because of the lack of transparent peer review, the quality of medical resources on the internet is difficult to determine, and this is no different with YouTube. Hence, it is important to know how to evaluate the quality of the medical sources available on YouTube. There are several studies that have analyzed the educational value of videos related to medical sources including those of surgical therapy, physical examination, and anatomy on YouTube (6-8). One research evaluated the quality of operative videos on YouTube and reported that YouTube can be an important learning tool, but low quality of popular videos should be aware (9). Another study evaluated YouTube videos in the field of otolaryngology and found that only a small number of videos contained information about tonsillectomy and vent tube placement that were helpful and accurate (10). Several studies analyzed the quality of operational instruction and found that there was a positive effect for medical trainees, but the quality was varying degrees (11,12). These papers suggest that YouTube medical materials are potentially beneficial to medical education, but it is still necessary to improve the quality.

There are many studies that have analyzed the educational value of the medical resources of YouTube videos, but the study of the characteristics of physicians who post videos or who are in the videos is insufficient. In this study, we performed a search for a surgical operation (pulmonary lobectomy) on YouTube and sought to analyze characteristics of surgeons and evaluate the quality of YouTube content on surgical operation.

Methods

In this study, the term, “pulmonary lobectomy”, was searched on the YouTube search engine with the filter set to sort videos by view rate (we changed the default YouTube setting). The data of the first 100 videos were gathered. Data recorded included video type (live surgery, academic information, patient interview, other), the number of views, the type of surgery [video-assisted thoracic surgery (VATS)/robotic vs. open], and the name of the primary surgeon. If the majority of the videos contained footage of pulmonary lobectomy surgery being performed on an actual patient, the videos were defined as “live surgery”.

We performed a search on Scopus for each surgeon who was the operating surgeon in a video or who posted a live surgical video to ascertain the h-index and lung-related publication history. Also, a general internet search was performed for each surgeon to confirm the location of their practice and to determine whether they maintained an affiliation with an institution that trains surgical residents or fellows.

Data were described by proportion, means, and ranges. h-index data were categorized as 0–10 and >10. The categorical variables were analyzed with a Chi-square test or Fisher’s exact test and the number of views was analyzed with t-test. Statistical analyses were conducted using SPSS version 16.0.

Results

There were approximately 8,650 videos results for the keyword, “pulmonary lobectomy”. According to the number of views, the top 100 videos were analyzed. In the top 100 videos, there were 79 live surgical videos, 12 academic informational videos, 5 patient interviews, and 4 miscellaneous videos (3 surgical discussions and 1 veterinary surgical video). A primary surgeon was identified in 54 of the 79 (68%) live surgeries, of which 47 of 54 (87%) were VATS/robotic surgeries, with the remaining procedures being traditional open lobectomy.

The 54 live surgical videos with an identifiable primary surgeon originated from 13 different nationalities, most often the US and Spain (Table 1). Thirty-eight unique primary surgeons were identified in 54 live surgery videos. Of these 38 surgeons, only 2 (5%) were oncological surgeons with the remainder (95%) being thoracic surgeons. The thoracic specialist, Diego Gonzalez-Rivas (Spain) had the highest number of live surgery videos, and presented 13 procedures.

Table 1 Origin of videos by country
Full table

For the 38 surgeons, the mean of publication was 8; 5 surgeons had no publications related to the lung, and 33 (87%) had at least 1 publication related to the lung. Thirty-five of the 38 surgeons had an h-index of more than 10 with a range between 0 and 56 and an average of 15. Academically affiliated surgeons accounted for 44 of the 54 (81%) videos.

The surgeons’ characteristics were analyzed, and the outcomes are summarized in Tables 2-4. Significantly, the US surgeons featured in the live surgery videos were more likely to have an h-index >10, and there was no difference in the rate of academic affiliation, lung-related publications, and the proportion of thoracoscopic procedures between surgeons from the US and the rest of the world (Table 2). Those having an h-index >10 were more likely to have published a lung-related publication or have performed VATS/robotic procedures (Table 3).

Table 2 Significant differences in surgeon nationality
Full table

Table 3 Significant differences in surgeon h-index
Full table

Table 4 Significant differences in thoracoscopic surgeons
Full table

There was no statistical difference between the live surgery videos and the remaining videos in the average number of views. The mean number of views for 54 live surgery videos was 23,985 and ranged from 2,470 to 383,765. The most watched video was an academic thoracic surgeon performing an operation in Russia. There was a statistically significant correlation between the number of views and h-index (Table 5).

Table 5 Comparison of the average number of views by surgeon characteristics
Full table

Discussion

YouTube is a popular unregulated video-sharing web site which is free for the public. Since its inception in 2005, YouTube has opened up online videos to the masses (13). It is unsurprising that medical professionals and patients have turned to YouTube to receive and deliver medical information (14). However, as the medical resources of YouTube are user-driven, these videos have not yet been peer-reviewed (15). In the absence of peer-review, characteristics of the resource and response of the audience may be potential quality surrogates for indicators of YouTube video quality (4). A recent study analyzed characteristics of the surgeon who uploaded videos or who were featured in the videos and found that 43% of surgeons had an h-index greater than 3, with an academic affiliation being identified in 32 of the 42 (76%) surgeons. Outcomes suggest trainees and educators alike should critically analyze the quality of video content (4).

In our study, most of the live pulmonary lobectomy videos came from professional thoracic surgeons affiliated with an academic institution including the Annals of Cardiothoracic Surgery. Moreover, many surgeons were experts in the surgical therapy of lung cancer including Dr. Diego Gonzalez-Rivas, who specializes in the single-port VATS procedure. These findings suggest that the videos were posted for the purpose of education and that video quality can be ensured to some degree. However, whether or not the surgeon-source is an expert in pulmonary surgery, which would elevate the educational value of the content, is difficult to directly measure. Meanwhile, most of the live surgery videos had no oral narration, which may be an impediment to the effective instruction of trainee viewers. Although surgeons in this study had adequate academic qualifications like an h-index and academic publications, these facts purely suggest intellectual expertise and may not be related to technical expertise.

With the advantage of it being available to the audience and being numerical in nature, the view rate as means to reflect the response of the audience may be a surrogate for the quality of YouTube videos. In this study, the highest view rates of videos reached up to 380,000, but the quality of the footage and picture was poor. Also, some videos were posted many years ago, and the view rate increased by time. Thus, the view rate may only reflect the popularity but not the quality of a video.

There were some limitations in this study. There was a lack of information concerning the audience, and it is unknown how many members of the audience were medical students and residents. YouTube comments are the direct responses of viewers to videos, but because of the difficulty in measurement, it is unknown how comments relate to the quality of videos. Methods of measurement to evaluate the value of YouTube sources should be improved. Research in this area is in its nascence, and future investigations are warranted.

Conclusions

Medical students and residents searching YouTube for videos on pulmonary lobectomy will encounter a large amount of live surgical videos with acceptable quality. These videos may have great potential to improve surgical education. However, trainees should critically examine the quality of video content, and surgical videos should be improved in quality before they can be used in medical teaching. Both the characteristics of the surgeons and the response of the audience to the videos are imperfect indicators in estimating the value of YouTube content. Developing a systematic program to comprehensively evaluate YouTube content is an essential and worthwhile endeavor.

Acknowledgments

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

References

1. Siegel R, Naishadham D, Jemal A. Cancer statistics, 2013. CA Cancer J Clin 2013;63:11-30. [Crossref] [PubMed]
2. Kovalchik SA, Tammemagi M, Berg CD, et al. Targeting of low-dose CT screening according to the risk of lung-cancer death. N Engl J Med 2013;369:245-54. [Crossref] [PubMed]
3. Ginsberg RJ, Rubinstein LV. Randomized trial of lobectomy versus limited resection for T1 N0 non-small cell lung cancer. Lung Cancer Study Group. Ann Thorac Surg 1995;60:615-22; discussion 622-3. [Crossref] [PubMed]
4. Shires CB, Wilson CD, Sebelik M. Thyroid surgery YouTube videos: estimating quality by surgeon characteristics and view rate. Gland Surg 2019;8:207-11. [Crossref] [PubMed]
5. Gutmann J, Kühbeck F, Berberat PO, et al. Use of learning media by undergraduate medical students in pharmacology: a prospective cohort study. PLoS One 2015;10:e0122624. [Crossref] [PubMed]
6. Sorensen JA, Pusz MD, Brietzke SE. YouTube as an information source for pediatric adenotonsillectomy and ear tube surgery. Int J Pediatr Otorhinolaryngol 2014;78:65-70. [Crossref] [PubMed]
7. Sunderland N, Camm CF, Glover K, et al. A quality assessment of respiratory auscultation material on YouTube. Clin Med (Lond) 2014;14:391-5. [Crossref] [PubMed]
8. Jaffar AA. YouTube: An emerging tool in anatomy education. Anat Sci Educ 2012;5:158-64. [Crossref] [PubMed]
9. Rodriguez HA, Young MT, Jackson HT, et al. Viewer discretion advised: is YouTube a friend or foe in surgical education? Surg Endosc 2018;32:1724-8. [Crossref] [PubMed]
10. Strychowsky JE, Nayan S, Farrokhyar F, et al. YouTube: a good source of information on pediatric tonsillectomy? Int J Pediatr Otorhinolaryngol 2013;77:972-5. [Crossref] [PubMed]
11. Fischer J, Geurts J, Valderrabano V, et al. Educational quality of YouTube videos on knee arthrocentesis. J Clin Rheumatol 2013;19:373-6. [Crossref] [PubMed]
12. Frongia G, Mehrabi A, Fonouni H, et al. YouTube as a Potential Training Resource for Laparoscopic Fundoplication. J Surg Educ. 2016;73:1066-71. [Crossref] [PubMed]
13. Koya KD, Bhatia KR, Hsu JT, et al. YouTube and the expanding role of videos in dermatologic surgery education. Semin Cutan Med Surg 2012;31:163-7. [Crossref] [PubMed]
14. Bezner SK, Hodgman EI, Diesen DL, et al. Pediatric surgery on YouTube™: is the truth out there? J Pediatr Surg 2014;49:586-9. [Crossref] [PubMed]
15. Steinberg PL, Wason S, Stern JM, et al. YouTube as source of prostate cancer information. Urology 2010;75:619-22. [Crossref] [PubMed]