High-profile studies frequently and repetitively present data on the same patients, particularly in immunotherapy studies

Introduction

Abstracts presented at conferences, as either poster or oral presentations, are an important avenue for the dissemination of study results, offering an expedient method of disseminating new data (1). However, they are also subjected to a differing level of scrutiny and quality as compared to papers published in peer-reviewed journals (2). Compared to full-length publications, abstracts sometimes present incomplete data that do not include long enough follow-up, do not evaluate all endpoints, has small sample sizes and other methodological flaws that would not be accepted in a more rigorous format (1). While there are legitimate reasons for wanting to publish this preliminary data, conference abstracts often report findings that are not followed up by full-length papers in future publications (3). Without following up an abstract with a full-length paper, there is no peer review conducted on the final product of the studies, allowing the conference abstract to serve as the final word (1). The variable quality of presented abstracts, particularly when reporting clinical trial data, has been commented upon previously in the literature (4). A consensus statement was published by the CONSORT group (1). This statement attempted to standardize the information published in abstracts concerning clinical trials (1) (Table 1). A recent study examined the adherence to the CONSORT consensus statement in the field of critical care medicine (5). This work found improvement in some areas but still reported substandard adherence to CONSORT statement (5). Another investigation in the field of HIV/AIDS research reported a small improvement in some areas but a similar inconsistency in complying with the CONSORT recommendations (6). Both studies validated the continuing importance of consistent reporting of clinical trials in abstracts. Although the CONSORT work is most well-known, there was concern regarding abstract structure several decades ago (7).

Table 1 Consort requirements for reporting of randomized clinical trials (3)
Full table

Other work has looked into the relationship between abstract presentation and subsequent paper publication. One study examining the 5-year publication rate of podium presentations at SICOT, an orthopedic surgery conference, found that only 31.3% of oral presentations were published as full-text papers (3). A similar study in the field of craniofacial surgery found a publication rate of 35% (8). Specifically, in the field of oncology, it has been shown that around a quarter of clinical trials presented as abstracts at American Society of Clinical Oncology (ASCO) meetings were not published within 5 years of presentation (9). Further work has also demonstrated that differences in data and conclusions are common between meeting presentations and ultimate publication (10). Interestingly, presentations made in the late-breaking trial sections of major meetings were more likely to reach full publication (10). It is unclear the effect of this discrepancy in abstract and full-length publication. In particular, the way in which data presented as abstracts differs from that published in full-length papers is difficult to evaluate.

Due to the intense interest in new treatments and rapidly advancing nature of the field, oncology investigators frequently report the results of major clinical trials through the presentation of abstracts at major conferences. ASCO 2016, a major meeting in clinical oncology, published 2,463 abstracts, of which 1,995 concerned clinical trials (11). Further, 37% of attendees indicated that their primary interest at ASCO was clinical trials (11). The aforementioned interest by conference attendees coupled with patient centered oncology websites demonstrate how closely these abstracts are watched by clinicians and patients alike (11). Traditionally, study results have been presented as abstracts at major scientific meetings at the conclusion of the analysis. Recently, presentations of studies in progress and updates to previously presented data have been allowed at major meetings. Often, this includes an update to a previous study in which the majority of the patients’ outcomes had previously been presented. This is often compounded by the nature of group collaboration where multiple senior investigators all present data from the same trial themselves at different major meetings. While this allows for appropriate recognition of academic contribution, it creates more presentations from the same underlying patient data. The frequency and implications of a single study being presented multiple times, particularly in high profile oral presentations, have not been fully evaluated.

In the present study, we investigated the degree to which clinical trials were presented multiple times within the fields of lung cancer and immunotherapy. We attempt to quantify the degree of repetition. We also distinguish between abstracts presented as oral presentations and those presented as posters with specific attention paid to the oral presentations, as these presentations often are viewed by more conference participants and generate additional press coverage. Because of the intense interest in immunotherapy, we specifically evaluated lung cancer immunotherapy trials.

Methods

Study selection

The field of lung cancer was surveyed across an approximate 1-year period by searching abstracts presented at international conferences from the three major societies devoted to lung cancer research. American Society of Clinical Oncology (ASCO 2015), World Conference on Lung Cancer (World Lung 2015), European Society of Medical Oncology (ESMO 2015) and ASCO 2016 were chosen as a representative sample. Abstracts were selected for inclusion in a two-step process. In the first step, abstracts were selected that pertained to lung cancer or immunotherapy by searching for the keywords: non-small cell lung cancer (NSCLC), small cell lung cancer (SCLC) and immunotherapy. The searches differed slightly based on individual website functionality. ASCO 2015 and 2016 were searched by track, with the included tracks listed in Table 2. The World Lung 2015 database was organized by thematic conference sessions inside large subject area tracts. Each session inside the subject area tracts in Table 3 was opened and abstract titles referencing “Non-Small Cell Lung Cancer”, “Small Cell Lung Cancer” or “Immunotherapy” were included. The ESMO 2015 library was not organized by category. Therefore, the database was searched by entering the queries “NSCLC”, “SCLC” and “Immunotherapy” and analyzing each abstract listed by hand for inclusion. In this step, abstracts included based on reference to NSCLC or SCLC were further evaluated to determine whether they pertained to immunotherapy. This was defined as having one or more of the drugs in an intervention arm of the study that were immune checkpoint inhibitors, immune system activating interleukins or anti-tumor vaccines.

Table 2 ASCO meeting tracts analyzed
Full table

Table 3 World Lung categories analyzed
Full table

In the second step, abstracts identified in the first step were selected for further analysis if the abstract:

• Reported on clinical trials;
• Presented clinical outcome data;
• Had an identifiable NCT number.

Immunotherapy abstracts not pertaining to lung cancer treatment were excluded in the second step. This exclusion was most relevant at ASCO and ESMO as these are not lung specific conferences and contained immunotherapy abstracts concerning other organ malignancies. Abstracts selected in the second step were noted in the database to be reporting on clinical trials. Additional information collected included conference presented, whether the abstract was presented orally or as a poster, authorship, NCT number and a brief study description noting the study design and endpoints.

Database analysis

Abstracts with an NCT number in common were compared by hand to determine if they presented outcome data from the same clinical trial based on endpoints and study design. Those that were from the same trial were linked. Finally, it was noted whether the original and subsequent abstract presentations were oral presentations or posters. The studies generating oral presentations were then searched to see if they resulted in full-length publication by July 2017. This was ascertained by searching the NCT number in the PubMed and Google Scholar search engines and recording if there was a full-length paper published that satisfied the same three criteria applied during abstract selection.

Results

A total of 851 abstracts were identified across the four conferences that pertained to NSCLC, SCLC or immunotherapy by searching thematic tracks and content areas or using keyword queries. In total, 357 of the 851 abstracts were found to fulfill the three criteria: presenting outcome data, pertaining to a clinical trial, and being associated with an NCT number. Of the 357 abstracts presenting clinical trial data, 110 (31%) were presented multiple times with a mean of 2.75 presentations and a range of 2 to 7 presentations over these four conferences. Notably, 107 of the 357 (30%) clinical trial abstracts pertained to immunotherapy. Of the 113 oral presentations, 75 (66%) presented data from clinical trials, either as posters or oral presentations. Further, 35 of the 113 (31%) oral presentations presented data from clinical trials that had generated other oral presentations. Nine of 35 (26%) oral presentations from repeated clinical trials pertained to immunotherapy.

There were 16 clinical trials that led to oral presentations at least twice during the study period (Table 4). Fourteen trials lead to two presentations, one lead to three and one lead to four (Table 4). Of these, 4 (25%) reported outcome data on an immunotherapy drug as previously defined. Multiple issues led to the duplicate presentations. Six of the trials solely presented updated patient data from the same trial, all with increased number of patients. They had a median patient increase of 61%. In addition, biomarker analyses of previously presented patient data were included in new presentations for six trials. Reanalysis by a different study variable generated further oral presentations as well in 5 of the 16 studies. For example, one trial was presented with overall survival as the main study variable and was then presented again with disease related symptoms as the study variable.

Table 4 Trials generating multiple oral presentations
Full table

Of the 16 trials generating multiple oral abstracts, 11 (69%) lead to a full-length publication by July 2017. Of those that had yet to lead to a paper, all but one was still collecting data based on the NIH clinical trial database (clinicaltrials.gov). All four of the immunotherapy trials were still active but had each resulted in a full-length publication. The majority of those published as a paper included larger patient cohorts than the abstracts during the study period.

Discussion

The present study shows that important clinical trials have generated multiple presentations at major meetings. This phenomenon is particularly prevalent among clinical trials leading to oral presentations, a proxy for the most influential clinical trials. This is particularly relevant in the rapidly advancing field of immunotherapy which was responsible for a significant portion of the trials leading to multiple presentations.

Research in consumer psychology leaves considerable debate regarding the optimal number of exposures to convey a message; however, even the most minimalist research conclusions in the field show that multiple exposures strengthen the response to the message (12). A large meta-analysis examining the state of messaging frequency research was performed in 2015 (13). This analysis included 37 individual studies measuring recall and branding (13). It concluded that maximum effect occurred at ten exposures to the stimulus. Conference presentations are clearly different from paid advertising. However, it is not hard to imagine an implicit bias occurring in the evaluation of research presented multiple times, especially given the aforementioned research into exposure frequency.

Physicians have been found to be vulnerable to repeat advertising and branding, most notably with the pharmaceutical industry (14). Studies have shown changes in prescribing practices based on advertising to physicians (15). In addition, funding of studies by pharmaceutical companies has been reported to lead to bias favoring the product made by the sponsoring company (16). Pharmaceutical companies do not control the acceptance or profile given to presentations at major meetings. Reviewers responsible for selecting presentations for a conference are often faced with a choice between selecting a study presenting lower impact data for a first time or a higher impact study simply presenting an update to previously presented data.

A recent study in the field of orthopedic surgery found that 15% of abstracts presented at one of two major orthopedic conferences were duplicated in a 3-year period (17). This study found that the duplications most often relied on changes to the title or authorship (17). Interestingly, a 2016 study in the field of malignant hematology found that 31% of abstracts at two major meetings were repeated (18). They also report that 75% of the repeated abstracts acknowledged industry support, while only 42% of abstracts presented once had industry support (18). While medicine has changed the way it interacts with the pharmaceutical industry to some degree and made attempts to standardize clinical trial reporting, it is important to question other ways in which the profession interacts with new information particularly in light of the stakes in medical decision-making.

The importance of information conveyed by conference abstracts is especially acute in geographic regions with limited access to medical literature (5). If physicians are only able to access conference abstracts in place of full-length papers, they are particularly reliant on the authors to represent all key pieces of information clearly in the abstract (5). This problem has been previously discussed with regards to standardization of conference abstracts (5). This discussion is also relevant in the factual representation of information. One study of pharmacology articles and abstracts found that up to 24.7% of the 243 publications reviewed contained omissions or inaccuracies in prior presentations (19). However, it is worth examining again in the light of clinical trials generating multiple abstracts and the effects of repeat exposure to information. Of note, the full-length publication rate of the 16 clinical trials generating multiple oral presentations (69%) is roughly in line with a Cochrane review finding 63% of biomedical clinical trial abstracts end up being published as full-length papers (20). Although the sample size for immunotherapy trials is low, it is reassuring that in this area of research that has generated so much interest, all of the studies associated with multiple oral abstracts have been published.

The key issue posed by repeated presentation at major conferences is the balance between important new information added to the field and the confounding effect that repeated data sets can have. While there is value in reporting updated data with large time intervals between reporting of the time in between updated presentations is short, it may appear confirmatory to a conference participant when in fact little has changed in the course of the study. Note, we only evaluated presentations over the course of a single year. In addition, statistical analyses designed at the outset of the trial are generally intended to be evaluated at a specific time point, calling into question the validity of continued interpretations of data over time. With the freedom to continue to present only select data across multiple conferences, there is the risk of investigators presenting solely those outcomes that improve over time.

Acknowledgements

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Hopewell S, Clarke M, Moher D, et al. CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS Med 2008;5:e20. [Crossref] [PubMed]
2. Herbison P. The reporting quality of abstracts of randomised controlled trials submitted to the ICS meeting in Heidelberg. Neurourol Urodyn 2005;24:21-4. [Crossref] [PubMed]
3. Al-Hourani K, Al-Aref R, Ley-Greaves R, et al. Five-year publication rate of podium presentations at SICOT Annual Conference: an observational study and new objective proposal of conference power. SICOT J 2017;3:36. [Crossref] [PubMed]
4. Pitkin RM, Branagan MA, Burmeister LF. Pitkin RM1, Branagan MA, Burmeister LF. JAMA 1999;281:1110-1. [Crossref] [PubMed]
5. Kuriyama A, Takahashi N, Nakayama T. Reporting of critical care trial abstracts: a comparison before and after the announcement of CONSORT guideline for abstracts. Trials 2017;18:32. [Crossref] [PubMed]
6. Bigna JJ, Noubiap JJ, Asangbeh SL, et al. Abstracts reporting of HIV/AIDS randomized controlled trials in general medicine and infectious diseases journals: completeness to date and improvement in the quality since CONSORT extension for abstracts. BMC Med Res Methodol 2016;16:138. [Crossref] [PubMed]
7. Froom P, Froom J. Deficiencies in structured medical abstracts. J Clin Epidemiol 1993;46:591-4. [Crossref] [PubMed]
8. Dobson H, Wall S. Publication Rates of Studies Presented at the International Society of Craniofacial Surgery Congress. J Craniofac Surg 2016;27:1943-5. [Crossref] [PubMed]
9. Krzyzanowska MK, Pintilie M, Tannock IF. Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA 2003;290:495-501. [Crossref] [PubMed]
10. Toma M, McAlister FA, Bialy L, et al. Transition from meeting abstract to full-length journal article for randomized controlled trials. JAMA 2006;295:1281-7. [Crossref] [PubMed]
11. American Society of Clinical Oncology. Demographics. ASCO Annual Meeting 2017. Available online: https://am.asco.org/about/demographics
12. Tellis GJ. Effective frequency: one exposure or three factors? J Advert Res 1997.75-80.
13. Schmidt S, Eisend M. Advertising Repetition: A Meta-Analysis on Effective Frequency in Advertising. J Advert 2015;44:415-28. [Crossref]
14. Sigworth SK, Nettleman MD, Cohen GM. Pharmaceutical branding of resident physicians. JAMA 2001;286:1024-5. [Crossref] [PubMed]
15. Orlowski JP, Wateska L. The effects of pharmaceutical firm enticements on physician prescribing patterns. There's no such thing as a free lunch. Chest 1992;102:270-3. [Crossref] [PubMed]
16. Lexchin J, Bero LA, Djulbegovic B, et al. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 2003;326:1167-70. [Crossref] [PubMed]
17. Kraeutler MJ, Carver TJ, McCarty EC. An Analysis of Duplicate Presentations at the 2014 Through 2016 AOSSM and AANA Annual Meetings. Orthop J Sports Med 2017;5:2325967117718531. [Crossref] [PubMed]
18. Ramchandren R, Schiffer CA. Pattern of Duplicate Presentations at National Hematology-Oncology Meetings: Influence of the Pharmaceutical Industry. J Oncol Pract 2016;12:254-9, 252-3.
19. Ward LG, Kendrach MG, Price SO. Accuracy of abstracts for original research articles in pharmacy journals. Ann Pharmacother 2004;38:1173-7. [Crossref] [PubMed]
20. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev 2007.MR000005. [PubMed]