Biostatistical resources in an academic medical center

Introduction

Biomedical research published in peer-reviewed medical literature is necessary for promoting understanding, addressing novel clinical problems, and educating professionals on new surgical techniques and approaches. In spite of this need, all forms of biomedical research, and surgical research in particular, requires addressing increasingly complex methodological trials and meta-analyses compared to traditional clinical studies conducted 20 years ago (1). Biostatistics is a tool which clinicians and researchers can rely upon to analyze associations and relationships within the data. The exact nature of analysis employed, and resulting data findings and conclusions are governed by several factors, which can be found in Table 1 (2,3).

Table 1 Factors which govern biostatistical approach and analyses (2,3)
Full table

Biostatistics are consequently applied to calculate mathematical relationships and trends in data. Peer-reviewed, published results is the primary way clinicians and researchers advance their practice, which is increasingly needed due to the given rate of change in the standard of care and the growing complexity in medical care such as innovative treatments, surgical approaches, guidelines, and patient involvement (4).

The number of clinical peer-reviewed publications, particularly surgery specific randomized trials, have noticeably increased over the last 12 years (5). Similarly, there has been a meaningful increase in the sophistication and complexity of the statistical analyses of these articles (4). Most articles, including many of the seminal articles published in the 1970s and 1980s relied upon t-tests and descriptive statistics (i.e., means, standard deviations, range, etc.) as the only statistical tests for the publication (6,7). These same statistical methods would likely not pass the current peer-review process. Analytical approaches have grown in both complexity and thoroughness in the past 12 years, with a variety of novel statistical tests, sub-analyses and post-hoc methods that are used to interpret, understand and analyze the increasingly rich and complex data collected with modern clinical trials and non-clinical trials such as meta-analysis and large data analysis. Most meaningfully, while there are more robust analytical methods available for evaluation of biomedical research data, elementary statistical tests, such as the t-test, remain the main or only statistical test in surgical research (8). Another report supports the concern that studies are using, and often misusing, basic parametric statistical tests more frequently even though statistical analytical methods have become more complex and robust in recent years (9). This frequently results in the application of incorrect test(s) being used to evaluate data, and potentially arriving at incorrect conclusions. Incorrect conclusions can impair the acceptance of novel medical treatments and potentially harm patients. Several recent reviews of published peer-reviewed studies determined almost half of current peer-reviewed clinical research articles have as a minimum one statistical error, many of which result in potential misinterpretation of results and incorrect conclusions (10-12).

A review systematically evaluated biostatistical analyses of 100 orthopedic surgery peer-reviewed publications. This review concluded that 17% of the publications’ conclusions were not warranted based on the analytical results, and another analytical method should have been used in 39% of the papers (13). Another review of peer-reviewed surgical specific clinical research found that 71 out of 91 analytical papers (78%) had errors in the usage of essential statistical analyses. The peer-reviewed manuscripts frequently failed to appropriate test for statistical significance, provided probability values (P values) without referencing a statistical test, and inappropriately applied statistical techniques (14).

Traditionally, these issues were addressed solely by the clinical researcher, who had relatively limited biostatistical analytical skills. However, these issues are most easily addressed by clinical researchers utilizing the many biostatistical resources available at most academic medical centers. Well-designed biostatistical resources for clinical research unburden the clinical researcher from having to analyze all of the data they collect, but instead provide an array of resources from which the researcher can use to appropriately and adequately analyze their valuable data. These resources include utilization of clinical and translational sciences (CTS) centers, more widespread and early involvement of biostatisticians in study design as well as the evaluation of data, and continuing education for new statistical methods. This paper aims to describe these resources in more detail.

Biostatisticians

Biostatistics, a discipline where statistical principles are applied to the fields of health and biology (15), has become increasingly critical in health care research. Biostatisticians are increasingly seen as valued members of interdisciplinary teams; they provide necessary insight into the development of research questions, determining necessary statistical power, and data analysis (15,16). They can also provide consultation in many different areas of research, increase the likelihood of obtaining grant funding, and aid in brainstorming sessions (17-19).

The demand for qualified biostatisticians, however, has far outpaced the supply. Funding has been the primary limiting factor in ensuring there are enough biostatisticians to perform even basic analyses (15,19). Biostatisticians require a rigorous education and an infrastructure complete with costly statistical software packages; therefore, funding is key (15,16).

Biostatistical specialization

Biostatisticians are even more valuable as members of CTS teams when they have specialized skills, such as comparative effectiveness research, clinical trials, or predictive modeling (15). This specialization often occurs through trial and error rather than formalized training; it also demonstrates the need for more interdisciplinary teams where biostatisticians can collaborate with others to facilitate specialization (19). Specialization can include multi-center randomized clinical trials, survival analysis, cluster analysis, factor analysis and multi-level models for dealing with clustered or repeated measures data.

There is a relative shortage of biostatisticians for the demand of biomedical research, resulting in generally overworked biostatisticians who work on multiple projects at the same time. This can result in burnout and even errors. Universities and CTS teams can minimize the burden of biostatistical shortages by providing more training to biomedical investigators that lack formal statistics education (16). This can include mentoring and consultation, walk-in clinics, and traditional didactic training by biostatisticians (17,19).

The other benefit to providing training to biomedical investigators is that each team member can see the importance of performing reproducible research that is well-designed and has a robust analysis plan. In doing so, biostatisticians can devote more time and energy to specialization.

Development

Continuing education and professional development is an important tool for both clinical scientists and biostatisticians. Short-courses, seminars and other didactic biostatistical activities may be part pre- or post-conference options for professional society meetings. Additional development opportunities are made available through academic centers through grand rounds or other forms. These development activities are challenging for CTS teams because they take time without having direct compensation and may be either too advanced or too basic for the needs of the clinical scientist. Anecdotally, biostatisticians are often overworked and have limited or no protected time for continuing education and development. Finding this “goldilocks” level of biostatistical developmental resources in an efficient manner is an ongoing challenge for academic medical centers.

Biostatisticians can and should receive additional training when available. Part of this training is on-the-job as they participate in research studies; however, to reduce error and time spent, it is critical that they find ways to collaborate with other biostatisticians in determining the best way to approach a research question (16,17). This on-the-job training can be obtained when statisticians are required to assist in projects which require complimentary skills to their current skillset. This continual broadening of biostatisticians skills to address the broad range of research activities.

Biostatisticians have further resources, if funding is available, in attending conferences with a focus on certain areas of specialization. Even if funding is inadequate to attend various conferences, biostatisticians can and should maintain and expand their skillsets through frequent literature searches and meetings with both biomedical investigators and other biostatisticians (19).

Infrastructure

Having some elements of a centralized biostatistical infrastructure is a beneficial resource to clinical researchers, particularly for junior faculty who are developing their own research agenda, and for seasoned clinical researchers whose valuable time would be better spent designing, conducting and interpreting research as opposed to analyzing data.

Academic medical center research infrastructure can take many forms and names, such as a Research Development Core, CTS Center and Biostatistics, Epidemiology and Research Design (BERD) units, but has the same general composition and goals. Most of these BERD units operate as a consultative basis, meaning that academic researchers Composition of these units often includes biostatisticians and epidemiologists of both Master’s and PhD level training, and may also include grant writers and other support staff. A recent study of 46 academic BERD units found wide disparities between the number of biostatisticians and epidemiologists, ranging from 3 to 86 (16). This study found that BERD units were mostly PhD level biostatisticians (median n=6, range, 2–23), followed by Masters level biostatisticians (median n=3, range, 0–15) and doctoral epidemiologists (median n=1, range, 2–10) (16). This same study also found that most of the BERDs consulted on 101–200 projects that, with most of the consulting provided for junior faculty (16).

This infrastructure, regardless of the exact title and composition, should be made available to all biomedical researchers. This requires both outreach to make researchers aware of the resources available, and solicitation of feedback from the researchers to ensure that their biostatistical needs are being met. Researchers biostatistical needs are dynamic and fluid, varying on both the type and stage of research.

Collaboration and consultation

Models for collaboration and consultation take many forms, but can generally be categorized into two architypes, which are not mutually exclusive within a given system: collaboration and consultation.

Collaboration usually occurs with a statistician who is housed within a department or division, and is not part of a BERD, but is used to describe any relationship where a biostatistician has ongoing effort with a project or study, as compared to a single time effort. These collaborations are generally on grants or other project where the biostatistician commits to a certain amount of effort if the grant or project is funded, often termed a percent effort model (19). This method can work particularly well if there is a need for a specialized statistical approach (e.g., repeated measures for randomized clinical trials) or specific content knowledge (e.g., assessment for effect modification between sedation time and outcome).

Consultation is an increasingly common method in academic medical centers because of centralization of biostatistical resources to BERDs. The consultation method is a pay for effort method of obtaining biostatistical support, and can encompass a wide range of topics and durations, often called a fee-for-service model (19). They are defined as time-limited engagements with a specified outcome or product. Those can be power/sample size calculations for a grant proposal to analysis of data which has been collected. These consultations may or may not result in a funded grant or project, and those grants or projects only include ongoing funding for BERD support half of the time (16). In funded grants where there is budgeted effort for ongoing BERD support, the relationship changes from a consultative nature to a collaborative nature.

Conclusions

Biostatistical support in academic medical centers requires a mixed model of biostatistical support covering the broad range of resources described above. This spectrum of resources allows CTS teams to identify their biostatistical needs and seek out the level of support. It also allows for a wide range of funding sources and financial stability for biostatisticians.

Acknowledgements

Funding: This study has been funded, in part, by grants from the National Institute for Occupational Safety and Health (NIOSH/CDC), Education and Research Center training grant T42/CCT810426-10. The CDC/NIOSH is not involved in this manuscript.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Demange MK, Fregni F. Limits to clinical trials in surgical areas. Clinics 2011;66:159-61. [Crossref] [PubMed]
2. Thiese MS, Arnold ZC, Walker SD. The misuse and abuse of statistics in biomedical research. Biochem Med (Zagreb) 2015;25:5-11. [Crossref] [PubMed]
3. Cassidy LD. Basic concepts of statistical analysis for surgical research. J Surg Res 2005;128:199-206. [Crossref] [PubMed]
4. Kurichi JE, Sonnad SS. Statistical methods in the surgical literature. J Am Coll Surg 2006;202:476-84. [Crossref] [PubMed]
5. Derossis AM, DaRosa DA, Dutta S, et al. A ten-year analysis of surgical education research. Am J Surg 2000;180:58-61. [Crossref] [PubMed]
6. Feinstein AR. Clinical biostatistics. XXV. A survey of the statistical procedures in general medical journals. Clin Pharmacol Ther 1974;15:97-107. [Crossref] [PubMed]
7. Emerson JD, Colditz GA. Use of statistical analysis in the New England Journal of Medicine. N Engl J Med 1983;309:709-13. [Crossref] [PubMed]
8. Oliver D, Hall JC. Usage of statistics in the surgical literature and the 'orphan P' phenomenon. Aust N Z J Surg 1989;59:449-51. [Crossref] [PubMed]
9. Greenland S, Senn SJ, Rothman KJ, et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol 2016;31:337-50. [Crossref] [PubMed]
10. Gore SM, Jones IG, Rytter EC. Misuse of statistical methods: critical assessment of articles in BMJ from January to March 1976. Br Med J 1977;1:85-7. [Crossref] [PubMed]
11. Kim JS, Kim DK, Hong SJ. Assessment of errors and misused statistics in dental research. Int Dent J 2011;61:163-7. [Crossref] [PubMed]
12. White SJ. Statistical errors in papers in the British Journal of Psychiatry. Br J Psychiatry 1979;135:336-42. [Crossref] [PubMed]
13. Parsons NR, Price CL, Hiskens R, et al. An evaluation of the quality of statistical design and analysis of published medical research: results from a systematic survey of general orthopaedic journals. BMC Med Res Methodol 2012;12:60. [Crossref] [PubMed]
14. Hall JC, Hill D, Watts JM. Misuse of statistical methods in the Australasian surgical literature. Aust N Z J Surg 1982;52:541-3. [Crossref] [PubMed]
15. Welty LJ, Carter RE, Finkelstein DM, et al. Strategies for developing biostatistics resources in an academic health center. Acad Med 2013;88:454-60. [Crossref] [PubMed]
16. Rahbar MH, Dickerson AS, Ahn C, et al. Characteristics of Biostatistics, Epidemiology, and Research Design Programs in Institutions With Clinical and Translational Science Awards. Acad Med 2017;92:229-36. [Crossref] [PubMed]
17. Oster RA, Lindsell CJ, Welty LJ, et al. Assessing statistical competencies in clinical and translational science education: one size does not fit all. Clin Transl Sci 2015;8:32-42. [Crossref] [PubMed]
18. Havermahl T, LaPensee E, Williams D, et al. Model for a university-based clinical research development infrastructure. Acad Med 2015;90:47-52. [Crossref]
19. Perkins SM, Bacchetti P, Davey CS, et al. Best Practices for Biostatistical Consultation and Collaboration in Academic Health Centers. Am Stat 2016;70:187-94. [Crossref] [PubMed]