Mismatching of population groups in thoracic surgery case control studies

Introduction

Thoracic surgery is a rapidly evolving field, in continuous need of high quality clinical evidence. Case control studies are a type of observational study in which two existing groups differing in outcome are identified and compared on the basis of some supposed causal attribute. Case-control studies are often used to identify factors that may/may not contribute to a medical condition by comparing subjects who have that condition/disease (cases) with patients who do not have the condition/disease but are otherwise similar (controls). They require fewer resources but provide less evidence than a randomized controlled trial. As analytical observational studies, control studies are analytical observational studies that represent level II-2 clinical evidence (1).

These are beneficial especially in groups with rare or long latency outcomes. Odd ratios provide the best measure of association between variables and outcomes. Unfortunately these studies are highly susceptible to bias. One of the key elements to increase the validity of case control studies is a meticulous selection of controls. This can be achieved by clearly defining the eligibility and selection criteria, selection of the groups from the same population, blinding, and addressing confounding factors in the early design stages (2). Despite these shortcomings, case control studies contribute greatly to the array of research methods used by thoracic surgeons.

By definition, this type of study approaches a hypothesis backwards, tracking the groups from outcome to exposure. A more modern approach studies dynamic populations by matching them based on time or population characteristics (3). While the study groups are defined by outcome (e.g., patients with or without lung cancer as treatment and control groups respectively), these are frequently confused with the study groups of cohort studies, in which they are defined by exposure [e.g., surgical treatment of lung cancer by video-assisted thoracoscopic surgery (VATS) or open thoracotomy]. This leads to mislabeling of cohort studies as case control and vice versa.

In this manuscript, we will provide an insight on how the validity of case control studies can be optimized, while reviewing representative manuscripts in the field of thoracic surgery. While not aimed at being a comprehensive review, it tries to highlight the key elements leading to high or low quality case control studies.

Case control study designs

The careful design of case control studies should take into consideration two different outcomes (e.g., presence or absence of lung cancer recurrence), which will define the case and control populations. Some of the studies are comparing two different populations without clearly defining the outcomes, study design or methodology. In a meta-analysis by Zhang et al. (4) compared VATS and open lobectomy in patients with borderline lung function. They have identified six representative studies, out of which three were case control. The authors concluded that all of these had high bias risk for selection, performance and detection. The mismatch and confounding factors in one of these studies (5) consisted of age differences, difference in comorbidities and lower neo-adjuvant therapy prevalence between in the case group. In another study (6) some of the important confounding elements were not reported. This raises questions in terms of the validity of the studies and ultimately of the meta-analysis itself, showing what impact the attention to study design, selection and confounding factors can have on the quality of clinical evidence.

Some of the studies are designed around the interviewing of patients and blinding, which is ideal for this type of study. Peng et al. have conducted a study investigated chronic pain following thoracic surgery (7). The detailing of the questionnaire and interview methodology increases the confidence of the readers in a well carried out case control study. Inadvertently, recall bias can occur, but it is minimized by a careful design, multiple interview methods and aids. It is commendable that the authors accurately identify and address potential bias such as recall bias and important confounding factors as part of their conclusion on chronic pain following thoracic surgery.

Selection of cases and control groups

In theory, all cases from a population could be included in a case control study. Due to practicality, most of the authors chose representative samples of their patient populations. Clearly defining the eligibility criteria for the selection and taking measures to match the case and control groups within the same populations are essential for the validity of this type of study. The poor choice of the control groups represents one of the frequent methods to introduce bias. The control groups should be as a principle free of the outcome being studied, while still being representative to the population. One of the ways in which investigators can reduce selection bias is by minimizing judgement in the selection process. This can be achieved either by blinding to the exposure or hypothesis. Some of the studies are inaccurately described as case control, when comparing cohorts of two different surgical techniques, as Weber et al. (8) have shown previously. Additionally exclusion of a significant proportion of the population from the comparison reduces the power of the study, potentially leading to false positive results or masking confounding factors.

Quantifying exposure

The measurement of exposure can prove difficult due to recall bias, when case and control patients might inaccurately remember their exposure. This can be influenced as well by the higher impact of negative events, leading to information bias. Additional bias from data gatherers who investigate more or less thoroughly depending on the status of the case or control participants can contribute to inaccuracies. These pitfalls can be avoided by blinding and improving the recall through memory aids. It is challenging to assess the methods aimed to avoid bias and the impact of bias on the results when they are not clearly specified in the methodology of case control studies.

A historical landmark study by Doll and Hill (9) quantified the effects of exposure to smoking on mortality and incidence of lung cancer in a large general and medical population. They observed an increasing incidence in the outcomes with increased smoking, estimating a mortality of up to 92% for smoking doctors. This was one of the first case control studies showing this relationship, while detailing all potential bias risks.

Due to the retrospective nature of these studies, some authors quantify exposures through objective measures and collecting data prospectively. The absence of clearly defined controls, exposure and outcomes can be misleading and raise questions on the quality of evidence (10). Maintaining comprehensive databases in a prospective manner can minimize the impact of recall bias or the logistical hurdles of organizing interviews retrospectively.

Minimizing confounding factors

Confounding factors are a great potential of bias in case control studies. This issue can be addressed either in the design phase or analysis phase, with the majority of groups opting for the latter. All studies should try to minimize confounding factors such as different surgeons, different radiologists interpreting imaging, passive smoking, and other environmental factors. One of the landmark studies in respiratory medicine by Hirayama et al. (11) of passive smoking in wives of heavy smokers. By not considering the age of the husband as a confounding factor, their results lacked significance, for which they were criticized. Their findings were confirmed only later by a group from Greece and another one in China, and now their findings are generally accepted and their work stands as a landmark finding for its time as Smith et al. shows (12).

Another eloquent example of analysis without adequately addressing confounding factors was raised by Geyer (13) in regards to the study by Markowitz et al. (14) on the relationship between asbestos, asbestosis, smoking and lung cancer. By not acknowledging confounding factors, both types of errors (false positives and true negatives) can increase. Revisiting these factors, adjusting population selection and the impact of exposure would improve the quality of these studies. Another common pitfall is the inability to recognize differences in operative techniques of different surgeons or the impact of different tumor grades (8). It is difficult to interpret the impact of each confounding factor when critically appraising clinical evidence, but their significance could be particularly important in reports of intriguing or borderline significant results.

Advantages

• Case-control studies require reasonable resources and can be performed by small teams or individual researchers in single institutions while huge and more structured experimental studies often cannot be;
• They provide the possibility to investigate a wide range of possible risk factors;
• Historically they have been at the base of a number of remarkable findings; the case-control study design is ideal for rare diseases or as a preliminary study where little is known about the association between the risk factor and disease of interest;
• Compared to prospective studies they do not have the problem of accrual to reach statistical significance; as a consequence they tend to be less expensive and shorter in duration.

Disadvantages

• Case-control studies are observational in nature and thus do not provide the same level of evidence as randomized controlled trials;
• It is not possible to obtain estimates of disease incidence among those exposed and those unexposed to a putative risk factor (except if the study is population-based);
• It may be difficult to select an appropriate control group (selection bias) or to obtain accurate unbiased measures of past exposures (information bias);
• The temporal sequence between exposure and disease may be difficult to establish (reverse causality);
• The results may be confounded by many factors and are therefore placed low in the hierarchy of evidence.

Conclusions

Case-control studies are studies in which a group of people with the condition of interest (cases) and a group without that condition (controls) are identified and the prevalence (or level) of the relevant exposure is measured and compared in the two groups. Well-designed and meticulously carried out case control studies are useful in adding to the thoracic surgery clinical evidence. Avoiding multiple bias risk factors could increase the strength of these studies. Reviewers should critically appraise the strengths and weaknesses of these studies, while maintaining a raised level of suspicion if the methodology is inaccurate or superficially described.

Acknowledgements

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Grimes DA, Schulz KF. An overview of clinical research: the lay of the land. Lancet 2002;359:57-61. [PubMed]
2. Schulz KF, Grimes DA. Case-control studies: research in reverse. Lancet 2002;359:431-4. [PubMed]
3. Vandenbroucke JP, Pearce N. Case-control studies: basic concepts. Int J Epidemiol 2012;41:1480-9. [PubMed]
4. Zhang R, Ferguson MK. Video-Assisted versus Open Lobectomy in Patients with Compromised Lung Function: A Literature Review and Meta-Analysis. PLoS One 2015;10:e0124512. [PubMed]
5. Berry MF, Villamizar-Ortiz NR, Tong BC, et al. Pulmonary function tests do not predict pulmonary complications after thoracoscopic lobectomy. Ann Thorac Surg 2010;89:1044-51; discussion 1051-2. [PubMed]
6. Lau KK, Martin-Ucar AE, Nakas A, et al. Lung cancer surgery in the breathless patient--the benefits of avoiding the gold standard. Eur J Cardiothorac Surg 2010;38:6-13. [PubMed]
7. Peng Z, Li H, Zhang C, et al. A retrospective study of chronic post-surgical pain following thoracic surgery: prevalence, risk factors, incidence of neuropathic component, and impact on qualify of life. PLoS One 2014;9:e90014. [PubMed]
8. Weber DJ, Okereke IC, Birdas TJ, et al. The "cut-in patch-out" technique for Pancoast tumor resections results in postoperative pain reduction: a case control study. J Cardiothorac Surg 2014;9:163. [PubMed]
9. Doll R, Hill AB. Lung cancer and other causes of death in relation to smoking; a second report on the mortality of British doctors. Br Med J 1956;2:1071-81. [PubMed]
10. Cattaneo SM, Park BJ, Wilton AS, et al. Use of video-assisted thoracic surgery for lobectomy in the elderly results in fewer complications. Ann Thorac Surg 2008;85:231-5; discussion 235-6. [PubMed]
11. Hirayama T. Non-smoking wives of heavy smokers have a higher risk of lung cancer: a study from Japan. Br Med J (Clin Res Ed) 1981;282:183-5. [PubMed]
12. Smith DR, Beh EJ. Hirayama, passive smoking and lung cancer: 30 years on and the numbers still don’t lie. Public Health 2011;125:179-81. [PubMed]
13. Geyer SJ. Asbestos, asbestosis, smoking, and lung cancer: study bias and confounding issues that complicate the interpretation of the results. Am J Respir Crit Care Med 2014;189:115-6. [PubMed]
14. Markowitz SB, Levin SM, Miller A, et al. Asbestos, asbestosis, smoking, and lung cancer. New findings from the North American insulator cohort. Am J Respir Crit Care Med 2013;188:90-6. [PubMed]