Assessment of clinical criteria for sepsis—was the cart put before the horse?

Sepsis is a worldwide major healthcare problem, with an estimated burden of 31.5 million cases treated each year around the globe, accounting for 5.3 million death per annum (1). Despite decades of intense research, there is to date no gold standard to define sepsis. Several international conferences have previously addressed this issue. The first one held in 1991 defined sepsis as the systemic inflammatory response to infection, and coined the acronym systemic infection response syndrome (SIRS) (2). The second international conference in 2001 extended the list of SIRS criteria, and acknowledged the lack of reliable biomarkers at this time (3). Yet, there were still flaws with the SIRS criteria as they may be present in patients without infection, and are lacking in more than 10% of patients with infection and organ dysfunction (4). A third conference has recently agreed upon a new definition of sepsis as a life threatening organ dysfunction caused by a dysregulated host response to infection (5).

Seymour et al. designed a study to provide the third International Consensus Task Force with data from large hospital databases to explore the validity of clinical criteria to identify patients with suspected infection and who are at risk of sepsis. A total of 4,885,558 adult medical records (>99% from US databases) were analyzed. These databases were selected because they included patient encounters from different phases of acute care (out of hospital, emergency department, hospital ward, ICU), under varied measurement conditions (academic and community hospitals, international locations of care, and with both community and hospital-acquired infections). Using multivariable logistic regression, a new simple score [i.e., quick Sequential (Sepsis-related) Organ Failure Assessment (qSOFA)] was developed, outperforming predictive validity for in-hospital mortality of both SIRS and SOFA score (6) in non-ICU encounters [qSOFA ranges from 0 to 3 points, with 1 point each for systolic hypotension (≤100 mmHg), tachypnea (≥22/min), or altered mentation]. However, predictive validity for in-hospital mortality of qSOFA was significantly lower than SOFA or Logistic Organ Dysfunction System (LODS) (7) scores in ICU encounters. These results were consistently observed in the external datasets and multiple sensitivity analyses confirmed the robustness of these findings.

This study of very high methodological quality provided evidence-based candidate criteria to the third international consensus definitions for sepsis and septic shock, while the two previous definitions were mainly based on expert-opinion (2,3). This study, along with the one provided in a companion paper (8), provide a major step forward rationalized risk stratification in sepsis, and should reduce heterogeneity of treatment effect in future randomized controlled trials conducted on sepsis patients (9). Care was taken by the authors to provide simple operational, and nevertheless reliable and robust tools to identify groups of patients with homogeneous risk of death. Several scores of organ dysfunction were identified as potential candidate clinical criteria for sepsis. Among them, a change of 2 points or more in SOFA score was ultimately chosen by the third International Consensus task force as clinical criteria for sepsis (in association with suspected or documented infection), as it is already widely used by the ICU medical community, is reliable and easily computed using clinical and biological variables routinely monitored in ICU (ecological validity).

However, this study has several methodological flaws. Most of the databases used in the study were retrospectively collected leading to a high rate of missing data for some variables (e.g., serum lactate level), poor accuracy of some variables (e.g., urine output in hospital ward encounters) and the use of surrogate variables (e.g., association of antibiotics administration and bacteriological sampling within a predefined time frame to define suspected infection). Another consequence of the retrospective design of the study is that potentially relevant variables not routinely monitored in electronic health records [e.g., skin mottling, a simple clinical sign of tissue hypoxia strongly related to septic shock mortality (10,11)] could not be assessed as candidate clinical criteria. Furthermore, data were almost exclusively driven from large US databases (with the exception of one small German database), and generalizability of these results in other healthcare systems is largely unknown. In addition, while a change of 2 points or more in SOFA score was chosen to account for preexisting organ dysfunction before the onset of infection in the new sepsis definition (5), neither the validity of qSOFA in subgroup of patients with preexisting organ dysfunction (neurological or respiratory), nor the validity of the 100 mmHg cut-off for systolic arterial pressure in hypertensive patients were addressed.

While this study correctly addressed construct validity (by showing acceptable agreement of qSOFA with more sophisticated scores or organ dysfunction) and predictive validity, content validity of qSOFA may be questionable since it does not detect 3 out of the 6 organ dysfunctions assessed by the SOFA score (i.e., renal, hepatic, hematologic). These unrecognized organ dysfunctions by qSOFA are nevertheless associated with in hospital mortality of ward patients with SIRS, although more loosely than cardiovascular, neurologic and pulmonary dysfunction (4). Furthermore, qSOFA and SOFA share the same limitation regarding content validity, since neither detect a dysregulated response to infection, a cornerstone of the new sepsis definition (5).

Will this study have any influence on patient care and outcome? Since qSOFA ≥2 seems to better identify a group of patients with higher mortality, and is readily available at the bedside, its use was advocated by the third International Consensus task force to (I) prompt clinicians to further investigate for organ dysfunction (using the SOFA score), (II) initiate or escalate therapy, (III) consider referral to critical care or increase the frequency of monitoring, (IV) and also to prompt consideration of possible infection in patients not previously recognized as infected. It is however unknown whether using qSOFA as a screening tool in non-ICU patients will improve sepsis morbidity and/or mortality. To the opposite, requirement for organ dysfunction to identify sepsis (as opposed to SIRS in previous definitions) may hinder early identification and treatment of sepsis patients, and ultimately may impair patient outcome. Future studies are strongly needed to confirm the effectiveness of qSOFA-based strategies on patient outcome. Although risk stratification is improved by the new sepsis definition, there is to date no evidence that patient heterogeneity (regarding host response to infection, biological pathway involved in organ dysfunction, response to therapy…) will be improved within subgroups of patient with similar mortality (as classified by the new sepsis clinical criteria). Healthcare professional adhesion to the new sepsis definition will probably depend upon demonstration that it helps to individualize therapeutic intervention as a function of patient risk of death at diagnosis, and ultimately improve sepsis and septic shock outcome. Meanwhile, it is possible that a meaningful proportion of caregivers think that the cart was put before the horse.

Acknowledgements

None.

Footnote

Provenance: This is an invited Commentary commissioned by the Section Editor Zhiheng Xu (State Key Laboratory of Respiratory Disease, Guangzhou Institute of Respiratory Disease, Department of Intensive Care, The First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China).

Conflicts of Interest: The authors have no conflicts of interest to declare.

Comment on: Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:762-74.

References

1. Fleischmann C, Scherag A, Adhikari NK, et al. Assessment of Global Incidence and Mortality of Hospital-treated Sepsis. Current Estimates and Limitations. Am J Respir Crit Care Med 2016;193:259-72. [Crossref] [PubMed]
2. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992;101:1644-55. [Crossref] [PubMed]
3. Levy MM, Fink MP, Marshall JC, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med 2003;31:1250-6. [Crossref] [PubMed]
4. Kaukonen KM, Bailey M, Pilcher D, et al. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med 2015;372:1629-38. [Crossref] [PubMed]
5. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:801-10. [Crossref] [PubMed]
6. Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996;22:707-10. [Crossref] [PubMed]
7. Le Gall JR, Klar J, Lemeshow S, et al. The Logistic Organ Dysfunction system. A new way to assess organ dysfunction in the intensive care unit. ICU Scoring Group. JAMA 1996;276:802-10. [Crossref] [PubMed]
8. Shankar-Hari M, Phillips GS, Levy ML, et al. Developing a New Definition and Assessing New Clinical Criteria for Septic Shock: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:775-87. [Crossref] [PubMed]
9. Iwashyna TJ, Burke JF, Sussman JB, et al. Implications of Heterogeneity of Treatment Effect for Reporting and Analysis of Randomized Trials in Critical Care. Am J Respir Crit Care Med 2015;192:1045-51. [Crossref] [PubMed]
10. Ait-Oufella H, Lemoinne S, Boelle PY, et al. Mottling score predicts survival in septic shock. Intensive Care Med 2011;37:801-7. [Crossref] [PubMed]
11. Coudroy R, Jamet A, Frat JP, et al. Incidence and impact of skin mottling over the knee and its duration on outcome in critically ill patients. Intensive Care Med 2015;41:452-9.