Driving blind: instituting SEP-1 without high quality outcomes data

Introduction

Sepsis afflicts around 1.7 million people annually in the United States and has an in-hospital mortality rate of 15.6% and estimated cost of over 20 billion USD (1,2). Most US hospitals participate in the federal government’s Medicare/Medicaid health insurance program administered by the Centers for Medicare and Medicaid Services (CMS). This program pays 63% of all inpatient medical costs making it the largest single payer in the US (1). CMS develops and institutes performance measures (PM) for illnesses having a major impact on the health of the US population with the goal of promoting quality and cost-effective care nationally. Given the significant health and financial burdens of sepsis, CMS instituted the Severe Sepsis and Septic Shock Performance Measure bundle (termed SEP-1) in 2015 (Table 1) (3).

Table 1 Components included in the 2015 and 2018 versions of SEP-1
Full table

CMS employs stepwise enforcement programs that progressively incentivize hospitals to comply with its PMs. Initially, to receive any reimbursement from CMS, US hospitals must participate in CMS’s Hospital Inpatient Quality Reporting Program (IQR) (4,5). Enrolled hospitals are only required to report their compliance data to not jeopardize CMS payment. The next step is for a PM to become part of the Hospital Compare program in which CMS publicly reports hospital’s compliance with a measure (6,7). This program is intended to stimulate a hospital to follow PMs based on its concerns that poor compliance data will cause consumers to spend health care dollars elsewhere. The final step CMS employs to promote PM compliance, is to include the measure in the Value-Based Purchasing program. Here, CMS bases hospital reimbursement directly on whether the measure was followed for patients (8,9). In 2018, using data collected during 2017, SEP-1 became part of CMS Hospital Compare program (6).

The fact that SEP-1’s reporting requirements and impact on reimbursement may be forcing hospital-based clinicians to change their practice for septic patients, has engendered strong opinions. Hospitals and health-care workers have reported that SEP-1 is much more complex and requires many more hours of valuable health worker time than most other approved PMs (10-12). Concerns have also been raised that SEP-1’s rigid requirements do not allow clinicians to adjust their practice sufficiently for septic patients, a group that differs greatly in underlying comorbidities and sepsis severity. In fact, sepsis experts have noted that SEP-1 interventions lack data to support benefit and when rigidly applied may harm some patients (9,12-17). Consistent with this latter concern, SEP-1 has been revised twice since it was originally proposed and then instituted, and several interventions have been removed. Unfortunately, SEP-1 continues to mandate care encompassing unproven components that may be ineffective or harmful to some patients (18). Yet for this unproven care, hospitals have had to expend valuable resources to comply with and report on the measure (14,19,20).

This review will examine the development of SEP-1, the measure’s potential risks in its current form, and initial data from the last four years regarding compliance with it and its possible effects on morbidity and mortality. Unfortunately, only uncontrolled and potentially biased observational data are presently available to assess these latter parameters. While compliance data may be measurable, SEP-1’s effects on morbidity and mortality can only be speculated about. Without reproducible high quality outcomes data from randomized controlled trials (RCT) comparing SEP-1’s individual components and overall combined requirements vs. clinician’s usual sepsis care, the safety and benefit of SEP-1 will remain unknown.

SEP-1 and the Surviving Sepsis Campaign sepsis bundles

Developers of the version of SEP-1 introduced in 2015 included several past presidents and members of the Society of Critical Care Medicine (SCCM) (21). The PM was closely patterned after the 2012 Surviving Sepsis Campaign’s (SSC) guideline sepsis bundle that some of these individuals were also closely involved with (21,22). Therefore, understanding the missteps that have been taken with the SSC bundles is helpful in understanding the problems that have occurred with SEP-1.

The SSC introduced two sepsis bundles in 2004, each directing a group of interventions be completed within specified time periods (23). These SSC bundles, like SEP-1 and other CMS endorsed PMs, were based on the idea that administering a group of evidence-based interventions together for an illness would result in significantly better outcomes than if these interventions were individually instituted (24). Unfortunately, the initial 2004 SSC sepsis bundles included several controversial interventions lacking strong and reproducible supporting evidence including activated protein C (APC), intense glucose control, corticosteroids and early goal directed therapy (EGDT). All of these interventions were proven ineffective or harmful in subsequent RCTs and over time removed from later SSC bundle versions (25-30).

No RCTs were ever done testing any of the SSC bundles either before or after their implementation, to provide high or even moderate quality evidence to support their use. Instead, ongoing promotion of the SSC bundles was based on low quality observational studies reporting that survival was improved in septic patients during a period following vs. before institution of a bundle or in groups of concurrent septic patients that did vs. did not complete a bundle (21). However, these observational bundle studies included many of the same interventions that were subsequently disproven in one or more RCTs. With the benefit of hindsight, the SSC bundles should have been supported with high-quality data showing consistent overall benefit before they were implemented. In a rapidly lethal disease with a high mortality rate like sepsis, the decision to publish guidelines based on low quality data was questionable at best and has required multiple guideline bundle revisions following studies disproving components. Although the SSC sepsis bundles have now evolved into a single bundle with fewer components, it unfortunately continues to include controversial interventions that lack strong supporting evidence and which experts report concerns about (31). Consistent with these concerns, a recent large analysis of experience with the SSC bundles demonstrated that close to 75% of patients had not been administered all components (32). Fortunately though, unlike the SEP-1 mandate that can be tied to hospital reimbursement and compel clinicians to use it, the SSC sepsis bundles have been part of a guideline that clinicians can institute or not based on their experience and judgement.

Development and institution of SEP-1

The National Quality Forum (NQF) initially evaluates PMs for CMS (21). The NQF is largely funded by grants from the US government and membership is made up of experts from industry, healthcare groups and academia. If the NQF finds sufficient criteria have been met, it will endorse a measure for adoption by CMS. In 2008, Henry Ford Hospital presented a sepsis bundle PM to the NQF for endorsement. There was a requirement in the submitted bundle to employ central venous pressure (CVP) and central venous oxygen saturation (ScVO₂) measures to guide hemodynamic support (21). Henry Ford Hospital and one of the PM’s authors had held the patent on a continuous ScVO₂ catheter that would provide these measures and allow this stipulation in the PM to be met (21,33). Ultimately, the NQF did not endorse the use of CVP and ScVO₂ measures in 2008 but did endorse the other submitted bundle components, although CMS did not formally adopt these at the time (21). The 2008 sepsis bundle then came up for re-endorsement by the NQF in 2012. For this re-endorsement, Henry Ford Hospital submitted its originally proposed PM and inexplicably again inserted the requirement for CVP and ScVO₂ measures for persistent hypotension that had been previously excluded by the NQF. On this second evaluation, the NQF reversed itself and endorsed the entire measure including the need for CVP and ScVO₂ guided hemodynamic support. The requirement for these measures was approved despite the following knowledge. First, rather than consensus, there was marked debate in the literature as to whether use of CVP and ScVO₂ measures benefited sepsis outcomes (21). Second, no new high-quality data was presented to justify including the hemodynamic measures in the PM five years after they had been originally rejected (21). Third, the experimental nature of these measures for sepsis still existed and three RCTs specifically testing their benefits were nearing completion (26,29,30). Finally, multiple professional medical societies had strongly encouraged NQF not to endorse the measures (21). Later examination of this review and endorsement process noted not only the potential financial conflicts related to Henry Ford Hospital holding the patent for a catheter specifically designed for CVP and ScVO₂ measures, but also the presence of troubling relationships between an NQF committee member, a trade association, a manufacturer of CVP and ScVO₂ catheters, and developers of the PM (21,33). These potentially conflicted relationships weaken the credibility of the NQFs final endorsement of SEP-1 (21).

CMS formally adopted SEP-1 in early 2014 (21). Shortly after this announcement, the first of the three RCTs testing EGDT (ProCESS) reported that CVP and ScVO₂ directed hemodynamic support for early sepsis was no better than usual care (29). These findings were confirmed by the two subsequently reported RCTs (26,30). CMS then delayed adoption of SEP-1 and revised the measure. Despite these three large RCTs demonstrating no clinical improvement with the use of ScVO₂ and CVP measures, CMS in 2015 adopted a revised version of SEP-1 that still included them (Table 1) (21,34). Specifically, this 2015 version of SEP-1 required that septic patients with hypotension 6 hours after presentation have either a five-component volume status and tissue perfusion exam (including vital sign, cardiopulmonary, capillary refill, peripheral pulse and skin exam) or two of the following: CVP measure, ScVO₂ measure, cardiac ultrasound or volume challenge test (Table 1).

The initial 2015 version of SEP-1 drew criticism from clinicians caring for septic patients (9,12,13,35). SEP-1 required considerable resources to document and comply with and was one of the most complex PMs ever adopted by CMS. Despite being of no proven survival benefit in addition to increasing costs, CVP and ScVO₂ measures were options included in this version. Other required hemodynamic interventions such as the five-component physical exam and cardiac ultrasound also appeared to lack sufficient supporting evidence to warrant inclusion in a nationally mandated PM. SEP-1 required administration of an inflexible 30 mL/kg fluid infusion for all hypotensive patients regardless of co-morbidities and pre-infusion volume status. In a patient fluid overloaded at the onset of septic shock, or in one with heart failure or end stage renal disease, this large volume could be harmful. Furthermore, SEP-1 required serial lactate measurements to guide hemodynamic support. However, lactate is not a direct or indirect measure of volume status and could be misleading (17,36). Lastly, SEP-1 required clinicians to administer broad-spectrum antibiotics within 3 hours of presentation in all patients with suspected sepsis. In stable patients, administering broad-spectrum antibiotics within 3 hours before infection is confirmed, could result in excessive and potentially harmful antibiotic use in those who after a more careful and extended examination might be proven not to have infection (9,16,37).

We were not familiar with any high-quality reproducible data supporting the hemodynamic interventions that CMS was mandating in 2015 version of SEP-1. We therefore performed a systematic review to objectively examine the strength of the evidence underlying the survival benefit of these required interventions (CVP and ScVO₂ measures were not re-examined since they had already been shown with high quality evidence to not be beneficial) (34). Serial lactate measures and 30 mL/kg fluid infusions were only supported with low quality evidence consisting of observational studies that did not control for confounders or RCTs with bias or that did not show benefit. No study had assessed the survival benefit of the five-component volume status and tissue perfusion exam or bedside cardiovascular ultrasound in septic patients. Three RCTs had examined the fluid challenge techniques in septic patients but had not found them to be beneficial. Only one low quality observational study had actually examined the 2015 version of SEP-1 in its entirety (38).

The findings from this analysis confirmed concerns that the hemodynamic interventions mandated in the 2015 SEP-1 version lacked any strong supportive scientific evidence to justify their implementation as the standard care for every suspected septic patient. CMS revised SEP-1 in 2018 and adopted a version that no longer stipulated how clinicians should direct hemodynamic support for persistent hypotension (Table 1). However, this simplified bundle still required that all patients presenting with suspected sepsis receive antibiotics within 3 hours and have a 30 mL/kg fluid infusion if meeting septic shock criteria and have a second lactate level measured if the initial lactate was ≥2 mmol/L. Criticism continued that mandating even this pared down but still not scientifically based and inflexible version of SEP-1 posed a risk if uniformly applied for all potentially septic patients (15,17,39).

We then performed another systematic review and meta-analysis and found that only low quality evidence from 17 observational studies overall supported potentially improved survival with sepsis bundles focusing primarily on antibiotic and fluid support (18). The timing of antibiotics and amounts of fluid required in these bundles varied. Only 1 of 17 studies examined a bundle that required serial lactate measures. None of the studies examined the adverse effects of implementing SEP-1 including excessive antibiotic treatment and fluid overload. Our systematic review of this low quality evidence concluded that while a sepsis PM could facilitate care of the septic patient, “without high-quality evidence for its safety and benefit, it should not preempt a qualified clinician’s judgement or hinder the clinician’s medical practice” (18).

Basis for potential harm with SEP-1

Consumers can decide with SEP-1’s compliance data now publicly available via CMS’s Hospital Compare program, where they will spend their healthcare dollars (6,7). Because SEP-1 is an all-or-none PM, these circumstances may result in pressure for some clinicians to alter how they typically treat sepsis no matter what hospital they work at. There is presently no high-quality data to inform clinicians as to the risks and benefits of SEP-1’s mandated interventions when administered uniformly to all potentially septic patients. The possibility that SEP-1 is harmful for some patients is of equal weight and importance to the possibility that it is beneficial for others.

SEP-1 requires all patients with suspected sepsis to be administered broad-spectrum antibiotics within 3 hours of the time a possible infection and 2 of 4 systemic inflammatory response syndrome (SIRS) criteria are documented. No one would disagree that antibiotic treatment is essential as early as possible for patients presenting with septic shock or a clear source of infection and studies support this approach (40-42). However, rapid antibiotic administration in many potentially septic but stable patients with unconfirmed infection may not be as urgently needed (42,43). The signs and symptoms used to identify patients with sepsis syndrome are neither specific nor diagnostic. Many patients with conditions not caused by infections can present with the same 2 of 4 SIRS criteria and an elevated lactate used to diagnose sepsis (e.g., cirrhosis, pancreatitis, heart failure, or asthma exacerbation to name a few). These non-infected patients would not benefit and might be harmed by antibacterial therapy. In a retrospective analysis of 2,579 patients admitted to ICUs in the Netherlands with a diagnosis of sepsis, 40% were subsequently found to have a low or no likelihood of infection based on CDC and International Sepsis Forum criteria (44). A retrospective analysis comparing patients with severe sepsis who progressed to septic shock versus those that did not, found that the median time to antibiotic administration in the 2945 patients not developing shock ranged up to almost 5 hours [median 2.76 h (IQR 1.60–4.82)] (43). Furthermore, in the other 984 patients, delaying antibiotics for as long as 5 hours did not appear to increase the risk of shock. While the limited data available now suggests that introduction of SEP-1 has been associated with increased antibiotic use, whether it has caused more patients without infection to receive antibiotics is unknown (14). However, looking at a prior similar experience when CMS reduced the required time for antibiotic administration for community acquired pneumonia (CAP) from 8 to 4 hours, diagnostic accuracy for the condition decreased from 74% to 66% in EDs and inappropriate antibiotic administration increased (45,46). In a survey, 42% of physicians noted that in order to comply with the CMS CAP guidelines, they prescribed antibiotics to patients they did not think had pneumonia (46). Inappropriate antibiotic administration has clearly documented risks. When broad spectrum antibiotics were prescribed longer and without a clear indication in a large group of post-operative patients, it resulted in higher rates of C. difficile infection and acute kidney injury (47). In 1,488 hospitalized general medical patients, 20% of adverse drug effects were due to antibiotics that were not clinically indicated (48). Finally, even one day of anti-pseudomonas β-lactam antibiotic coverage was shown to increase the risk of developing antibiotic resistance to anti-pseudomonas agents (49). Pressure to complete SEP-1 within a limited time period will likely lead to wider prescribing of broad-spectrum antibiotics (which include anti-pseudomonas coverage) and will have unintended and potentially harmful consequences. Of note, the Infectious Disease Society of America (IDSA) did not endorse the 2016 SSC guidelines, which are similar to SEP-1 due in part to their concern that these recommendations would result in excessive and harmful antibiotic use (37).

SEP-1 still requires administration of a fixed 30 mL/kg fluid volume within 3 hours for patients presenting with initial hypotension or meeting the measure’s definition of septic shock [i.e., 2 of 4 SIRS criteria with either systolic blood pressure <90 mmHg or a lactate ≥4 mmol/L]. This requirement appears to be based on recommendations in the SSC’s 2012 and 2016 guidelines (22,50). Once again, no one would disagree that a patient presenting with sepsis and hypotension needs prompt hemodynamic support to limit organ injury and this usually includes early fluid resuscitation. However, the actual volume a patient requires or tolerates will vary greatly based on comorbidities, presenting fluid status and the severity and type of infection. The fluid requirement for an obese 80-year-old patient with a history of congestive heart failure and chronic renal dysfunction presenting with sepsis due to a urinary tract infection would likely be very different than an otherwise healthy but dehydrated 40-year-old with a ruptured appendix. Because of such differences, there is substantial concern that SEP-1’s mandated 30 mL/kg fluid volume will be harmful, being too much for some patients or too little for others (13,15,35). While the 2016 SSC guideline based this 30 mL/kg on the average pre-randomization fluid volumes usual care patients received in the ProCESS, ARISE and ProMISe trials testing EGDT, these amounts varied greatly as can be seen by the large standard deviations or intra-quartile ranges (IQR) in the fluids actually administered: ProCESS—mean 28 mL/kg, SD ±21 mL/kg; ARISE—mean 34.7 mL/kg, SD ±20.1 mL/kg; and ProMISe—median 1,790 mL, IQR 1,000–2,500 mL] (26,29,30,50). Similar variability in the fluid volumes septic patients initially received was reported in an analysis of 3,686 patients from 9 hospitals in New York state between 2014 and 2016 (mean 27 mL/kg, SD ±20 mL/kg) (51). This variability is consistent with clinicians titrating fluids in hypotensive patients with smaller challenges that are discontinued when blood pressure rises, vasopressors are started, or there is an inadequate response to fluids and concern for fluid overload. In an observational study by the European Society of Intensive Care Medicine in 2013, the median (IQR) volume employed in fluid challenges across 2,213 critically ill patients with hypotension or other evidence of organ hypoperfusion and including 595 (27%) with sepsis, was 500 (500–999 mL) (52). In a worldwide survey of academic intensivists with 3,138 responses, only 20% reported a fluid bolus should be greater than 750 mL (53). The total fluid volume administered to patient’s pre-randomization in the Restricted Fluid Resuscitation in Sepsis-associated Hypotension (REFRESH) trial was only 1,000 mL while the first challenge after enrollment was to be 1,000 mL followed by 500 mL boluses as needed (54). Taken together, these data indicate clinicians use smaller volumes to assess volume responsiveness and titrate fluid administration to effect instead of indiscriminately administering a large fixed volume (i.e., 30 mL/kg) to all patients with suspected septic shock regardless of their underlying comorbidities. Finally, the SSC guidelines appropriately grade the evidence supporting use of a 30 mL/kg fluid challenge for every septic shock patient as weak, a grade consistent with findings from our systematic review (34,50). Standardization of this rigid unproven fluid requirement in a nationally mandated performance measure may have unintended consequences and result in net harm in many health care centers. In the absence of high quality RCTs to support it as usual care, it is difficult to justify this SEP-1 component for a rapidly lethal disease with a high mortality rate (34).

Finally, SEP-1 requires a follow-up lactate by 6 hours if an initial one is ≥2 mmol/L, presumably to be used as an indication for continued hemodynamic resuscitation and support. Increased lactate levels and their persistence may be a predictor of poor outcome (55). However, in RCTs conducted in the US, use of hemodynamic support protocols targeting reductions in lactate levels either in only septic patients (two studies) or in septic and other critically ill patients (one study) did not improve or had unclear effects on survival (56-58). Failure to show benefit with this approach is probably because lactate elevation is complex and has many etiologies not necessarily associated with septic shock, including; increased adrenergic stimulation during states of stress; thiamine deficiency; reduced lactate clearance due to renal/hepatic failure; infarcted tissue; impaired tissue oxygen consumption; and shock states causing very low cardiac output (17,36,59,60). Administering fluids based on lactate elevations seems unfounded since it is not a measure of volume status per se and doing so risks the development of fluid overload (especially in renal failure) and tissue edema that may itself interfere with tissue oxygen extraction (36). The 2016 SSC guidelines grade both the recommendation and evidence supporting the use of serial lactate levels for hemodynamic decisions as weak yet it must be measured at least once and sometimes serially in SEP-1 (50).

Experience with SEP-1: hospital compliance

At least eight studies are now available describing compliance with SEP-1 based on data collected over periods between October 2015 and March 2018 and that would pertain primarily to compliance with the 2015 SEP-1 version (38,61-67) (Figure 1, Table 2). Overall compliance rates (i.e., the percentage of patients within a hospital reportedly receiving all SEP-1 components) varied widely across hospitals in these studies from 31 to 80% (Table 2) (38,61-67). One study noted a compliance of 31% before and 71% after introduction of a code sepsis protocol (67). Overall however, 75% of studies reported overall mean or median compliance rates that were ≤54%. These findings indicate that to date, compliance with SEP-1 has been low. Several hospital and patient characteristics associated with the reported compliance rates are outlined in Table 2. However, given the concerns and questions that have been raised about SEP-1, there are important reasons justifying non-compliance including: uncertainty about the diagnosis of sepsis in a particular patient, the presence of contraindications for required bundle interventions, need for additional time to confirm an infection diagnosis and avoidance of unnecessary antibiotic administration in a minimally ill patient, and clinician concern that required interventions are unproven (e.g., survival benefit of performing serial lactate measures). Although an option may be available to withhold bundle components, how often clinicians have employed this when needed and for what reasons are unknown. Thus far, little data is available to determine the reasons for SEP-1 noncompliance in patients. However, a study in 1,027 patients from two hospitals in New York which looked specifically at compliance with the 30 mL/kg fluid infusion requirement (present in a New York State Department of Health mandate and SEP-1), found this compliance was significantly lower in patients with CHF or chronic kidney disease and contraindications to excessive fluid administration (15).

Figure 1 The median or mean (designated by * or ** respectively) percentages of either hospitals or patients reportedly compliant with the Centers for Medicare and Medicaid Services sepsis bundle performance measure (SEP-1) in eight studies. Interquartile ranges (IQR) or standard deviations (±) are shown where they were provided. At the bottom of the figure are shown the total number of hospitals or patients these percentage compliances were measured across. For one study by Barbash et al. that compared compliances in safety-net versus non-safety-net hospitals and the study by Whitfield et al. that compared compliances before (pre) and after (post) implementation of a “sepsis code” hospital protocol, two bars are shown.

Table 2 Summary of compliance data and other results from eight studies examining experience with the CMS sepsis bundle performance measure (SEP-1)
Full table

Experience with SEP-1: impact on survival

At least four observational studies have examined the impact of SEP-1 on mortality (Table 3) (14,38,62,67). While the unadjusted odds ratio of survival (95% CI) in the only multi-center study was significantly increased in compliant patients [OR: 1.82 (95% CI: 1.19, 2.80), P=0.006], statistical significance was lost after adjustment for other variables impacting survival [OR: 1.36 (95% CI: 0.85, 2.18), P=0.21] (62). In the largest study, mortality rates did not differ significantly comparing patients from the first three months of 2015 before introduction of SEP-1 versus patients from similar time periods in either 2016 or 2017 after the measure’s introduction [86 of 615 (14%) vs. either 87 of 591 (14.7%) or 91 of 596 (15.3%), P≥0.53 for each comparison] (14). The two remaining studies reported unadjusted survival was increased either after introduction of SEP-1 [35 survivors of 48 total (72.9%) before vs. 94 of 110 (85.4%) after] or a code sepsis protocol designed to increase SEP-1 compliance [288 of 300 (96%) before vs. 150 of 150 (100%) after] (38,67). Reported p-values for these changes were 0.05 and 0.01 respectively. But these two retrospective studies were confounded. The former study reported baseline imbalances favoring SEP-1 patients (i.e., decreased hypotension and creatinine levels) and the latter introduced a code sepsis protocol to improve SEP-1 compliance that potentially itself improved survival. Furthermore, in the study which reported baseline imbalances, our calculations suggest that the odds ratio of survival (95% CI) may cross the no effect line [2.18 (0.95, 5.00)] and P=0.06 (Chi-square) (38).

Table 3 Summary of outcome data and other results from four studies examining experience with the CMS sepsis bundle performance measure (SEP-1)
Full table

In the limited data to date, it is unclear whether SEP-1 has had a substantial impact on mortality. CMS will presumably publish the much larger overall experience with the measure and provide analysis of it. However, without prospective high quality data controlling for bias such as with RCTs, it will be unknown whether any apparent reduction in mortality with SEP-1 was due to the measure’s components or other factors. As noted above, while repeated observational studies suggested that both the 6- and 24-hour SSC sepsis bundles improved survival, subsequent RCTs showed that costly components in the bundles were ineffective or harmful. A striking example was a retrospective analysis of a 7.5-year experience with the SSC bundles (32). This analysis included almost 30,000 patients that had unrefereed data submitted to the SSC in a manner analogous to the data collected for SEP-1. This analysis reported that after adjustment for multiple variables, the components directing hemodynamic support with CVP and ScVO₂ measures were significantly (P<0.001) associated with improved survival. Yet three large well-conducted RCTs then showed this practice was no more beneficial than usual care but increased cost (26,29,30). The SSC analysis also reported that policies directing administration of corticosteroids or APC were associated with improved risk-adjusted survival either significantly or in a strong trend (P=0.03 and 0.06 respectively) but RCTs have also not supported these results. Multiple factors will confound any observational analysis of the benefits or harm from SEP-1 or its individual components including; ascertainment bias; the possibility that one highly beneficial component could mask non-beneficial or possibly harmful ones; uncontrolled data collection; use of adjunctive aids with SEP-1 that impact survival; and failure to adequately adjust for unrecognized but influential variables during survival analysis.

Conclusion: do not drive blind—obtain high quality data

Sepsis is a syndrome caused by many different of types of infection and is a major US health problem with a high morbidity and mortality. A PM to assess and help guide hospitals’ diagnosis and treatment of septic patients may have benefit. However, sepsis is a complex syndrome. First, it has signs and symptoms that overlap with other serious conditions and that make confirmation of its initial diagnosis difficult. Second, the severity and course of sepsis varies greatly among patients based on the type of infection producing it and underlying comorbidities complicating it. There is no question that patients presenting with sepsis require prompt diagnosis, antibiotic therapy, and when needed—hemodynamic support. However, experts agree that these necessary therapies have risks if administered indiscriminately and are not adjusted based on the likelihood and severity of the underlying infection and on complicating comorbid conditions (9,12,13,15,17,37).

There have been ongoing concerns that SEP-1 mandates an inflexible “one size fits all” therapeutic approach for sepsis that lacks high or even moderate level evidence demonstrating its benefit and defining its risks in the highly diverse group of patients it is directed at. While the source of the low compliance reported so far with SEP-1 can be from many etiologies, it may very well reflect these concerns and clinicians’ need to individualize care in patterns not consistent with the measure. Without high quality evidence based on reproducible RCT, the true benefits and risks associated with SEP-1 are unknown. Before standardizing therapies in a rapidly lethal disease with a high mortality rate, we must hold to the dictum “do no harm” and obtain high quality evidence proving benefit and thereby safety. Our patients deserve no less.

Acknowledgments

We greatly appreciate Ms. Kelly Byrne’s editorial assistance. The Clinical Center at the National Institutes of Health (NIH) intra-mural funding supported this work. This manuscript represents the views of the authors and does not necessarily represent the official position of the NIH or the U.S. Government.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

References

1. Torio C, Andrews R. National Inpatient Hospital Costs: The Most Expensive Conditions by Payer, 2011. Agency for Healthcare Research and Quality 2013.
2. Rhee C, Dantes R, Epstein L, et al. Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014. JAMA 2017;318:1241-9. [Crossref] [PubMed]
3. CMS to Improve Quality of Care during Hospital Inpatient Stays. CMS.gov Fact Sheet. 2014. Available online: . 26 November 2019.https://www.cms.gov/newsroom/fact-sheets/cms-improve-quality-care-during-hospital-inpatient-stays
4. Hospital Inpatient Quality Reporting (IQR) Program. CMS.gov. Available online: https://www.qualitynet.org/inpatient/iqr
5. Hospital Inpatient Quality Reporting Program. CMS.gov. 2017. Available online: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/HospitalRHQDAPU
6. Claypool S. Hospital Compare lifts the veil on sepsis care. Check your hospital’s score. 2018. Available online: https://www.statnews.com/2018/07/26/hospital-compare-lifts-veil-on-sepsis/
7. Davey RT Jr, Dewar RL, Reed GF, et al. Plasma viremia as a sensitive indicator of the antiretroviral activity of L-697,661. Proc Natl Acad Sci U S A 1993;90:5608-12. [Crossref] [PubMed]
8. Hospital value-based purchasing. In: Medicare Learning Network. 2017. Available online: https://www.cms.gov/Outreach-and-Education/Medicare-Learning-Network-MLN/MLNProducts/MLN-Publications-Items/CMS1255514
9. Faust JS, Weingart SD. The Past, Present, and Future of the Centers for Medicare and Medicaid Services Quality Measure SEP-1: The Early Management Bundle for Severe Sepsis/Septic Shock. Emerg Med Clin North Am 2017;35:219-31. [Crossref] [PubMed]
10. Sepsis Bundle Project (SEP). Specifications Manual for National Hospital Inpatient Quality Measures, 2017.
11. Walkey AJ, Lindenauer PK. Keeping It Simple in Sepsis Measures. Journal of hospital medicine 2017;12:1019-20. [Crossref] [PubMed]
12. Klompas M, Rhee C. The CMS Sepsis Mandate: Right Disease, Wrong Measure. Ann Intern Med 2016;165:517-8. [Crossref] [PubMed]
13. Marik PE, Malbrain M. The SEP-1 quality mandate may be harmful: How to drown a patient with 30 mL per kg fluid! Anaesthesiol Intensive Ther 2017;49:323-8. [Crossref] [PubMed]
14. Esposito A, Silverman ME, Diaz F, et al. Sepsis Core Measures - Are They Worth the Cost? J Emerg Med 2018;55:751-7. [Crossref] [PubMed]
15. Truong TN, Dunn AS, McCardle K, et al. Adherence to fluid resuscitation guidelines and outcomes in patients with septic shock: Reassessing the "one-size-fits-all" approach. J Crit Care 2019;51:94-8. [Crossref] [PubMed]
16. Septimus EJ, Coopersmith CM, Whittle J, et al. Sepsis National Hospital Inpatient Quality Measure (SEP-1): Multistakeholder Work Group Recommendations for Appropriate Antibiotics for the Treatment of Sepsis. Clin Infect Dis 2017;65:1565-9. [Crossref] [PubMed]
17. Marik PE. SEP-1: The Lactate Myth and Other Fairytales. Crit Care Med 2018;46:1689-90. [Crossref] [PubMed]
18. Pepper DJ, Sun J, Cui X, et al. Antibiotic- and Fluid-Focused Bundles Potentially Improve Sepsis Management, but High-Quality Evidence Is Lacking for the Specificity Required in the Centers for Medicare and Medicaid Service's Sepsis Bundle (SEP-1). Crit Care Med 2019;47:1290-300. [Crossref] [PubMed]
19. Wall MJ, Howell MD. Variation and Cost-effectiveness of Quality Measurement Programs. The Case of Sepsis Bundles. Ann Am Thorac Soc 2015;12:1597-9. [PubMed]
20. Barbash IJ, Rak KJ, Kuza CC, et al. Hospital Perceptions of Medicare's Sepsis Quality Reporting Initiative. J Hosp Med 2017;12:963-8. [Crossref] [PubMed]
21. Jaswal DS, Natanson C, Eichacker PQ. Endorsing performance measures is a matter of trust. Bmj 2018;360:k703. [Crossref] [PubMed]
22. Dellinger RP, Levy MM, Rhodes A, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med 2013;41:580-637. [Crossref] [PubMed]
23. Dellinger RP, Carlet JM, Masur H, et al. Surviving Sepsis Campaign guidelines for management of severe sepsis and septic shock. Crit Care Med 2004;32:858-73. [Crossref] [PubMed]
24. Resar R GF, Haraden C, Nolan TW. Using Care Bundles to Improve Health Care Quality. IHI Innovation Series white paper Cambridge, Massachusetts: Institute for Healthcare Improvement, 2012.
25. Finfer S, Chittock DR, Su SY, et al. Intensive versus conventional glucose control in critically ill patients. N Engl J Med 2009;360:1283-97. [Crossref] [PubMed]
26. Peake SL, Delaney A, Bailey M, et al. Goal-directed resuscitation for patients with early septic shock. N Engl J Med 2014;371:1496-506. [Crossref] [PubMed]
27. Ranieri VM, Thompson BT, Barie PS, et al. Drotrecogin alfa (activated) in adults with septic shock. N Engl J Med 2012;366:2055-64. [Crossref] [PubMed]
28. Sprung CL, Annane D, Keh D, et al. Hydrocortisone therapy for patients with septic shock. N Engl J Med 2008;358:111-24. [Crossref] [PubMed]
29. Yealy DM, Kellum JA, Huang DT, et al. A randomized trial of protocol-based care for early septic shock. N Engl J Med 2014;370:1683-93. [Crossref] [PubMed]
30. Mouncey PR, Osborn TM, Power GS, et al. Trial of Early, Goal-Directed Resuscitation for Septic Shock. N Engl J Med 2015;372:1301-11. [Crossref] [PubMed]
31. Marik PE, Farkas JD, Spiegel R, et al. POINT: Should the Surviving Sepsis Campaign Guidelines Be Retired? Yes. CHEST 2019;155:12-4. [Crossref] [PubMed]
32. Levy MM, Rhodes A, Phillips GS, et al. Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5-year study. Crit Care Med 2015;43:3-12. [Crossref] [PubMed]
33. Burton TM. New Therapy for Sepsis Infections Raises Hope but Many Questions. Wall St J 2008:A1.
34. Pepper DJ, Jaswal D, Sun J, et al. Evidence Underpinning the Centers for Medicare & Medicaid Services' Severe Sepsis and Septic Shock Management Bundle (SEP-1): A Systematic Review. Ann Intern Med 2018;168:558-68. [Crossref] [PubMed]
35. Aaronson EL, Filbin MR, Brown DF, et al. New Mandated Centers for Medicare and Medicaid Services Requirements for Sepsis Reporting: Caution from the Field. J Emerg Med 2017;52:109-16. [Crossref] [PubMed]
36. Marik P, Bellomo R. Lactate clearance as a target of therapy in sepsis: A flawed paradigm. OA Critical Care 2013. doi:. [Crossref]
37. IDSA Sepsis Task Force. Infectious Diseases Society of America (IDSA) POSITION STATEMENT: Why IDSA Did Not Endorse the Surviving Sepsis Campaign Guidelines. Clin Infect Dis 2018;66:1631-5. [Crossref] [PubMed]
38. Ramsdell TH, Smith AN, Kerkhove E. Compliance with Updated Sepsis Bundles to Meet New Sepsis Core Measure in a Tertiary Care Hospital. Hosp Pharm 2017;52:177-86. [Crossref] [PubMed]
39. Klompas M, Rhee C. Current Sepsis Mandates Are Overly Prescriptive, and Some Aspects May Be Harmful. Crit Care Med 2018. [Epub ahead of print]. [Crossref] [PubMed]
40. Kumar A, Roberts D, Wood KE, et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006;34:1589-96. [Crossref] [PubMed]
41. Liu VX, Fielding-Singh V, Greene JD, et al. The Timing of Early Antibiotics and Hospital Mortality in Sepsis. Am J Respir Crit Care Med 2017;196:856-63. [Crossref] [PubMed]
42. Seymour CW, Gesten F, Prescott HC, et al. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. N Engl J Med 2017;376:2235-44. [Crossref] [PubMed]
43. Whiles BB, Deis AS, Simpson SQ. Increased Time to Initial Antimicrobial Administration Is Associated With Progression to Septic Shock in Severe Sepsis Patients. Crit Care Med 2017;45:623-9. [Crossref] [PubMed]
44. Klein Klouwenberg PM, Cremer OL, van Vught LA, et al. Likelihood of infection in patients with presumed sepsis at the time of intensive care unit admission: a cohort study. Crit Care 2015;19:319. [Crossref] [PubMed]
45. Welker JA, Huston M, McCue JD. Antibiotic timing and errors in diagnosing pneumonia. Arch Intern Med 2008;168:351-6. [Crossref] [PubMed]
46. Nicks BA, Manthey DE, Fitch MT. The Centers for Medicare and Medicaid Services (CMS) community-acquired pneumonia core measures lead to unnecessary antibiotic administration by emergency physicians. Acad Emerg Med 2009;16:184-7. [Crossref] [PubMed]
47. Branch-Elliman W, O'Brien W, Strymish J, et al. Association of Duration and Type of Surgical Prophylaxis With Antimicrobial-Associated Adverse Events. JAMA Surg 2019;154:590-8. [Crossref] [PubMed]
48. Tamma PD, Avdic E, Li DX, et al. Association of Adverse Events With Antibiotic Use in Hospitalized Patients. JAMA Intern Med 2017;177:1308-15. [Crossref] [PubMed]
49. Teshome BF, Vouri SM, Hampton N, et al. Duration of Exposure to Antipseudomonal beta-Lactam Antibiotics in the Critically Ill and Development of New Resistance. Pharmacotherapy 2019;39:261-70. [Crossref] [PubMed]
50. Rhodes A, Evans LE, Alhazzani W, et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016. Intensive Care Med 2017;43:304-77. [Crossref] [PubMed]
51. Leisman DE, Doerfler ME, Schneider SM, et al. Predictors, Prevalence, and Outcomes of Early Crystalloid Responsiveness Among Initially Hypotensive Patients With Sepsis and Septic Shock. Crit Care Med 2018;46:189-98. [Crossref] [PubMed]
52. Cecconi M, Hofer C, Teboul JL, et al. Fluid challenges in intensive care: the FENICE study: A global inception cohort study. Intensive Care Med 2015;41:1529-37. [Crossref] [PubMed]
53. Glassford NJ, Martensson J, Eastwood GM, et al. Defining the characteristics and expectations of fluid bolus therapy: A worldwide perspective. J Crit Care 2016;35:126-32. [Crossref] [PubMed]
54. Macdonald SPJ, Keijzers G, Taylor DM, et al. Restricted fluid resuscitation in suspected sepsis associated hypotension (REFRESH): a pilot randomised controlled trial. Intensive Care Med 2018;44:2070-8. [Crossref] [PubMed]
55. Vincent JL, Quintairos ESA, Couto L Jr, et al. The value of blood lactate kinetics in critically ill patients: a systematic review. Crit Care 2016;20:257. [Crossref] [PubMed]
56. Kuan WS, Ibrahim I, Leong BS, et al. Emergency Department Management of Sepsis Patients: A Randomized, Goal-Oriented, Noninvasive Sepsis Trial. Ann Emerg Med 2016;67:367-78.e3. [Crossref] [PubMed]
57. Jones AE, Shapiro NI, Trzeciak S, et al. Lactate clearance vs central venous oxygen saturation as goals of early sepsis therapy: a randomized clinical trial. Jama 2010;303:739-46. [Crossref] [PubMed]
58. Jansen TC, van Bommel J, Schoonderbeek FJ, et al. Early lactate-guided therapy in intensive care unit patients: a multicenter, open-label, randomized controlled trial. Am J Respir Crit Care Med 2010;182:752-61. [Crossref] [PubMed]
59. Hernandez G, Bellomo R, Bakker J. The ten pitfalls of lactate clearance in sepsis. Intensive Care Med 2019;45:82-5. [Crossref] [PubMed]
60. Gattinoni L, Vasques F, Camporota L, et al. Understanding Lactatemia in Human Sepsis. Potential Impact for Early Management. Am J Respir Crit Care Med 2019;200:582-9. [Crossref] [PubMed]
61. Venkatesh AK, Slesinger T, Whittle J, et al. Preliminary Performance on the New CMS Sepsis-1 National Quality Measure: Early Insights From the Emergency Quality Network (E-QUAL). Ann Emerg Med 2018;71:10-5.e1. [Crossref] [PubMed]
62. Rhee C, Filbin MR, Massaro AF, et al. Compliance With the National SEP-1 Quality Measure and Association With Sepsis Outcomes: A Multicenter Retrospective Cohort Study. Crit Care Med 2018;46:1585-91. [Crossref] [PubMed]
63. Liao J, Aaronson E, Kim J, et al. Association of Hospital Characteristics With Early SEP-1 Performance. Am J Med Qual 2019.1062860619857028. [PubMed]
64. Greenwood-Ericksen MB, Rothenberg C, Mohr N, et al. Urban and Rural Emergency Department Performance on National Quality Metrics for Sepsis Care in the United States. J Rural Health 2019;35:490-7. [Crossref] [PubMed]
65. Barbash IJ, Kahn JM. Sepsis quality in safety-net hospitals: An analysis of Medicare's SEP-1 performance measure. J Crit Care 2019;54:88-93. [Crossref] [PubMed]
66. Barbash IJ, Davis B, Kahn JM. National Performance on the Medicare SEP-1 Sepsis Quality Measure. Crit Care Med 2019;47:1026-32. [Crossref] [PubMed]
67. Whitfield PL, Ratliff PD, Lockhart LL, et al. Implementation of an adult code sepsis protocol and its impact on SEP-1 core measure perfect score attainment in the ED. Am J Emerg Med 2019. [Epub ahead of print]. [Crossref] [PubMed]