Quick Sequential Sepsisrelated Organ Failure Assessment q SOFA

Quick Sequential [Sepsis-related] Organ Failure Assessment (q. SOFA) and St. John Sepsis Surveillance Agent to Detect Patients At-Risk of Sepsis: An Observational Cohort Study Session Number S 89: Oral Presentations - Leveraging Informatics to Improve Surveillance of Disease and Events in Health Systems Robert C. Amland, Ph. D Cerner Corporation Twitter: #AMIA 2017

Disclosure I disclose the following relevant relationship with commercial interests: Dr. Amland is employed by Cerner Corporation, developer of the Millennium electronic health record system and the St. John Sepsis Surveillance Agent system. AMIA 2017 | amia. org 2

Learning objectives After participating in this session the learner should be better able to: § Evaluate sepsis surveillance systems involving how to measure time to event and compare different alert system clinimetric performance on detection of at-risk patients and outcomes AMIA 2017 | amia. org 3

Introduction Background Objectives § St. John Sepsis Surveillance Agent incorporates the 2012 consensus definition of sepsis § Compare incidence rates of q. SOFA to St. John Sepsis Surveillance Agent § Critique of 2012 definition suggest SIRS is non-specific to sepsis § Understand the temporal relationship between onset of infection and q. SOFA versus St. John Sepsis Surveillance Agent § Estimate the clinimetric performance of q. SOFA and St. John Sepsis Surveillance Agent on patient outcomes § 2016 consensus guideline on sepsis introduced a new surveillance alert “Quick Sequential [Sepsis-related] Organ Failure Assessment” (q. SOFA) AMIA 2017 | amia. org 4

METHODS AMIA 2017 | amia. org 5

Project setting § Eight acute care hospitals in the USA § All facilities had the Cerner Millennium EHR and cloud-based St. John Sepsis Surveillance Agent operational 24/7 § Sepsis programs were designed to improve early recognition and early intervention for patients at-risk of sepsis v The sepsis protocol was guided by the Surviving Sepsis Campaign resuscitation and management bundles. AMIA 2017 | amia. org 6

Research design § Observational cohort study Clinical Parameter SIRS § Patients w/suspected or confirmed infection • Abx given < 72 hrs. post cultures drawn • Cultures drawn < 24 hrs. post Abx given § Sepsis screening and first alert activation • q. SOFA clinical event created by merging suspected infection flag and q. SOFA flag • St. John Sepsis Surveillance Agent has tiered alert activation o Sepsis alert: 2 SIRS + 1 OD o SIRS alert: 3 SIRS criteria AMIA 2017 | amia. org Temperature CMS/NQF q. SOFA >38. 3 or <36 Heart Rate >90 Respiratory Rate >20 ≥ 22 >12 or <4, or >10% bands <90 or MAP < 65 ≤ 100 WBC Cardiovascular SPB Metabolic Lactic Acid >2. 0 Hepatic Total bilirubin ≥ 2. 0 Renal Serum creatinine Coagulation Platelets INR or a. PTT Neurologic GCS score ≥ 2. 0 <100, 000 >1. 5 or >60 sec. <15 7

Sample selection § Admitted inpatient in 2016 Q 1 and discharged from hospital § Age ≥ 18 years § Study population Ø 5, 992 (35%) patients with suspected or confirmed infection AMIA 2017 | amia. org 8

Data collection procedures § Retrospective § Millennium EHR • Patient registration • Patient location history • Clinical events (ie, VS, lab results) • Microbiology • Medication Administration Record § St. John Sepsis Surveillance Agent AMIA 2017 | amia. org 9

Quantitative measures § Primary outcome: in-hospital mortality § Secondary outcome: composite of in-hospital mortality or ICU admission § Time to event • onset of suspected infection to alert activation • alert activation to onset of suspected infection • alert activation to the primary or secondary outcome AMIA 2017 | amia. org 10

Data analysis § Hierarchical multivariable logistic regression to estimate effects of model risk factors on primary and secondary outcomes • Baseline model: [block 1] age, male sex, and recent discharge, and [block 2] onset of infection in ED and admitting facility • q. SOFA + baseline model • St. John Sepsis Surveillance Agent + baseline model § Kaplan-Meier survival model framework to examine the proportion of patients experiencing the composite outcome in one hour increments after detection by alert type AMIA 2017 | amia. org 11

RESULTS AMIA 2017 | amia. org 12

Sample characteristics § 5, 992 patients with suspected or confirmed infection § Demographics • median age 65, IQR = 51 to 77 years • 52% female sex • 11% recently discharged from hospital (< 30 days) • 64% onset of infection occurred in ED § Outcomes • 4% in-hospital mortality • 18% composite in-hospital mortality or ICU admission AMIA 2017 | amia. org 13

Hours from alert to onset of infection q. SOFA § 1 in 9 (11%) patients post onset of infection • median 4. 8 hours (IQR = 2. 0 to 11. 2 hours) St. John Sepsis Surveillance Agent § 1 in 10 (10%) patients before onset of infection • median 1. 1 hours (IQR = 0. 4 to 4. 0 hours) § 1 in 4 (27%) patients post onset of infection • median 1. 0 hours (IQR = 0. 6 to 3. 6 hours) AMIA 2017 | amia. org 14

Adjusted odds ratios of patient outcomes In-hospital mortality or ICU admission In-hospital mortality 2. 0 AMIA 2017 | amia. org 4. 0 2. 0 4. 0 15

Model discrimination on patient outcomes Mortality outcome § q. SOFA model reclassified 2% patients; a relative ∆↑ 9% sensitivity § St. John Sepsis Agent model reclassified 15% patients; a relative ∆↑ 23% sensitivity Composite outcome § q. SOFA model reclassified 5% patients; a relative ∆↑ 4% sensitivity § St. John Sepsis Agent model reclassified 13% patients; a relative ∆↑ 31% sensitivity AMIA 2017 | amia. org 16

Timing of alert to composite outcome by cohort Cohort: St. John Sepsis Agent “Sepsis alert” § 33% (n = 306 of 930) with composite outcome § median 5. 7 (IQR = 4. 9 to 6. 6) hours after detection § 6% censored cases Cohort: q. SOFA clinical event § 25% (n = 163 of 649) with composite outcome § median 5. 3 (IQR = 3. 5 to 7. 1) hours after detection § 12% censored cases Cohort: St. John Sepsis Agent “SIRS alert” § 22% (n = 292 of 1, 298) with composite outcome § median 6. 4 (IQR = 4. 7 to 8. 0) hours after detection § 13% censored cases AMIA 2017 | amia. org Censored ≥ 96. 0 hours 17

DISCUSSION AMIA 2017 | amia. org 18

Key findings Measure Finding Incidence of St. John Sepsis Surveillance Agent was three-fold higher than q. SOFA among patients with suspected or confirmed infection Temporal relationship Patients who activated the St. John Sepsis Agent after onset of infection did so much earlier than q. SOFA, and a sizeable proportion of patients triggered the Agent prior to the onset of infection Clinimetric performance St. John Sepsis Surveillance Agent model was more accurate in predicting mortality and ICU admission outcomes than q. SOFA, resulting in greater reclassification benefit from baseline risk AMIA 2017 | amia. org 19

Conclusion § 2012 consensus definition of sepsis was more robust for surveillance applications when compared to 2016 q. SOFA • More patients at-risk of sepsis were screened-in • Patients were identified earlier in their infectious process • Greater precision in predicting adverse outcomes § Considering consequences of missing patients or introducing avoidable delay, q. SOFA fell far behind on these objectives § We sought balance between clinical precision of detection and timing of alert activation; the St. John Sepsis Surveillance Agent was promising in striking this balance AMIA 2017 | amia. org 21

Acknowledgements A special thank you to Dr. Bharat B. Sutariya, co-author of this study, who has provided focused leadership by taking Neal Patterson’s vision of sepsis surveillance and making it a reality. AMIA 2017 | amia. org 22

References Levy MM, Fink MP, Marshall JC, et al. International Sepsis Definitions Conference. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Intensive Care Med. 2003; 29(4): 530 -538. Dellinger RP, Levy MM, Rhodes A, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013; 41: 580 -637. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016; 315(8): 801– 810. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016; 315(8): 762 -774. Amland RC, Hahn-Cover KE. Clinical decision support for early recognition of sepsis. Am J Med Qual. 2016; 31(2): 103110. Amland RC, Sutariya BB. Quick Sequential [Sepsis-related] Organ Failure Assessment (q. SOFA) and St. John Sepsis Surveillance Agent to detect patients at-risk of sepsis: an observational cohort study. Am J Med Qual. Published online ahead of print on February 1, 2017. AMIA 2017 | amia. org 23

Question Early recognition and treatment of sepsis shows promise in improving patient outcomes. Thus, health systems seek to identify patients with suspicion of sepsis through implementation of high-reliable surveillance and clinical decision support systems. A new definition for sepsis has been recommended, which challenges providers familiar with current early warning tools. Assuming each of the following options are available when configuring a sepsis surveillance system for provider adoption and response, which decision is the most appropriate toward improving early identification of patients with sepsis? AMIA 2017 | amia. org 24

Possible answers A. Clinical indications of sepsis may detect many deteriorating patients early in their infectious process, not all of whom will be septic and realizing a trade-off in false-positive sepsis rule-in decision. B. Clinimetric performance of surveillance systems using historical clinical data repositories provides an accurate depiction of current clinical processes and workflow. C. Threshold criteria for clinical indicators responsible for triggering a sepsis surveillance alert should be increased to achieve high specificity for sepsis rule-in, realizing this tradeoff increases false-negatives. D. To help clinicians at bedside in recognizing sepsis syndrome among patients with known suspicion of serious infection, only a few key clinical indicators are necessary for inclusion in surveillance system, despite potential for introducing delay in detection, missing affected organ failure, and reducing compliance of protocols. AMIA 2017 | amia. org 25

Correct answer (A) Clinical indications of sepsis may detect many deteriorating patients early in their infectious process, not all of whom will be septic, and realizing a tradeoff in false-positive sepsis rule-in decision. Explanation: Surveillance systems optimize the tradeoff between sensitivity and specificity, while considering timeliness of alert notifications. Systems that tilt toward higher sensitivity incorporate online real-time verification of alert data within the patient's electronic health record. Provider adoption of the surveillance system and obtaining 90% response to surveillance alert notifications occurs when the system's clinical criteria and definitions adhere closely to consensus and guidelines, and demonstrated alarm reliability ≥ 70% positive predictive value for rule-in. AMIA 2017 | amia. org 26

AMIA is the professional home for more than 5, 400 informatics professionals, representing frontline clinicians, researchers, public health experts and educators who bring meaning to data, manage information and generate new knowledge across the research and healthcare enterprise. AMIA 2017 | amia. org @AMIAInformatics @AMIAinformatics Official Group of AMIA @AMIAInformatics #Why. Informatics 27

Thank you! Email me at: bamland@cerner. com