Making Sense of SRTR and CUSUM Objectives To

Making Sense of SRTR and CUSUM

Objectives • To be able to understand the SRTR graphs • To have an understanding of risk adjustment • To be able to understand what CUSUM is representing and the impact on my program

OVERVIEW OF SRTR

What is SRTR Scientific Registry of Transplant Recipients

Data Flow Organ Procurement Organizations Transplant Programs Histo. Compatibility Laboratories Organ Procurement and Transplantation Network (OPTN/UNOS) Data Collection Scientific Registry of Transplant Recipients (SRTR) Public Professional Community (Programs, OPOs, Etc… OPTN Membership & Professional Standards Data Analysis Payers (CMS, Transplant*Adapted from Insurance the SRTR Quality Programs Conference Companies) Presentation

Data turns into information:

Different Audiences, Different Questions: Different Statistics and Interpretations • Patients and families § What will happen to me? § Percent survival at 1 year, 3 years. § Chances of transplant or death while on the waiting list. § CMS-required consent process. • Payers (including CMS) and MPSC § Does a program perform up to standard or systematically fail to do so? • Transplant programs § What choices do our patients have? § What can we tell our patients about waiting time and survival? § How well are we doing? How can we improve?

So what is this trying to explain… 100, 00% 99, 00% 98, 00% 97, 00% 96, 00% 95, 00% 94, 00% 93, 00% 92, 00% Observed Expected US 98, 07% 97, 39% 96, 81% 98, 48% 97, 20% 96, 79% 97, 68% 96, 94% 96, 77% 98, 19% 97, 27% 96, 87% 98, 10% 97, 32% 96, 93% 98, 20% 97, 56% 97, 03% 98, 77% 97, 74% 97, 07% 98, 26% 97, 82% 97, 13% 97, 22% 97, 87% 97, 19% 98, 17% 98, 08% 97, 38% 97, 21% 97, 99% 97, 42%

What we get from SRTR…

What we get from SRTR… OBSERVED

What we get from SRTR… EXPECTED

Presented graphically at QAPI

Two Main Components: OBSERVED EXPECTED • The actual number of patients in that cohort that lived post a year transplant. • The anticipated number of patients that were supposed to live based on characteristics of – Recipients – Donors – Transplant center

Why Compare Observed and Expected Outcomes? • Allows fair comparison among programs that treat different types of patients. § Programs that treat older or sicker patients might provide excellent care even though outcomes are worse than average. § Programs that treat healthier patients might not provide excellent care even though outcomes are better than average.

So I am confused, is this all real time data? NO!

Timeline for Program-Specific Reports • Updated every 6 months (June, December). • Patient and graft survival tables report 1 -month, 1 -year, and 3 -year outcomes for 2. 5 -year cohorts of recipients.

One Year Patient Survival Actually looking at patients between 07/01/2012 -12/31/2014 WHY?

Why Don’t We Have 1 -Year Survival for the Last 6 Months of Transplants? 1. 1 -year outcomes are not available for 18 months. § One year needed to determine 1 -year survival. § Time needed for programs to submit 1 -year follow-up forms to OPTN. § Two months needed for SRTR to calculate statistics and for centers to comment. 2. Must include enough transplants to allow stable estimates; PSRs use 2. 5 years. Together, these factors require a 2. 5 -year cohort ending 12 to 18 months before the report date, with some transplants occurring as long as 3. 5 to 4 years before.

Moving on…. Risk Adjustment

Examples of Factors Used for Risk Adjustment • Recipient and donor demographic characteristics • ABO compatibility • Primary disease • Donor cause of death • Ischemia time • • • Previous transplant Life support HLA mismatch and CPRA (KI) Duration on dialysis (KI) Creatinine (LI)

Interpreting Model Coefficients Estimate Hazard Ratio = exp (Estimate) Donor age: 0 -17 yr (ref 35 -49 yr) 0. 002 1. 00 Donor age: 18 -34 yr -0. 044 0. 96 Donor age: 50 -64 yr 0. 220 1. 25 Donor age: ≥ 65 yr 0. 377 1. 46 Recipient race: Black (ref white) 0. 211 1. 23 Recipient race: Hispanic, Latino -0. 107 0. 90 Recipient race: Asian -0. 148 0. 86 Recipient race: other/missing -0. 279 0. 76 Hazard Ratio > 1, failure/death more likely, lower than expected. Hazard Ratio < 1, failure/death less likely, higher than expected.

Lightbulb Moment Risk Adjustment Our EXPECTED value

How does this impact you? Every piece of data we enter into UNOS impacts our EXPECTED VALUE and thus our outcomes!

Example: Adjusting for Age • Nationally: § Average survival, 85%. • 50% of patients are young with 95% survival. • 50% of patients are old with 75% survival. • Program A treats only older patients, 80% survival: § Program survival of 80% is worse than national average of 85%. § But, 100% are older patients with expected 75% survival. § Thus, Center A patients have better than expected survival compared with similar patients nationwide.

Risk-Adjustment Example: Expanded Criteria Kidney Donors • Will accepting ECD donors adversely affect center survival statistics? No, because: • ECD risk adjustment controls for donor factors: § Hypertension: hazard ratio 1. 14 § Creatinine, per 0. 5 mg/d. L above 1. 5: hazard ratio 1. 06 § Donor age: ≥ 65 yr (ref, 35 -49 yr): hazard ratio 1. 71 § COD stroke (vs. other COD): hazard ratio 1. 16 § ECD classification: hazard ratio 1. 01 • All adjustments are documented at www. srtr. org HRs calculated as exp(B) from 1 yr KI GS model, PSRs released 07/11/2008.

CUSUM

Purpose • Provide programs with close to real-time data • CUSUM is designed to track outcomes over time for individual programs • Doesn’t replace PSRs • CUSUM charts DO NOT go through a formal review period

WHAT are CUSUM Charts? • Quality control method used in statistical analysis • Used to detect a change in a process • Looks at performance over time (3 years time) • If there are no deaths the CUSUM line trends down…. if deaths occur the line goes up

O-E CUSUM Chart

O-E CUSUM Chart Baseline is zero, your observed was the same as your expected

O-E CUSUM Chart CUSUM line indicating trends in outcomes

O-E CUSUM Chart A period of higher than expected events

O-E CUSUM chart Always locate the zero/ baseline first A period of lower than expected events

One Sided CUSUM

One Sided CUSUM “Indicates when a program may wish to consider a formal process review”

One Sided CUSUM When the 5% threshold is hit it resets to zero Each tick marks an event

Quiz Time

Question 1: How are we trending?

Question 2: How is our program doing?

Question 3: Does this include 2015 patients? 100, 00% 99, 00% 98, 00% 97, 00% 96, 00% 95, 00% 94, 00% 93, 00% 92, 00% Observed Expected US 94, 59% 94, 50% 93, 51% 94, 91% 94, 54% 93, 55% 93, 85% 94, 27% 93, 60% 94, 84% 94, 60% 93, 81% 95, 00% 94, 08% 95, 54% 95, 23% 94, 31% 97, 12% 95, 48% 94, 52% 96, 67% 95, 70% 94, 66% 95, 33% 95, 63% 94, 70% 96, 28% 95, 56% 94, 81% 95, 14% 95, 71% 94, 96%

Questions? Need further information? http: //www. srtr. org/