Learning Objectives Describe the use of the Outcomes

Learning Objectives • Describe the use of the Outcomes Impact Analysis model to assess economic impact of a CME activity • Discuss the potential role of Outcomes Impact Analysis modeling in CME practice 2

What We’ve Been Hearing… • “How do I convey the value of CME to non-CME administrators? ” • “I need to summarize an entire year of outcomes in a condensed format. ” • “I’ve been asked to justify my continuing medical education budget. ” • “How can I compare different outcomes from a number of activities? ” 3

The Problems Confronting CME • Funding and resources are decreasing – The core message of the value of CME is not reaching internal and external stakeholders, policy makers, and society – There are few resources for professionals to demonstrate the return on CME investment to nonmedical or noneducator audiences 4

Solutions • Demonstrating the benefits of CME depends on effective communication • Economic data provide a universally understood outcome measure—monetary impact • Economic modeling of CME is a rapid, cost effective, and repeatable method of predicting cost impact 5

Outcomes Impact Analysis

Outcomes Impact Analysis: A Novel Analytic Framework • Evaluating the economic impact of CME outcomes is based on the established principles of health economic modeling, decision analysis, probability theory, and statistical analysis • No single existing economic analysis model is ideally suited for assessing CME outcomes 7

Blood Conservation Symposium: Economic Implications • CME symposium was attended by 133 participants • Reviewed strategies for blood conservation and implementation of the Society of Thoracic Surgeons Blood Conservation Clinical Practice Guidelines • 93. 8% were committed to change their practice based on what they learned • Major barriers to blood conservation cited were cost, administration buy-in, and inertia 8

Evaluating the Economic Impact • Outcomes impact analysis was used to estimate the potential economic impact of applying CME-related learning to clinical practice for cardiac surgical outcomes: – Prevention of bleeding-related complications – Prevention of reoperation for bleeding in coronary artery bypass graft (CABG) surgery • Costs averted were estimated from the perspective of the healthcare provider 9

Models and Parameters Bleeding-related complications (BRC) • A study of 103, 826 cardiac operations showed that mean hospital costs were $12, 128 higher in patients with a BRC than in those without a BRC; the additive costs from 142, 533 thoracic operations was $15, 8991 Reoperation for bleeding in CABG • Nineteen studies evaluated additive costs of reoperation for bleeding following CABG (n=755, 382); weighted mean was $24, 0482 -4 Assumptions • Operative volume – The STS/AATS Workforce Report showed that adult cardiac surgeons (n=1211) performed a mean of 155 cardiac operations and 214 thoracic operations in 20095 1. Stokes ME et al. BMC Health Serv Res. 2011; 11: 135 -148. 2. Brown PP et al. Ann Thorac Surg. 2008; 85: 1980 -1987. 3. Herwaldt LA et al. Infect Control Hosp Epidemiol. 2003; 24: 44 -50. 4. Speir AM et al. Ann Thorac Surg. 2008; 85: 1980 -1987. 5. Shemin RJ et al. Ann Thorac Surg. 2012; 93: 348 -355. 10

Base Case Analysis • 3 in 10 participants changed clinical practices as a result of CME learning in such a way as to prevent the outcome of interest (BRC or reoperation) • Resulted in prevention of BRC and RFB in 2% and 1. 5% of operations, respectively • One year time period 11

Sensitivity Analysis • One-way sensitivity analysis evaluated the potential savings for the base case when the proportion of participants who prevented outcomes of interest varied from 1 in ten to 5 in ten • Probabilistic sensitivity analysis with second-order Monte Carlo simulation was used to evaluate robustness of the estimate to parameter uncertainty and to calculate a mean and confidence interval for the estimated value Adjustments • Estimated values were standardized to 2012 $US using the medical care component of the Consumer Price Index 12

Results Model Estimated Value ($US 2012) Base Case PSA Mean (95% CI) Bleeding-related complications, cardiac 1, 500, 112 1, 502, 769 (869, 860– 2, 359, 068) Bleeding-related complications, thoracic 2, 715, 104 2, 715, 246 (1, 590, 308 -4, 217, 092) Reoperation for bleeding, CABG 2, 230, 873 Ravyn D et al. J Cont Educ Health Prof. 2014; 34(S 1): S 41 -S 46. 2, 233, 988 (1, 223, 901– 3, 648, 719) 13

Dynamic Estimator 14

Outcomes Impact Analysis Strengths Limitations • Rapid, cost effective, and highly versatile • Can be used in funding decision making or to evaluate outcomes after an activity • Cost data may be unavailable or unreliable • Estimates of the monetary return on education • Does not provide direct evidence of patient impact; outcomes inferred from utilization • Does not require patient-level or medical record data • Serves as an adjunct to, not a replacement of educational outcomes • Independent of educational modality, audience, or therapy area • Does not usually consider indirect or intangible costs • Allows outcomes comparisons across heterogeneous activities • Limited to expenditures, does not examine quality of life • Data, scenario, assumptions, and time horizon are tailored to the decision maker’s needs • Requires expertise in therapy area, health services, outcomes, and economic modeling • Results easily understood by nonmedical, noneducator, and lay audiences • As with all economic predictive models, it may be necessary to validate estimated values after implementation • Sensitivity analysis demonstrates results across a wide range of plausible scenarios 15

Conclusions • Plausible economic estimates suggest that CMErelated learning favorably impacting practice yields substantial cost savings • Model prediction of averted costs associated with CME allows estimation of the economic impact on outcomes in the absence of patient-level outcomes data related to CME activities • This model may also be used to optimize CME resources by identifying areas of educational need most likely to result in cost savings 16

For more information contact Dana Ravyn dravyn@cmeology. org