Building Evaluation Capacity for Prescription Opioid Overdose Prevention

Building Evaluation Capacity for Prescription Opioid Overdose Prevention: Lessons From a Multi-state Initiative Session Chair: Sarah Bacon, Ph. D Presenters: Kari Cruz, MPH, Bradley Biggers, MPH, Emilia Pasalic, MPH, and Aleta Christensen, MPH Presentation to the American Evaluation Association Annual Conference October 29, 2016

500, 000 drug overdose deaths since 2000 4 x as many opioid deaths in 2014 as 1999

For every Rx opioid overdose death in 2011, there were. . . 12 treatment admissions for opioids 25 emergency department visits for opioids 105 people who abused or were dependent on opioids 659 nonmedical opioid users 0 100 SAMHSA NSDUH, DAWN, TEDS data sets. 200 300 400 500 600 700

Overdose Prevention In States (OPIS) • Three competitive cooperative agreements, all to state health departments or their bona fide agents § Prevention for States § Data Driven Prevention Initiative § Enhanced Surveillance for Opioid Morbidity and Mortality

Ø Move toward universal PDMP registration and use Ø Make PDMPs easier to use and access Ø Move toward a real-time PDMP Ø Expand improve proactive reporting Ø Conduct public health surveillance with PDMP Ø Implement or improve opioid prescribing interventions for insurers, health systems, or pharmacy benefit managers. This includes: Ø 1 2 Enhance and Maximize PDMPs Community or Health System Interventions Prior authorization, prescribing rules, academic detailing, CCPs, PRRs, Ø Enhance adoption of opioid prescribing guidelines Prevention for States Program Rapid Response Projects Ø Allow states to move on quick, flexible projects to respond to changing circumstances on the ground and move fast to capitalize on new prevention opportunities. 4 State Policy Evaluation 3 Ø Build evidence base for policy prevention strategies that work like pain clinic laws and regulations, or naloxone access laws

Evaluation Capacity Building: Tools for State Partners National Center for Injury Prevention and Control Division of Unintentional Injury Prevention

Evaluation Officers

Current Funded States WA MT ME ND MN OR ID WY UT IA AZ PA IL CO NY MI NE NV CA WI SD KS OK NM MO DC WV VA DE NC MA KY TN MD AR SC MS TX CT OH IN AL GA NH NJ RI LA VT FL AK HI Funded State: Data-Driven Prevention Initiative (DDPI) Funded State: Prevention for States (Pf. S)

CDC State Support Teams Program Consultant Science Officer Evaluation Officer

State Prescription Drug Overdose Teams Program Manager Epidemiologist Evaluator

Technical Assistance Tailored assistance that is purposeful, responsive, science-based, and systematic in building their infrastructure and capacity around implementation and evaluation to improve outcomes associated with opioid misuse, abuse and overdose prevention programming.

Evaluation Tools Monitoring and Evaluation Tool (MET) Evaluation plans Indicators

Capacity Building: Monitoring and Evaluation Tool (MET) National Center for Injury Prevention and Control Division of Analysis, Research, and Practice Integration

Monitoring and Evaluation Tool (MET) Background § § Microsoft Access Database Started in 2011 Developed primarily by Deloitte Supports Prevention for States (Pf. S) and three other CDC cooperative agreements

Improve responsiveness of CDC staff in support of Pf. S awardees Identify capacity of state prescription drug overdose programs Identify emerging needs and gaps so CDC can deliver timely technical assistance

Capacity Building: Developing Evaluation Plans with State Partners National Center for Injury Prevention and Control Division of Unintentional Injury Prevention

Evaluation Plan Development Process • Template developed • States develop evaluation plans Evaluation Plans submitted by state to CDC Evaluation Plan reviewed by CDC Evaluation Officer • Review criteria • Review worksheets • Gems! National Center for Injury Prevention and Control Division of Unintentional Injury Prevention • • Feedback form Formal discussion TA COP CDC provides feedback to state

Evaluation Plan Template sections: • Project specific logic model • Detailed evaluation plan for project years one and two • Narrative description of evaluation plan for years three and four

Evaluation Plan Template Development National Center for Injury Prevention and Control Division of Unintentional Injury Prevention

Evaluation Plan Review Processes • • • Common criteria worksheets • Logic model review worksheet • Evaluation Plan Narrative review worksheet Gathering resources: Evaluation Gems! Evaluation Plan Feedback Template National Center for Injury Prevention and Control Division of Unintentional Injury Prevention

Logic Model Review Worksheet National Center for Injury Prevention and Control Division of Unintentional Injury Prevention

Evaluation Gems! An internal tracking spreadsheet prompts Evaluation Officers to document: • Details of exemplary evaluation plans • Resources received from the state • Additional Comments

Evaluation Plan Feedback Template

Evaluation Profiles

Lessons Learned • Both reviewers and reviewees appreciate clear criteria! • Limit the amount of mandatory forms • Systematic approach = Proactive technical assistance

Capacity Building: Indicator Development National Center for Injury Prevention and Control Division of Unintentional Injury Prevention

Evaluation Capacity and Data Use § Data Sources: • Vital statistics death data • Emergency department data • Hospital discharge data • Prescription drug monitoring program (PDMP) data

Developing the Indicators – Challenges • Definition • How to calculate (numerator & denominator) • Limitations

Prevention for States Indicator Support Toolkit Contents: 1. Background, Introduction, and Purpose 2. Indicator Considerations 3. Preparing the datasets 4. Indicator quick guide 5. One-pagers for each indicator 6. Sample SAS code

Developing the Indicators – Engaging Stakeholders

Indicator and Toolkit Pilot § 2 piloting phases • Funded state partners • ASTHO • Pilot though email and webinars

Progress and Next Steps

For more information please contact Centers for Disease Control and Prevention 1600 Clifton Road NE, Atlanta, GA 30333 Telephone: 1 -800 -CDC-INFO (232 -4636)/TTY: 1 -888 -2326348 E-mail: cdcinfo@cdc. gov Web: www. cdc. gov The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.