Transforming Implementation Improvement Into Science A skills building

Transforming Implementation & Improvement Into Science: A skills building series January 25, 2018

Engage with CIIS Guide & Innovate • Provide guidance, support & innovation to design projects that rigorously evaluate the effectiveness of efforts to implement change Accelerate & Promote Sustainability • Identify strategies that accelerate the adoption & promote sustainability of effective healthcare interventions Educate • Provide implementation & improvement sciences education to faculty, trainees, students

Overview: Implementation & Improvement Sciences Implementation Science Focuses on optimal strategies to promote evidence uptake in realworld settings Addresses Did stakeholders perform the desired endeavor? Why or why not? How well? Aims Translate research intro practice Systematically implement evidence-based practices Improvement Science Focuses on rigorously measuring outcomes associated with efforts to improve care delivery Addresses Did the new endeavor measurably improve desired outcomes? Improve the quality of healthcare

Identifying High-Quality Projects Main NIH Criteria Overall impact Significance Innovation Approach Investigator team Research environment Issues Applying NIH Criteria to Implementation & Improvement (IIS) Broad, non-specific to IIS Criteria could be operationalized to better describe high-quality IIS

Proctor’s 10 Key Ingredients NIH Criteria Evidence-based treatment Significance; Innovation Conceptual model, theoretical justification Approach; Innovation Stakeholder priorities, engagement in change Impact; Approach; Research Environment Setting’s readiness to adopt new services Impact; Approach; Environment Implementation strategy/process Impact; Significance; Innovation Team experience with the setting, treatment, processes Approach; Investigator Team Feasibility of proposed research design Approach; Investigator Team Measurement & analysis section Approach; Investigator Team Policy/funding environment; support for sustaining change Impact; Significance Proctor’s 10 Key Ingredients for Implementation Research Proposals Quality/care gap Impact; Significance Source: Writing Implementation Research Grant Proposals: Ten Key Ingredients. Proctor EK, Powell BJ, Baumann AA, Hamilton AM, Santerns RL. Implementation Science. 2012; 7: 96.

Upcoming Sessions Tentative Date Session Title Proposal Areas Addressed 10/25/2017 Identifying Your Implementation & Improvement Sciences Research Question Quality/Care Gap, Evidence. Based Practice 12/6/2017 Using & Discussing Implementation Science Models Conceptual Model 1/25/2018 Implementation Strategies Versus Study Interventions Implementation Strategy 2/28/2018 Designing an Implementation & Improvement Sciences Study Design 3/22/2018 Designing Your Implementation & Improvement Sciences Study Design [PRACTICE-BASED SESSION] 4/18/2018 Measuring Implementation & Improvement Outcomes, Quantitative & Qualitative Methods 5/10/2018 Engaging with Stakeholders to Conduct Feasible & Meaningful Research Stakeholder Engagement, Feasibility, Team, Policy Environment

Series Goal Proctor’s 10 Key Ingredients CIIS Educational Series High-Quality Implementation & Improvement Sciences Significant Contributions to Improve Care, Advance Fields

Implementation Strategies vs. Interventions Differentiate between implementation strategies & intervention components Discuss how to link implementation strategies to study aims, setting, process & outcome measures Provide examples of how to navigate challenges associated with chosen implementation strategies

Implementation Strategies vs. Study Interventions Megan Mc. Cullough, Ph. D Research Health Scientist Center for Healthcare Organization & Implementation Research (CHOIR) ENRM Veterans Hospital

VA Disclaimer The views expressed in this presentation are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the U. S. government.

What is an Evidence-Based Intervention? Interventions with proven efficacy and effectiveness at improving health behaviors, health outcomes, or healthrelated environments Intervention could include programs, practices, policies & guidelines Rabin, B. A. , Brownson, R. C. , Haire-Joshu, D. , Kreuter, M. W. , & Weaver, N. L. (2008). A glossary for dissemination and implementation research in health. Journal of Public Health Management and Practice, 14(2), 117 -123. ; Brown CH, Curran G, Palinkas LA, Aarons GA< Wells KB, Jones L. , et al. An overview of research and evaluation designs for dissemination and implementation. Annu Rev Public Health, . 2017; 38: 1 -22

Implementation Research/Strategies Implementation research: scientific study of the use of strategies to adopt & integrate evidence-based health interventions into clinical and community settings in order to improve patient/population outcomes “Methods or techniques used to enhance adoption, implementation, & sustainability of a clinical program or practice” (Proctor et al. , 2013) (Proctor et al. , 2009) Proctor EK, Powell BJ, Mc. Millen JC. Implementation Strategies: Recommendations for Specifying and Reporting. Implementation Science. 2013; 1(8): 139. ; Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation Research in Mental Health Services: An Emerging Science with Conceptual, Methodological, and Training challenges. Administration and Policy in Mental Health and Mental Health Services Research. 2009; 36(1): 24 -34. doi: 10. 1007/s 10488 -008 -0197 -4.

Recommendations for Specifying & Reporting § The actor (who enacts the strategy) • Name the strategy • Define the strategy • Specify the strategy § The action (use active verb statement to specify the specific actions, steps, or processes that need to be enacted) § Action target (specify targets and identify units of analysis) § Temporality (when the strategy is used) § Dosage § Implementation outcome affected § Justification Proctor EK, Powell BJ, Mc. Millen JC. Implementation Strategies: Recommendations for Specifying and Reporting. Implementation Sicence. 2013; 1(8): 139.

Questions? Thoughts? Discussion?

Operationalizing an Example: The Anticoagulation Clinic Improvement Initiative (ACCII) Hybrid Study Type 2

ACCII Study: Overview Use evidence-based dosing algorithm Algorithm linked to improve percent time in therapeutic range (TTR) Higher TTR = improved patient outcomes Use a dashboard to measure site-level TTR & related process measures Utilize improvement approaches to increase standardization across facilities Appoint an ACC Coordinators (internal facilitators) from each site to facilitate change

Promoting Action on Research Implementation in Health Services (PARIHS) Framework Evidence: Stakeholder’s perception of validity of intervention & EBP algorithm Facilitation: Concerned with enabling & supporting the implementation of an intervention through deployment of EF & IF Implementatio n Context: Environment where ACCII is introduced: may be more or less receptive to change

ACCII Study: Bundle of Implementation Strategies How we got the intervention implemented (Powell et al. , 2015) • Develop a formal implementation blueprint • Assess for readiness • Blended facilitation (focus today) • Stakeholder engagement • Plan Do Study Act (PDSA) cycles • Educational outreach visits • • Local needs assessment Ongoing training Create a learning collaborative Inform local opinion leaders Ongoing consultation Tailor strategies Visit other sites Powell BJ, Walz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kircher JE. A Refined Compilation of Implementation Strategies: Results from the Expert Recommendations for Implementing Change (ERIC) Project. Implementation Science. 2015; 10: 21.

ACCII Study: Methods Semi-Structured Interviews • Annual interviews over 4 years with Internal Facilitators (ACC coordinators), all ACC staff at 8 VA hospitals • Interviews with External Facilitation Team • 2 sets of interview with pharmacy leadership Qualitative • Deductive approach using PARIHS framework & emergent thematic analysis (inductive) • Assessed/measured implementation outcomes Quantitative • Interrupted time series to measure site & VISN level TTR along with key process measures (outcome measures)

ACCII Study: Participants Year 1 2 3 4 Total # Internal # Main Study Facilitator Interviews 67 11 63 12 67 12 57 12 254 47 # External Facilitator Interviews 5 5

ACCII Study: Intervention & Strategy Exploration of intervention & strategy together ACCII example: blended facilitation

ACCII Study: External Facilitation Strategies Organize face-to-face meeting Facilitate bi-monthly calls for quarterly for ACC coordinators Introduced ACC coordinators to: Facilitate ACC journal club LEAN Process mapping Annual site visits with focused PDSAs & other quality improvement work improvement skills Implementation strategies Enlist stakeholders to support Work 1: 1 with ACCII coordinators Draft policy documents & Audit & feedback for ACC shepherd policy proposals coordinators through committees for approval

ACCII Study: Internal Facilitation Strategies Organize face-to-face meeting Work 1: 1 with external quarterly to learn: facilitation staff Quality improvement Site visit to Portland, OR to approaches visit top performing VA ACC Process mapping & evaluation Facilitating implementation of Group activities to standardize & centralize ACCII at home site Twice monthly ACC Serving as resource/support Coordinator’s calls for frontline staff during ACCII

ACCII Study: Implementation Outcomes What are the challenges? How have you measured these in your projects?

Uptake of Intervention ACCII Clinical. Study: Uptake of Intervention Uptake Levels Intervention High Medium Low ACC Dosing Algorithm is implemented & used a high percentage of the time by all staff Algorithm is inconsistently used among staff Algorithm is rarely used among all staff Dashboard is used not only to measure performance, but as a tool for targeting patients needing more attention Most or all staff increasing use of dashboard Dashboard is used to measure performance & only 1 -2 features used inconsistently as a tool Dashboard used rarely to measure performance & rarely or not at all as a tool Often only select staff use the dashboard with little or no general usage Little to no general usage & little or no select staff usage TTR improves TTR shows some improvement TTR shows little improvement Time in Therapeutic Range (TTR) Mc. Cullough MB, Gillespie C, Petrakis BA, Jones, EA, Park AM, Lukas CVD, Rose AJ. Forming and Activating an Internal Facilitation Group for Successful Implementation: A Qualitative Study. Research in Social and Administrative Pharmacy. 2017: 1 -14.

Implementation Outcomes ACCII Study: Participation in Uptake of Intervention Activities: High Uptake Levels Medium Low ACC Coordinators IF Group Site regularly sends appropriate IF to participate Site inconsistently sends IF to participate Site rarely or never sends IF to participate Journal Club (IF, staff) Staff regularly participate & contribute Staff unevenly participate & contribute Staff rarely or never participate & contribute PDSA cycles related to ACC improvement led by IF IF has staff regularly participate IF has staff inconsistently participate IF has staff rarely participate IF inconsistently elicits feedback IF rarely participate IF facilitates feedback from IF regularly elicits frontline staff to EF on feedback implementation work Site seeks out &/or accepts assistance by EF ACC Improvement Team IF & site reach out for IF & site unevenly assistance & remain in reach out for contact with EFs assistance from EFs IF & site do not reach out for assistance, does not respond to EFs offers of assistance Mc. Cullough MB, Gillespie C, Petrakis BA, Jones, EA, Park AM, Lukas CVD, Rose AJ. Forming and Activating an Internal Facilitation Group for Successful Implementation: A Qualitative Study. Research in Social and Administrative Pharmacy. 2017: 1 -14.

ACCII Study: Outcome Measures Thoughts? Discussion on importance of outcome measures

ACCII Study: Outcome Measures Rose AJ, Park A, Gillespie C, Lukas CVD, Ozonoff A et al. Results of a Regional Effort to Improve Warfarin Management. Annals of Pharmacology. 2017.

ACCII Study: Outcome Measures Rose AJ, Park A, Gillespie C, Lukas CVD, Ozonoff A et al. Results of a Regional Effort to Improve Warfarin Management. Annals of Pharmacology. 2017.

ACCII Study: Outcome Measures Abbreviations: • TTR: Time in Therapeutic Range • VISN: Veterans Integrated Service Network Rose AJ, Park A, Gillespie C, Lukas CVD, Ozonoff A et al. Results of a Regional Effort to Improve Warfarin Management. Annals of Pharmacology. 2017.

ACCII Study: Overall Results for Blended Facilitation Sites with greater improvement were distinguished by 3 factors: 1 • Full participation in internal facilitation team is key to successful implementation 2 • Being an internal facilitator and the challenges of encouraging change in frontline staff 3 • Greater support from site-level managers and superiors lead to high uptake levels

ACCII Study: Implications • Blended facilitation can serve as an effective implementation strategy • Findings can help guide researchers who seek to incorporate internal facilitator groups as part of their implementation strategies • Contributes to a growing literature regarding which implementation strategies work best and under what circumstances

Key Takeaways • Implementation strategies: Methods used to adopt, integrate evidence-based interventions • Be specific when describing strategies • Actors, actions, target, time, dose, outcome, justification • Conceptual models can help identify & map strategies to study aims, setting, outcome measures

Tell Me About Your Projects • What is your evidencebased intervention? • Framework & study design? • What strategies would you use to get the intervention into practice? • How do you determine an intervention versus a strategy? • What are the measures? • Outcome measures? • Implementation measures? • Where do you get stuck? What is a puzzle here that is hard to sort out and solve?

Upcoming Sessions Tentative Date Session Title Proposal Areas Addressed 10/25/2017 Identifying Your Implementation & Improvement Sciences Research Question Quality/Care Gap, Evidence. Based Practice 12/6/2017 Using & Discussing Implementation Science Models Conceptual Model 1/25/2018 Implementation Strategies Versus Study Interventions Implementation Strategy 2/28/2018 Designing an Implementation & Improvement Sciences Study Design 3/22/2018 Designing Your Implementation & Improvement Sciences Study Design [PRACTICE-BASED SESSION] 4/18/2018 Measuring Implementation & Improvement Outcomes, Quantitative & Qualitative Methods 5/10/2018 Engaging with Stakeholders to Conduct Feasible & Meaningful Research Stakeholder Engagement, Feasibility, Team, Policy Environment

Thank You! Contact CIIS Website: http: //sites. bu. edu/ciis/ Email: ciisinfo@bu. edu