An Introduction to Implementation Tools to Help Build

An Introduction to Implementation Tools to Help Build Implementation Capacity Michelle A. Duda, Dean L. Fixsen, Karen A. Blase and Barbara Sims University of North Carolina at Chapel Hill SPDG Evaluators May 2012

Introductions § § § Name § What you hope to learn States you support/work with Level of the Education “system” you support (State, Region, District, Building) SISEP 2012

SISEP Intentions § § Develop implementation capacity in 6 States § Establish a protocol for selecting States that are ready § Select evidence-based programs ready for scaling Evaluate the state-level application of the approach § Assess State capacity; improve outcomes for students § Extend findings to additional States § Disseminate findings § § Work with RRC and TA Centers Develop Communities of Practice Make presentations and write papers Develop web site and share tools

OBJECTIVES I. Introduce tools to help support implementation capacity development a) State Capacity Assessment b) Stages of Implementation Analysis c) Stage-Based Implementation Driver Analysis II. A deeper review of a selected tool a) Explore how it leads to developing Action Plans b) Discuss Administration and Implications c) Next Steps

Active Implementation Frameworks ü Implementation Teams ü Implementation Stages ü Implementation Drivers ü Improvement Cycles

Active Implementation Frameworks § Successful implementation on a useful scale requires. . . Ø Organized, expert assistance – “IMPLEMENTATION TEAMS” Ø Purposeful matching of critical implementation activities to the stage of the process – “STAGES OF IMPLEMENTATION” Ø Active use of implementation core components “best practices”– “IMPLEMENTATION DRIVERS” Ø A focus on continuous, purposeful improvement – “IMPROVEMENT PROCESSES” SISEP 2012

“IMPLEMENTATION DRIVERS” Common features of successful supports to help make full and effective uses of a wide variety of innovations

Consistent Use of Educational Innovations tat ion Dr ive rs Reliable Student Benefits Im ple me n Performance Assessment (fidelity) rs ive Dr nc y rs Co ive Dr ete Facilitative Administration n Integrated & Compensatory io t za mp Systems Intervention ni Selection ga Training Or Coaching Interventions Meet Implementation Decision Support Data System Leadership Drivers Technical SISEP 2012 Adaptive © Fixsen & Blase, 2008

Stage–Based Assessments/ Stage. Based TA • • The Exploration Stage Installation Stage Initial Implementation Full Implementation

Stages of Implementation Analysis § Purpose § Help Team plan for and/or assess the use of stagebased activities to improve the success of implementation efforts for EBPs or evidence-informed innovations (action planning/anticipatory guidance) § The tool can be used to Self-Assess current stage related activities (e. g. “We are in the midst of Exploration”) or past efforts related to a stage (e. g. “We just completed most of Installation? How did we do? What did we miss? ) (manage expectations)

Administration of the Stages of Implementation Analysis § Define the desired function of the tool in advance (assess current status? Action planning? ) § For self-assessment Implementation lead or Implementation Team completes entire assessment to achieve “strength of stage” score for each stage of Implementation §For Items “Initiated or Partially in Place” and “Not Yet Initiated” Action Plans can be developed to determine next steps or what needs to be revisited §Usually used during exploration stage, but can be used thorough out all stages to check back or when implementation dips occur (change in leadership, staff turnover, etc. )

Using the Implementation Drivers ive Dr ten cy mp e Co ers riv n. D Integrated & Compensatory tio a niz ga Or ü What infrastructure exists? ü How implementation-informed is it? ü What additional infrastructure is needed? ü What are the “next right steps” to improve the infrastructure supports? rs Interviews and Self-Assessments to analyze the implementation drivers for action planning: Leadership Drivers

Implementation Drivers Tools § High level tool to get State, Districts, or Buildings thinking about § The functions § Who be accountable for them? § How can they be improved to better support implementation?

Tools to Help Build Capacity Stage of Implementation Assessments Exploration Assessment of Implementation Stages Imple. Map Installation Stage Assessment Installation Stage Action Planning Guide Initial Implementation Component Assessment Initial Implementation Action Planning Guide Full Implementation Component Assessment Implementation Tracker Full Implementation

Exploration: Imple. Map § Purpose: § To assess and understand “the implementation landscape” § A data collection tool to assess Implementation drivers and implementation best practices § Pre-Requisites § Interviewer needs to have expertise in Implementation Science § Site has a history of successful (or not) use of evidence-based or evidence informed practices

Administration of the Imple. Map § Semi-structured interview lead by individuals fluent in Implementation Science or has deep knowledge of the Implementation Drivers § Participants include individuals who are immersed in helping building staff use innovations and leadership involved in selecting and sustaining these innovations § Select more than one EBP to discuss so that themes can be identified SISEP 2012

Imple. Map Administration INTERVENTION Enter the name of each intervention provided by the respondent. You may know the intervention by another name, but record the name used in this provider organization. Ask questions to get information about the vetting process. SISEP 2012 WHAT Ask about the “core intervention components” as they are described by the respondent. Core intervention components are the critical functions that define an intervention. HOW Ask about the Implementation Drivers are components related to: ►developing staff competency (selection, training, coaching, performance assessments); ►organization supports (decision support data systems, facilitative administration, systems interventions); and ►leadership supports (technical and adaptive). WHO Ask about the person accountable for providing each Implementation Driver. Record the name, position, and physical location of each person.

Installation & Initial Implementation Stage Driver Component Assessments § § Purpose § Serve as a Self-Assessment for Implementation Infrastructure for Evidence-based Practices Identified § Help Teams Create an Implementation Action Plan for installing Driver Best Practice Components Pre-Requisites § Implementation Team or “Accountability/Leadership” Team that can achieve action items that will be identified § Clearly Defined Evidence-based Practice Components

Administration of the Installation Stage Assessment § Facilitated by Implementation Specialist or Member of Implementation Team §May choose to review 1 – 2 Drivers/per meeting §For components identified as “Fully In Place” provide a clear description or “evidence” that this component is transparent §Score components to help prioritize Action Items §Outcome: Implementation Action Plan

Scoring key § All Implementation best-practice components scored using the following format. - All dimensions of element adhered to and evidence available to support this Fully In Place 2 points Partially In Place 1 point - Some dimensions of element adhered to and/or some dimensions attended to - Action Planning occurs with these elements Not in Place 0 points - Element not adhered to Action Planning occurs with these elements or teams may not be developmentally ready to build in this component

Installation Stage: Self-Assessment (3 districts) Information for Action Planning To what extent are best practices being used? Academics Behavior In Place Partially in Place Not in Place Recruitment and Selection Items 1% 28% 70% 7% 40% 50% Training Items 3% 39% 57% 21% 40% 38% 3% 26% 64% 12% 32% 52% 0% 33% 61% 15% 43% 40% 8% 44% 41% 20% 46% 28% Supervision/ Coaching Items Performance Assessment/Fidelity Leadership Items

School District #1 Assessment of District Capacity for Implementation Subscale Scores : 2012, March

School District #2 Assessment of District Capacity for Implementation Subscale Scores : 2012, January 100% % of Items in Place 80% 60% 40% 20% 0% Perfomance Assessment Selection Training Coaching DSDS Facilitative Systems Adm. Support Intervention

For More Information § Bertram, R. M. , Suter, J. C. , Bruns, E. J. , & O'Rourke, K. C (2010). Building a research agenda: Implementation research and wraparound literature. Journal of Child and Family Studies. 20: 6. § Ogden, T. , Bjørnebekk, G. , Kjøbli, J. , Patras, J. , Christiansen, T. , Taraldsen, K. , & Tollefsen, N. (2012). Measurement of Implementation components ten years after a nationwide introduction of empiraclly suported programs-a pilot study. Implementation Science. 7: 49. SISEP 2012

Benefits of Driver-Based Planning § § § Infrastructure needed becomes visible to all § Resources can be aligned and re-purposed to improve implementation Strengths and progress get celebrated Next right steps are planned and results measured

Stay Connected! www. scalingup. org @SISEPcenter SISEP For more on Implementation Science http: //nirn. fpg. unc. edu www. implementationconference. org

Implementation Science Implementation Research: A Synthesis of the Literature Fixsen, D. L. , Naoom, S. F. , Blase, K. A. , Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). HTTP: //NIRN. FPG. UNC. EDU