Get Started Get Better Using Improvement Cycles for
Get Started – Get Better: Using Improvement Cycles for SSIP Caryn Ward, Ph. D Dean Fixsen, Ph. D OSEP PD Conference August 2016
Formula for Success Effective Innovations Effective Implementation Educationally Significant Outcomes Enabling Contexts
Active Implementation Frameworks ü Usable Innovations ü Implementation Stages ü Implementation Drivers ü Improvement Cycles ü Implementation Teams ü Enabling Change
IMPROVEMENT CYCLES Changing on purpose to support the new way of work
“Key Aspects of Improvement” “Many initiatives fail for lack of study and reflection on what is actually being done and what the results are from having done it. Observing, describing, and documenting are key aspects to a program improvement cycle, and particularly critical during the pilot phase when key functions of interventions are emerging. ” The Child Wellbeing Project, Improvement Cycle Tool
Improvement Cycles What can be changed and improved? How can problems be fixed? Did you do it? What happened? PDSA Cycle (Deming, 1986) Cycle Act Plan Study Do What is it you intend to do? Can you do it in practice?
Implementation of Improvement Cycles Meta-analysis N=73 Improvement Studies • Plan: 100% • Do: ? ? ? • Study: 15% • Act: 14% • Cycle: 19% PDSAC = 3% = Low Fidelity use of PDSAC Process Taylor et al. (2014)
Improvement Cycles Three critical Improvement Cycles: 1. Rapid Cycle Problem Solving 2. Usability Testing 3. Practice-Policy Communication Cycle
Rapid-Cycle Problem Solving Protocol 1. Clarify and validate the problem 2. Decide what variable(s) to problem solve 3. Set improvement target 4. Generate possible solution to improve attribute(s) Do 5. Implement a solution Study Act 6. Evaluate if solution worked 7. Continue or expand process Plan
Rapid-Cycle Problem Solving § Examples Problem Oriented: § Generating more timely reports for monitoring progress of students who receive new curriculum and instruction § Incremental Improvement: § Improving the integration of meaningful parent input into the school-wide anti-bullying program.
Improvement Cycles Usability Testing Much more is learned from 4 cycles with 5 participants in each cycle than from one pilot test with 20 participants (Nielsen, 2000)
Usability Testing (cont. )
Practice-Policy Communication Cycle Feedback Practice Informs Policy Do Policy Enables Practices Plan Study - Act External Implementation Support Policy Practice FORM SUPPORTS FUNCTION Policy Structure Procedure Practice
Improvement Cycles: Comparison Rapid-Cycle Problem Solving Usability Testing Practice-Policy Communication Cycles: System Change Focus Primary Point of Application Individual Practices Initially identifying problems and solutions Groups of practices; operating methods Testing the feasibility of solutions and developing administrative supports Modification of units and relationship between units Executive leadership and others with authority to change system units Cycle Duration Daily or weekly cycles Weekly or monthly cycles Monthly or bimonthly cycles Purposeful Use Development team and/or ad-hoc team On-going improvement team
District Scenarios • On your own, read the four district scenarios. • With a partner, answer the following questions: • What are the similarities amongst the four approaches? • What are the differences? • Which one is making intention use of improvement cycles?
Putting it all together…. Practice Policy Communication Cycle Usability Testing Rapid Cycle Problem Solving
Transformation Zone § “Vertical slice” of the system § Representative § Small enough to be manageable § Large enough to “disturb” the system and impact key aspects of the system, yet not impact the entire system
State Management Team SISEP Support State Implementation Team Regional Implementation Team District Implementation Team Building Implementation Building Implementation Teams Teams Repurpose Roles, Functions, & Structures to Maximize Outcomes District Implementation Team Building Implementation Teams
Factors to Consider in Size and Location of the TZ § How deep and significant does this change the status quo? § Do we have the necessary expertise and knowledge related to the innovation itself? § How experienced and accessible are our implementation teams? § How well resourced is the implementation effort?
Transformation Zone Components § Clearly defined program § Core Components § Indicators of Fidelity § Selection of initial implementers § Selection Criteria § Requisite variety § System components
Transformation Zone Components (cont. ) § Training § § Just in time § Just enough § New knowledge vs. new skills Data Collection § What information? § Reported to whom? § On what schedule?
Transformation Zone Components (cont. ) § Criteria for Success § Indicators for next iteration § Indicators for re-design § Program vs. system information § Next Iteration
Transformation Zone Criteria for Success § State Capacity Assessment at 80% or better on each subscale § Regional Capacity Assessment at 80% or better on each subscale § District Capacity Assessment at 80% or better on each subscale § Drivers Best Practices Assessment at 80% or better § Fidelity Assessment criteria met for 50% or more of the teachers
Within a Transformation Zone: Cohort 1: District A District B District C Cohort 2: District D District E District F Cohort 3: District G District H District I State & Regional Implementation Teams Cohort 4: District J District K District L
Scaling within a SEA Cohort 1: Transformation Zone A Cohort 2: Transformation Zone B Cohort 3: Transformation Zone C State Management & Implementation Teams Cohort 4: Transformation Zone D
Case Study In pairs: § Review the state case study and underline the critical characteristics of a transformation zone § How is it similar to your state’s SSIP respective project work? § How is it different to your state’s SSIP or respective project?
Reinvention for Impact Existing System Is Changed To Support The Effectiveness Of The Innovation Effective Innovations Are Changed to Fit The System Effective Innovation
Get Started, Get Better Purposeful = Improvable
For more information Allison Metz Allison. metz@unc. edu Caryn Ward Caryn. ward@unc. edu Frank Porter Graham Child Development Institute University of North Carolina Chapel Hill, NC http: //nirn. fpg. unc. edu/ www. scalingup. org www. implementationconference. org
Get Connected! http: //implementation. fpg. unc. edu Workgroups BIC Co. P @SISEPcenter SISEP For more on Implementation Science http: //nirn. fpg. unc. edu www. scalingup. org
Citation and Copyright This document is based on the work of the National Implementation Research Network (NIRN). © 2013 -2016 Allison Metz, Leah Bartley, Jonathan Green, Laura Louison, Sandy Naoom, Barbara Sims, and Caryn Ward This content is licensed under Creative Commons license CC BY-NC-ND, Attribution-Non. Commercial-No. Derivs. You are free to share, copy, distribute and transmit the work under the following conditions: Attribution — You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work); Noncommercial — You may not use this work for commercial purposes; No Derivative Works — You may not alter, transform, or build upon this work. Any of the above conditions can be waived if you get permission from the copyright holder. email: nirn@unc. edu web: http: //nirn. fpg. unc. edu The mission of the National Implementation Research Network (NIRN) is to contribute to the best practices and science of implementation, organization change, and system reinvention to improve outcomes across the spectrum of human services.
- Slides: 32