Measuring Quality Improvement Brandon Bennett Improvement Science Consulting

What are we trying to accomplish? • Distinguishing types of variation, performance and the use of targets • Idea of cascading measures • Revisiting our team level dashboards • Measurement for PDSA cycles Improvement Science Consulting

Two considerations on using data • What is the process/system currently doing? • Two more considerations: • Common Causes Compare here • Special Causes Remove/return to common cause Improvement Science Consulting Institute improvement to achieve • What performance would we like? • Targets • Specifications • Aims imposed, hoped for, improved towards • Inspected out Once here, intend special cause variation through change to move toward the specification of interest

But Why? • Two Dangers for Leadership – Tampering – intervening/taking action on the process/system when performance is statistically stable – wastes resources and can make things worse – Failing to Intervene – when the process/system is indicating a need for correction/improvement, this failure can waste resources through poor management and/or lead to a more expensive fix further in the future Improvement Science Consulting

System of Feedback O 1 B O 2 B Measure Types O = Outcome Measure P = Process Measure P 1 S 1 P 2 S 2 P 1 S 3 P 2 S 1 S 2 B = Balance Measure S 3 S = Process Step Measure = Learning Cycle Measure PDSA PDSA 4 PDSA 3 2 1 Improvement Science Consulting PDSA 1 2 4 3 © Improvement Science Consulting

System of Feedback (Hip Fracture) O - Mortality O – Patient Satisfaction B B Measure Types O = Outcome Measure P = Process Measure P - ALOS S 1 P 1 S 2 P 2 S – Time to Theatre S 1 S 2 B = Balance Measure S 3 S = Process Step Measure = Learning Cycle Measure PDSA PDSA 4 PDSA 3 2 1 Improvement Science Consulting PDSA 1 2 4 3 © Improvement Science Consulting

Break Out Exercise • Review your team level dashboard • Are you aimed too high? With too much emphasis on the outcome or process measures of interest? • If so, what are the steps in your process (Process Map) • What Process Step Measures could be developed to help you understand performance closer to the frontline of care? • Do you collect that information currently? How could you collect and use that information at the frontline? • i. e. CAM tool used reliably on all patients (CAM tooled used/All Admissions to ward 4 reported daily/weekly on a run/control chart) Improvement Science Consulting

Improvement Science Consulting

As the scale of the test increases we move from qualitative to quantitative evidence Sequence of learning and change Evidence primarily Qualitative Very small scale test of a change idea Evidence primarily Quantitative with noticeable impact on process measures Large scale test of change idea or Implementation of a change idea © Improvement Science Consulting

Break Out Exercise • Revisit one Change Idea you have for improving care in your process area • See if you can develop measures that will help you learn about the Change Idea’s operational efficacy (increase degree of belief) starting with a test of change focused at the smallest PDSA level (i. e. 1 patient, 1 nurse, 1 day) and moving through 2 or three steps to the Process Step Measure Level • Note: The learning questions and individual PDSA measures should evolve with each cycle, if nothing else than to reflect the increased scale of the cycle Improvement Science Consulting

Break Out Exercise • Revisit the Process Step Measures you identified earlier • How would your team collect the information identified? • What would a data collection tool look like for these measures? • Where and how would you display the information collected for your team? Improvement Science Consulting

Model for Improvement What are we trying to accomplish? What change can we make that will result in improvement? How will we know that a change is an improvement? Act Plan Study Do Improvement Science Consulting Langley, et al.