Measurement for improvement Workshop Whakakotahi Learning Session 1
- Slides: 56
Measurement for improvement Workshop Whakakotahi Learning Session 1 Sue Wells A/Prof Quality Improvement University of Auckland
Clinician-led cycle of care improvement (Prof Ian Scott)
Clinician-led cycle of care improvement (Prof Ian Scott) How will we know a change is an improvement? What are we trying to accomplish? How will we know a change is an improvement? What changes can we make?
Clinician-led cycle of care improvement (Prof Ian Scott)
Components of a system Structure Process Culture Outcome
Categorising measures Structure Process Outcome Balancing measures- check for unanticipated consequences Measures which allow us to make inferences about the quality of health care Qualitative information Quantitative information
Why Measure? • To understand current situation • Move away from anecdote • To manage by fact- need to quantify and verify possible causes (need evidence) • To baseline current performance • Get everyone on the same page • To develop solutions – need data to determine most effective approach • To show impact of changes • To prompt need for further improvement
Example Improving patient experience: A patient requiring long term medication has complained about the difficulty about getting a prescription filled on the same day. You ask… how easy it is for our patients to request and receive a repeat prescription?
Understand your current state • You need to measure “In God we trust. All others bring data. ” Deming
Gather and review relevant data • How many scripts/week requested? • What day do they generally come in? • Morning or afternoon? • What route do they come in? • • Phone Portal Walk-in Pharmacy
Gather feedback from patients and staff © NHS Institute for Innovation and Improvement 2011
What are the major barriers and challenges to achieving same day service?
Map your current process for prescriptions © NHS Institute for Innovation and Improvement 2011
Check Sheet: Delays in getting prescriptions in last week Causes of delay in getting scripts filled Problem Tally Subtotal Doctor busy 39 Nurse busy 4 Receptionist busy 5 Patient loses script 3 Script gets lost by pharmacist 3 Script request gets lost in practice 13
Bar Chart from Check Sheet 45 40 35 30 25 20 15 10 5 0 Doctor busy nurse busy receptionist busy patient loses scripts pharmacy loses script practice loses script request
Pareto Chart- prescription delays 45 100. 00%100% 95. 50% 91. 00% 40 85. 10% 35 80% 77. 60% 30 25 60% 58. 20% 20 40% 15 10 20% 5 0 Doctor busy practice loses script request receptionist busy Count nurse busy cum percent patient loses scripts pharmacy loses script 0%
Things to think about BEFORE you collect data 1) Your question/s- well defined 2) Stratification needed? separation of data according to strata or factors that might influence care patterns • Time of day or day of week • Season • Type of order (urgent vs routine) • Big or small practice • Type of worker • Patient factors eg; severity scores patient illness 3) Type of data 4) Sampling frame (e. g. PMS), sample size, sample strategy
Understanding types of data • Continuous data- measured on a ‘continuous scale’ • Discrete- categories • Discrete- count data • Qualitative
Sampling in QI • NOT research…. . QI is simply asking “what is happening here? ” • no intention to generalise the results beyond the local setting • critical question is…. “does this sample represent the care/processes so that the team will accept results and act upon the findings? ”
Minimum sample sizes (rule of thumb) • Estimating mean • Standard deviation • Proportion • Errors • Histogram/pareto chart • Scatter graph • Run/Control chart 5 -10 25 100 sample until 5 errors 50 25 25 -30 Thornley Group Lean Six Sigma Training 2010
Simple sampling strategies • Block sampling – straight sequence in a single time frame eg consecutive patients first 2 weeks of this month • Random sampling – simple, stratified Systematic or Purposive sampling- regular selection every 10 th patient, every hour on the hour or set time of day, day of week R Lloyd 2011
Variation variation should be viewed in one of two ways special or common cause. • Common cause • Inherent part of every process • Random fluctuation • Stable, predictable or “in control”
Special Cause Variation • Indicates something has changed • Unstable or “out of control” • “systematic” change or shift from the usual process • Maybe a purposeful change • Or indicates something is not right
Run Charts & Control Charts plotting measurements of process or outcomes over time useful for understanding variation & demonstrating impact of interventions
Run charts • Allows a team to study data over specific period of time • Performance of the process/system • Used to detect shifts, trends or cycles • Measure performance before and after an intervention • X axis always time • Y-axis- whatever process/measure • Simple rules for interpretation
How do I plot/interpret a Run Chart? Median Brassard and Ritter, 1994
Run chart exercise
Prescription process • Plot % of patients who get same day prescriptions: Week Percent Week Percent 1 2 3 4 5 6 7 8 9 60% 55% 70% 66% 58% 72% 62% 75% 10 11 12 13 14 15 16 17 18 71% 66% 58% 57% 64% 55% 64% 74%
Fill in the template and join the dots Percent of patients with same day prescriptions 100% 90% 80% 70% 60% 50% 40% 1 2 3 4 5 6 7 8 9 10 week 11 12 13 14 15 16 17 18
Do you get something like this? Percent of patients with same day prescriptions 100% 90% 80% 70% 60% 50% 40% 1 2 3 4 5 6 7 8 9 10 week 11 12 13 14 15 16 17 18
Work out the median 1. Rewrite each data point (percentages) so that they are in rank order, going from highest to lowest (or lowest to highest) 2. Identify the median (middle) value (if an even number of values take the average of the 2 middle values)
Do you get something like this? Percent (in rank order, descending) 75 74 72 71 70 66 66 66 64 64 62 61 60 58 58 57 55 55 Median
Draw the median line on the graph Percent of patients with same day prescriptions 100% 90% 80% 70% 60% 50% 40% 1 2 3 4 5 6 7 8 9 10 week 11 12 13 14 15 16 17 18
Do you get something like this? Percent of patients with same day prescriptions 100% 90% 80% 70% 60% 50% 40% 1 2 3 4 5 6 7 8 9 10 week 11 12 13 14 15 16 17 18
Before you can apply the 4 rules • Count the number of “useful observations” • Count the number of “runs”
Useful observations • All observations apart from any that fall on the median
How many useful observations? Percent of patients with same day prescriptions 100% 90% 80% 70% 60% 50% 40% 1 2 3 4 5 6 7 8 9 10 week 11 12 13 14 15 16 17 18
How many useful observations did you get? Percent of patients with same day prescriptions 100% 90% 80% 70% 60% 50% 40% 1 2 3 4 5 6 7 8 9 10 week 11 12 13 Two points on the line have same value as median Useful observations = 18 - 2 =16 14 15 16 17 18
Runs • >1 data points on same side of median • Exclude data on the median
How many runs? Percent of patients with same day prescriptions 100% 90% 80% 70% 60% 50% 40% 1 2 3 4 5 6 7 8 9 10 week 11 12 13 14 15 16 17 18
How many runs did you get? Percent of patients with same day prescriptions 100% 90% 80% 70% 60% 50% 40% 1 2 3 4 5 Number of runs= 10 6 7 8 9 10 week 11 12 13 14 15 16 17 18
Four run rules A special cause of variation may be signalled by meeting one of more of the following 4 run rules: 1. Shift in the process (too many data points in a run: >6 consecutive points above or below median) 2. Trend (>5 consecutive points all increasing or decreasing) 3. Too many or too few runs (use table to work out) 4. An “astronomical” data point (the interocular test)
Examples
Are any of the run chart rules met for our example? Percent of patients with same day prescriptions 100% 90% 80% 70% 60% 50% 40% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 week Rule 1: Shift (>6 above/below median)? Rule 2: Trend (>5 all increasing or decreasing)? Rule 3: Too many or too few? (see table next slide – need to know number of useful observations [16] & number of runs [10]) Rule 4: Astronomical data point?
Rule 3: Too many or too few? Number of useful observations Minimum number of runs Maximum number of runs 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 5 5 5 6 6 6 7 7 7 8 8 8 10 10 10 11 12 13 13 14 15 16 16 17 17 18 18 19 19 20 20 21
Clinician-led cycle of care improvement (Prof Ian Scott)
What was the impact of the changes? Useful obs 32 weeks- 3 points on median =29 useful observations percentage of patients with same day prescriptions 100 80 60 40 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
What was the impact of the changes? Useful obs 32 weeks- 3 points on median =29 10 + 2 = 12 runs percentage of patients with same day prescriptions 100 80 60 40 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Four run rules A special cause of variation may be signalled by meeting one of more of the following 4 run rules: 1. Shift in the process (too many data points in a run: >6 consecutive points above or below median) 2. Trend (>5 consecutive points all increasing or decreasing) 3. Too many or too few runs (use table to work out) 4. An “astronomical” data point (the interocular test)
Rule 3: Too many or too few? Number of useful observations Minimum number of runs Maximum number of runs 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 5 5 5 6 6 6 7 7 7 8 8 8 10 10 10 11 12 13 13 14 15 16 16 17 17 18 18 19 19 20 20 21
Control charts
Control charts simplified • Run charts and control charts same purpose • Central line mean (run chart median) • Limits (UCL and LCL) provide additional tests to identify special cause • More statistically robust • Better able to identify special cause variation (more sensitive) • Type of control chart depends on the type of data (many types vs only one type of run chart) • Control chart rules different
Control chart type determined by data type “Variables” (continuous) data • Quantitative data that can be measured • Infinite number of possible values depending on the precision of measurement • Does not have to be collected as a whole number • E. g. weight, height, BP, volume of workload “Attribute” (non-continuous) data Count (“non-conformities”, “defects”) • • • Occurrences only (don’t count those that don’t occur) Numerator only E. g. falls, medication errors, CLAB infections Classification (“nonconforming units”, “defectives”) • • Can count occurrences and non-occurrences Numerator and denominator E. g. % mortality, % readmitted, % c-sections
Which control chart should I use? Continuous? No Yes Classification? 1 observation per subgroup? (usually: is there 1 measurement per time period? ) No Yes I P Do you need to convert to a rate? No C Yes
Key points about measurement • Purpose: learning; x judgement • Be aware of the limitations of the measure • Balanced set: process, outcome, balance • Plot over time • Report regularly (has the process improved, stayed the same or become worse? ) • Measures should be: • linked to the team’s aim • Used to guide improvement and test changes • Integrated into the team’s daily routine • Focus on the vital few (Pareto principle / 80: 20 rule)
- Ibm process improvement
- Process improvement workshop agenda
- Rapid process improvement workshop
- You cannot improve what you cannot measure explain
- Cuadro comparativo e-learning m-learning b-learning
- Facilitate learning session
- Sample lac session topics
- Kontinuitetshantering i praktiken
- Typiska novell drag
- Nationell inriktning för artificiell intelligens
- Returpilarna
- Varför kallas perioden 1918-1939 för mellankrigstiden?
- En lathund för arbete med kontinuitetshantering
- Adressändring ideell förening
- Personlig tidbok
- A gastrica
- Densitet vatten
- Datorkunskap för nybörjare
- Stig kerman
- Tes debattartikel
- Autokratiskt ledarskap
- Nyckelkompetenser för livslångt lärande
- Påbyggnader för flakfordon
- Arkimedes princip formel
- Offentlig förvaltning
- Kyssande vind
- Presentera för publik crossboss
- Argument för teckenspråk som minoritetsspråk
- Kanaans land
- Treserva lathund
- Epiteltyper
- Bästa kameran för astrofoto
- Centrum för kunskap och säkerhet
- Programskede byggprocessen
- Mat för idrottare
- Verktyg för automatisering av utbetalningar
- Rutin för avvikelsehantering
- Smärtskolan kunskap för livet
- Ministerstyre för och nackdelar
- Tack för att ni har lyssnat
- Mall för referat
- Redogör för vad psykologi är
- Matematisk modellering eksempel
- Tack för att ni har lyssnat
- Borra hål för knoppar
- Vilken grundregel finns det för tronföljden i sverige?
- Standardavvikelse
- Tack för att ni har lyssnat
- Steg för steg rita
- Verksamhetsanalys exempel
- Tobinskatten för och nackdelar
- Toppslätskivling effekt
- Datumr
- Egg för emanuel
- Elektronik för barn
- Antikt plagg i rom
- Strategi för svensk viltförvaltning