Measuring Quality Understanding methods of performance measurement Objectives

Measuring Quality Understanding methods of performance measurement

Objectives After this session, participants will understand: • Where we can measure quality • How to choose and calculate an indicator • How to collect the data 2

Measuring Quality • Once we know how care should be delivered, how do we know whether we are doing it? We measure it 3

Importance of measurement • Measuring is important to: – Identify where a problem areas/gaps – prioritize where QI is needed – Guide discussion on possible causes of gaps and their solutions – Measure change over time 4

How can we measure quality? In other words, how do we know there is a problem?

How do we know there is a problem? • You see it or hear about it • You use routine data reporting or measurement • You do targeted measurements 6 6

How do we know there is a problem? (1) • You see it or hear about it – You hear of a “never” event – something you NEVER want to happen. • Can someone think of any example? – It is “obvious” – A patient makes a complaint – Morbidity and mortality reports 7 7

How do we know there is a problem? (2) • Routine measurement – MOH reports (monthly/quarterly reports) – Formal chart reviews • Targeted measurements – How well is TB referral going 8 8

Important steps in measuring • • • Identify the area you are interested in Define what you will measure indicators Collect data Analyze data* Discuss and use the results* *discussed in later talks 9

Important steps in measuring • Identify the area you are interested in • Define what you will measure: indicators • Collect data • Analyze data • Discuss and use the results 10

Three areas to measure the quality of care Structure • Material resources • Human resources • Organizational characteristics • Systems Process • Health care services provided Outcome • Health status • Knowledge acquired by patients and family members • Behavior of patients or family members • Satisfaction of patients and their family members 11

Structure • Material resources • Human resources • Organizational characteristics • Systems • Clinic infrastructure and inventories • Laboratory infrastructure and lab test availability • Pharmacy – availability or stock outs of key medications • Success of referrals • Staff training needs • Systems: How a process is completed in the clinic 12

Process – Were the recommended services provided? Process • Health care services provided • TB screening (active case finding) • CTX prophylaxis • Monitoring: CD 4 or Viral load testing • Antiretroviral treatment (appropriate, timing) • Adherence assessment • HBV/HCV testing 13

What were the outcomes? • Retention in care & treatment services • Adherence • Undetectable viral load • Fewer missed visits • Lab and pharmacy availability • Patient satisfaction Outcome • Health status • Knowledge acquired by patients and family members • Behavior of patients or family members • Satisfaction of patients and their family members 14

Identify what we are interested in • Example: Let’s say we are interested in knowing whether patients are getting PCP prophylaxis as they should be. 15

Important steps in measuring • • • Identify the area you are interested in Define what you will measure indicators Collect data Analyze data Discuss and use the results 16

What is an indicator? • Definition: A variable that measures one aspect of a program or project. • In health care a quality indicator is used to measure how well we are delivering care • Usually based on standards (MOH or WHO guidelines) which define how a service should be delivered. 17

Example: Standard vs. Indicator • Standard (MOH guideline): Cotrimoxazole should be prescribed to all patients with CD 4 ≤ 350 or WHO stage III or IV disease. • Indicator: Cotrimoxazole prescription for eligible patients at the most recent visit. 18

What is a “good” indicator • Relevant (you care about the area) – Linked to a condition or service that occurs frequently or has large impact. • Measurable – Can be realistically measured given resources – Measurement can show change • Accurate – Based on accepted guidelines or developed through groupdecision making methods/consensus • Improvable – Can be improved (by the site or program) 19 Adapted from HIVQUAL guide for QI in HIV care, 2006

Raise your hand if the indicator is good • The percent of patients with CD 4 < 350 who received CTX at their most recent visit • The percent of rooms with pink walls • The percent of patients receiving ART within one month of eligibility • The number of patients on ART who were born in June 20

A few examples of good indicators in HIV • % of patients screened for active TB at most recent visit • % of eligible patients receiving CTX • % of eligible patients on ART • % of patients with CD 4 testing in last 6 months. • % of children up to date with vaccinations 21

Important steps in measuring • Identify the area you are interested in • Define what you will measure: indicators • Collect data • Analyze data • Discuss and use the results 22

Measuring performance • Measure whethere is a gap in an area by measuring the performance. • Example: You’ve decided you want to know whether your clinic is providing good quality of care in the area of PCP prophylaxis and you will turn the indicator into a measure: – “What percentage of patients with CD 4 < 350 received CTX or Dapsone at the most recent visit. ” 23

Steps to measure performance in a selected indicator 1. Who is your starting population? – All active patients at OPC on the date of review 2. Who is in the denominator? – Number of patients who should have received the service 3. Who is in the numerator? – Number of patients who received services ⅞ 24

Steps to measure indicator Population targeted for performance measurement All active patients at date of review Denominator Number of patients who should have received the service Numerator Number of patients who received the service 25

How would you measure the following indicator? Percent of patients eligible for cotrimoxazole who received it at the most recent visit at one OPC with 300 active patients

Who is eligible to have their quality measured? • All active patients on the date of review • At least one visit in the previous year • 300 patients are targeted for performance measurement • But…who is eligible for cotrimoxazole measurement? 27

Denominator • What part of this population should have received the component of care being measured? • All patients with CD 4 <=350 or WHO stage III/IV as of the last visit • After reviewing the 300 charts you find that 100 patients are eligible for CTX prophylaxis at most recent visit 28

Numerator • What number of those who should have received the service (CTX) did receive the service? • All patients with CD 4 <=350 and WHO stage III/IV who were prescribed CTX at most recent visit • Out of 100 eligible patients, 80 patients were prescribed CTX 29

Steps to measure indicator Population targeted for performance measurement All active patients at date of review N=300 Denominator Number of patients who should have received CTX N=100 Numerator Number of patients who received CTX N=80 80/100 = 80% of patients who should have been receiving prophylaxis were actually receiving it 30

Important steps in measuring • Identify the area you are interested in • Define what you will measure: indicators • Collect data • Analyze data • Discuss and use the results 31

How much data should you collect? • Do you have to review every chart every 6 months? No. • It is not a good use of resources to collect data from every active patient in your clinic • The goal of measurement in QI is to routinely collect “just enough” data 32

“Just enough” • You sample enough so you are confident that your data gives you a good estimate of the clinic’s performance in that area • Keep it simple enough to be able to do routinely • Sampling: Collecting data from a smaller number of charts that are representative of your population (active patients) 33

So how many charts do we need to review? • Choose the minimum number of charts to be sure enough of the true performance. • Allow a margin of error (we use +/- 8%) • The size of your sample is determined how big is your population. 34

Sample size when measuring indicators • This table was constructed to give us 95% confidence that the performance rate we get is correct +/- 8%. • Eg. If we find the rate is 80% then we are very confident that the real rate lies between 72 and 88%. Good enough for QI Number 1 -20 21 -30 31 -40 41 -50 51 -60 61 -70 71 -80 81 -90 91 -100 101 -119 120 -139 140 -159 160 -179 180 -199 200 -249 250 -299 300 -349 350 -399 400 -449 450 -499 500 -749 750 -999 1000 -4999 Sample size All 26 32 38 43 48 53 57 61 67 73 78 82 86 94 101 106 110 113 116 127 35 131 146

Choosing the charts – registry available • If clinic registry is available. Choose from clinic registry • Random sample – random number generator • Or systematic sample – Example: choose 94 (based on table) in 200 ART: choose every 2 nd patient (2=200/94). – Pull patient chart out corresponding with the selected ID patient above. 36

Choosing the charts – no registry available • Choose patient charts from the shelves. • Example: OPC with 200 patients. Sample size: 94 charts. • Select every 2 nd chart (2= 200/94) and check to see if patient is active – Yes – Abstract data – No – choose the next chart on the shelf. Note: who is on the shelf will affect your sample. Be careful if charts are grouped (ART/Care/death – ask TA provider for assistance) 37

How do you ensure data quality? • Standardized tool or method of collection • Staff trained in how to collect the data • Review during data collection to ensure being done right. • More to come in data collection and software training. 38

Summary (1) • Performance measurement is built from indicators based on approved standards • Indicators may be qualitative or quantitative and are generally a mix of process, structural and outcomes 39

Summary (2) • “Just enough”: You don’t need to review all charts for QI. Amount of data should be feasible for routine collection. If you are not going to use it, don’t collect it. • Indicators should be chosen based on relevance; ability to measure, intervene and monitor for improvement over time. 40