Using data to improve the reliability of care
Using data to improve the reliability of care Bruce Guthrie Professor of Primary Care Medicine, University of Dundee GP NHS Fife With thanks to everyone in the Safety Improvement in Primary Care Programme (SIPC), and The Health Foundation for funding SIPC
Outline • Getting started with data • High-volume processes and sampling – What are we trying to improve? – How do you measure a nightmare? • Reliability and bundles – Examples from other contexts – Examples from SIPC • What to do in real life?
Getting started with data • “Education, education” Tony Blair 1997 • “It’s the economy, stupid” Bill Clinton 1991 • No politician has yet said: – “Data, data” – “It’s the data, stupid”
Getting started with data • Warfarin sheets audit No (%) of patients with element recorded Name and date of birth Phone number Diagnosis Planned duration Target INR 36 (100) 31 (86) 18 (50) 21 (59)
Data is people • 68 year old woman on warfarin, diagnosis and duration not recorded, target 2. 5 – Warfarin for a pulmonary embolus shortly after surgery for oesophageal cancer – Should have stopped (and is now)… No (%) of patients with element recorded Round 1 No (%) of patients with element recorded Round 2 Name and date of birth 36 (100) Phone number Diagnosis Planned duration Target INR 36 (100) 31 (86) 18 (50) 21 (59) 36 (100) 27 (75) 36 (100)
Process volume • Previous example is low volume, but the rest of warfarin is higher volume Criteria Last INR within +/- 0. 5 of target Last INR within +/- 0. 75 of target % of patients (n=35) 71% 89% Standard/comments 50% - MET 80% - MET Last INR <1. 5 -1. 9 4 -4. 9 ≥ 5 Number of weeks to ‘next test’ 1 2 3 4 5 6 3% 19% 3% 0 Out of range are mostly low which suggests we are tending to undertreat to target 26% 40% 6% 26% 0 3% Testing much more than is necessary given our good levels of control
Process volume • SIPC 1 projects vary in volume – Warfarin process is high volume – DMARDs and heart failure low volume • SIPC 2 projects are all high volume – Medicines reconciliation at discharge – Outpatient letters – Results handling • Exciting and innovative improvement work – “Solving some of the most important problems of healthcare” (Health Foundation) • Boring, boring work…
Safety systems • Learning from other industries • Error doesn’t usually cause harm. . . • . . . so should act on error and risk not harm
Safety systems • “It appears to me that there were significant flaws in the quality of Occidental’s management of safety which “Relying the absence anyof affected theon circumstances of theof events the disaster. of Senior management feedback problems as were too easily satisfied that the Permit to Work indicating that operated all wascorrectly, well” system was being relying on the absence of any feedback of problems as indicating that all was well. ” (Lord Cullen)
Data is people • Letter comes in, a 64 year old woman just diagnosed with terminal lung cancer in 2010 – “In retrospect, the lesion is visible on a previous CXR” • Central Vision CXR report from 2005 – “Chest shows a well rounded opacity on the left side of the chest which is strongly suspicious of bronchial lesion. Referral to the chest physicians for further investigation is clearly appropriate. ” • A heartsink moment… • …but not our fault • What would stop it happening to us?
Data is people • Home visit to a man with tenesmus in 2004 • Recently referred to colorectal with ? cancer (appointment tomorrow) • Had been seen with loose bowels 15 months ago – Short history, normal examination inc PR, “do some tests, come back if abnormal/persistent” – FBC and stool culture normal, FOB positive • Clinical care fine, results book failure
Data is people
Safety systems • Remember Lord Cullen’s criticism of Occidental management? – “Relying on the absence of any feedback of problems as indicating that all was well” • ‘Usual’ feedback isn’t reliable • Collecting data and measuring can be • Health care is more dangerous to patients than the oil industry is to employees – But we harm people one at a time
Process volume • SIPC topics – High risk medicines - warfarin, DMARDs – High risk conditions – heart failure – Medicines reconciliation at discharge – Outpatient letters – Results handling • Boring, boring work… • …but getting the paperwork wrong can kill people
How to measure? • Remember our warfarin audits? – Data is all in the warfarin book – Only 36 patients – All the measures were easily countable • Can’t always count everything every time – Volume may be too high, but high volume is an opportunity to sample – Take 5 or 10 results and check if actions done – Take 5 or 10 DC summaries or letters and check medications reconciled/actions done
DMARD FBC in last 6 weeks
How to measure • There’s no right answer to: – How many to measure each time – Complete audit vs small sample – How often to sample – Depends on the context • Have to decide what to measure first…
Measures and bundles • Quality measures are hard to define well • Traditionally have usually been stand alone – % of patients with HT with controlled BP – % of patients on warfarin with INR in range – % of patients over 75 receiving a flu jag • Increasingly measures are ‘bundled’ in various ways – AKA composite measures
All or nothing measures • ‘Care bundles’ are all or nothing measures – The % of patients who achieve ALL individual measures/get all appropriate care • Appropriate when: – Each element is important in its own right – Patient outcome is improved by ALL measured care being received (the whole is greater than the sum of the parts) – Each element should be necessary every time
Some examples Diabetes data from 59 practices Measure % of patients achieving GHB done BP done Cholesterol done Smoking recorded GHB≤ 7. 4% BP<140/80 Cholesterol≤ 5 Non smoker Guthrie, B. , A. Emslie-Smith, et al. (2009). Diabetic Medicine 26(12): 1269 -1276. 95. 4 95. 0 93. 6 96. 2 55. 3 38. 7 75. 0 82. 9
Some examples Measure GHB done BP done Cholesterol done Smoking recorded GHB≤ 7. 4% BP<140/80 Cholesterol≤ 5 Non smoker % of patients achieving 95. 4 95. 0 93. 6 96. 2 55. 3 38. 7 75. 0 82. 9 % of all care done (average of the measures 95. 0 51. 4
Some examples Measure GHB done BP done Cholesterol done Smoking recorded GHB≤ 7. 4% BP<140/80 Cholesterol≤ 5 Non smoker % of patients achieving 95. 4 95. 0 93. 6 96. 2 55. 3 38. 7 75. 0 82. 9 % of patients with all care done 88. 3 Could do better? 16. 2 Ouch!
A SIPC 1 example • Heart failure bundle – Prescribed licenced beta blocker – Target or maximum tolerated BB dose – NYHA status recorded in the last year – Patient given education about deterioration – Pneumococcal immunisation ever • Each element necessary? • Whole greater than the sum of the parts? • Each element necessary every time?
A SIPC 1 example On BB BB target NYHA Info about deterioration Pneumococcal Composite = got all Yes Yes No No No Yes Yes Yes No Yes No 80% 60% 40% 100% 20% Ouch! All or nothing composites are brutal or Reliability is hard
Bundles and the Swiss Cheese model Result doesn’t arrive Test not tracked out Test n Result not actioned ot tracked out Patient not sure how to get result Missed diagnosis HARMTest n Missing result noticed ot tracked out Patient not informed Reason, James (1990). Human Error. Cambridge University Press.
Choosing measures? • What are the individual measures? – For Boards and practices to decide – SIPC has a number of measures at various stages of development; other examples • Single measures or care bundle/all or nothing approaches? – Each element necessary? – Whole greater than the sum of the parts? – Each element necessary every time?
The SIPC approach • Make every measure count – 4 -5 measures in total – Measure what you need not just what you can – Processes plausibly ‘tightly linked’ to outcomes – Whole is plausibly more than the sum of its parts – Room for improvement – Suitable for repeated measurement in PDSAs • Be flexible (change, drop, add) • But there’s more to life than care bundles…
New political slogans? • Data, data (David Cameron 2012) • It is the data, stupid (Barack Obama 2012) – Because data is people – Because boring processes can kill people when they are unreliable – Because we can be better • Good luck!
- Slides: 28