An Error Reporting Tool Designed to Stimulate System

An Error Reporting Tool Designed to Stimulate System Changes Ashok Singh MA MB BChir (Cantab) John Taylor MBA Ranjit Singh MA MB BChir (Cantab) MBA Bruce Naughton MD Diana Anderson Ed. M Gurdev Singh MSc Eng. Ph. D (Birm)) SAR-AIMER Systematic Appraisal of Risk And Its Management for Error Reduction Patient Safety Research Center Family Medicine Research Institute 462 Grider Street Buffalo. NY 14215. USA gsingh 4@buffalo. com www. fammed. buffalo. edu/safety NAPCRG 2005

Overview of This Presentation • Context and Background • Objective • Design • Setting and Participants • Main Outcome Measures • Results • Conclusions

The National Burden of Systemic Errors in the Health Care US (US) al He e ar c th More than 3 fully occupied Jumbo jets of the Health Care Industry drop out of the sky every day ! (Adapted from Leape: the Patient Safety Guru of USA) And then there are other adverse Events!!

Healthcare Safety Comparison with Civil Aviation Adverse Events Per Million Opportunities 1. 000, 000 MEDICATION 100, 000 IOM 10, 000 100 10 aviation 1 1 2 σ 3 SIGMA 4 5 6 σ Adapted from Robert Galvin (GE, MOTOROLA)

Context Fear Kill the messenger Micromanage (Barking up the wrong tree) (denial; shift the blame) Filter the data (game the system) Scherkenbach’s Cycle of Fear, 1991

Context S S • Health industry leaders must make safety their top priority • They must encourage and facilitate reporting and learning from errors as well as sharing and experiences with a structured and consistent analytical methodology that stimulates system enhancements The Patient Safety and Quality improvement Act 2005 The IOM’s 5 th Safety Principle (2000) • This reporting method should be: * Easy and Safe to use * Worthwhile

Context S S Error reports are a rich source for understanding: *Causes of errors *Cascades of errors *Consequences Understanding Safety Improvements

REPORTS PROFILES AUDITS PLUS =Mechanistic 95% Healthcare workers (ALL) over time develop an understanding of how the system works (or doesn’t work) But remember that reports are just a tip of the iceberg Humanistic =Pursuit of Excellence Assessment Review & Learn Plan Feedback Implementation PRACTICE BASED Let us at least see FMEA this tip more clearly AAFP/RGC Safety Symposium, WONCA 2004

REPORTS PROFILES AUDITS PLUS Humanistic =Pursuit of Excellence =Mechanistic 95% Healthcare workers (ALL) over time develop an understanding of how the system works (or doesn’t work) Assessment Review & Learn Feedback PRACTICE BASED FMEA AAFP/RGC Safety Symposium, WONCA 2004 Plan Implementation

Objective Overall Goal of the Program: To prepare senior residents for leadership roles in safety and quality enhancement in ambulatory and hospital settings by training aimed at the ACGME competency of: System-Based Practice and Learning NAE/IOM Report 2005: Building a Better Delivery System

Objective Specific Objective of the work presented: To develop and test an approach for teaching system analysis skills to senior residents: - analysis of errors and adverse events - identification of error prevention strategies - encouragement of situational awareness System-Based Practice and Learning NAE/IOM Report 2005: Building a Better Delivery System

Setting/Participants University-Based Family Medicine Residency Program One of the Oldest and Biggest in the USA With 5 Ambulatory Sites And 46 Residents (15 PGY 3) 38 faculty

Design 1) PGY-3 residents receive a 4 -part course entitled: “Systems Approach to Patient Safety” 2) To demonstrate application of safety techniques, principles and strategies taught, residents keep a pocketsized Safety Journal in which they record examples from their practice of: - Observed errors - Anticipated errors - Near misses And suggest solutions, based on principles taught. 3) Pre- and Post-Testing: 2 exercises to assess System-Based Practice Skills: - RCA exercise (analysis of an Adverse Event) - FMEA exercise (analysis of a lab ordering process)

Design as a part of Our Vision ACGME Requirements Through the Prism of SAFETY IOM call for Patient Safety Training Achieved through didactic, experiential, and evaluative components Competencies 1. Patient Care 2. Medical Knowledge 3. Practice-Based Learning 4) Communication Skills Objectives 1. Improve team building skills 2. Reduce inappropriate prescribing (esp. with geriatric patients) 3. Encourage and facilitate selfevaluation to instill a culture of safety 4. Enhance communication skills with patients/families/colleagues 5)Professionalism 5. Improve patient safety ethics 6) System-Based Practice 6. Analyze system components and address system problems to improve safety

Main Outcome Measures • Journal entries: - Ability to identify/anticipate errors in their practice and to correctly identify and categorize causative factors - Ability to devise appropriate system-based solutions to identified problems • Pre- and Post tests - RCA exercise: Ability to identify causative factors and devise appropriate solutions - FMEA exercise: Ability to analyze a process and identify opportunities for errors

Evolution of the Safety Journal • Year 1: - Format consisted of a series of questions asking residents to describe the error, contributing factors, consequences and suggested solutions. - Results were not satisfactory – most proposed solutions were individual-based rather than systembased. • Year 2: - We devised a visual format for the journal, based on Reason’s Adverse Event Trajectory. - We hypothesized that asking residents to categorize causative agents within this framework would prompt them to develop system-based solutions.

Adverse Event (Accident) Trajectory a dynamics that aids causal analysis (RCA) THE SAFETY JOURNAL Anticipated errors Close Calls Actual Errors Situational Factors After Reason Latent Failure Active Failure (Human Factors) SAFETY BARRIERS • Humans are “set up” to make errors that they are not responsible for • Most errors result from faulty systems rather than human error

Situational Factors “Unlucky/Unforeseen Circumstances” Examples: * Very unusual workload * Distraction * 2 patients with the same name * Power supply failure at critical juncture These factors put additional stress on the system and can thereby trigger Latent Failures that may have existed for many years without causing any adverse events.

Latent Failures 1 AKA “Blunt End” failures - these present the greatest threat to safety These are errors in: Design Operation Maintenance Organization Management They exist before the active error occurs.

Latent Failures 2 Latent errors are “built into” the system - They are difficult for staff to see because they may be “hidden/dormant. ” - They may facilitate/trigger an error/s or amplify the resulting adversity. Fixing of this latent source of errors yields the greatest improvement in safety.

Latent Failures 3 Examples * Disregard of human factors/fallibility * Poor equipment design * Poor document control (e. g. lab results) * Bug in the software * Unclear allocation of responsibility * Inappropriate scheduling/planning (no overlap/continuity) * Poor interdepartmental coordination * Poor prescribing protocol * Inadequate training * Employment of unsuitable staff

Active Failures: AKA “Sharp End” errors - made by workers in the processes of care Their effects are felt/observed relatively rapidly or immediately Blame is often inappropriately focused here

Active Failures Examples * wrong drug prescribed * wrong dose prescribed * wrong medication given * wrong patient * wrong site procedure * picking up wrong bottle from shelf * mislabeling of specimen * missed allergy

Safety Barriers: They hinder humans from committing an active failure or absorb the effects of active failures before adverse event can occur. They can be: • Technical (“strong”) • Administrative : Policies and Procedures (“weak”)

Safety Barriers: Technical (“strong”): examples • • plugs, sockets, connections incompatibility equipment failing to safe default state forcing functions informatics safe and distinct storage of drugs safe dosage packaging safety guards standardization of equipment layouts and displays • intuitive equipment operation

Safety Barriers: Administrative (“weak”): examples • • • standard protocols and procedures clear clinical pathways adequate credentialing requirements emergency drills, simulations double checks, teach-backs making processes transparent and visible adequate looseness in coupling autonomy to handle the unexpected non-hierarchical team culture

Safety Barriers: Another useful way of looking at Barriers: TYPE • Physical e. g. forcing functions, walls, locks etc • Natural e. g. time and distance • Information e. g. labels, alarms, signs etc • Measurement e. g. measurements, inspections • Knowledge e. g. training, work aids, checklists etc • Administrative e. g. checklists, procedures, policies etc

Safety Journal: sample entry Safety Journal The “Story” Type of Incident (circle) *Observed Error Near Miss Anticipated Error 56 yr-old female with Type 2 DM on insulin came on to the ER after an Describe in detail what happened. Do not use dates or names of people or places. accidental overdose of Regular insulin. She takes 30 units NPH with 8 units In your description be sure to include the following, where relevant: Regular before breakfast. Both types of insulin come in a pen with a dial to ·how the error happened set the insulin dose. Both pens look the same. She was at home that ·the consequences of the error and how the error/consequences were detected morning getting ready to take her insulin when she accidentally picked up ·details of any special circumstances that contributed the Regular ·the roles played by individuals, teams, and leadership pen and gave herself 30 units of Regular. She realized her mistake right CONTINUED………… ·details of the systems that were in place and how they contributed to and/or mitigated the effects of theaway errorand so she came to the ER. Both pens look the same Distracted by grandchildren Possible strategies to prevent recurrence (Focus on eliminating latent failures and creating safety barriers, especially “strong” ones 1. Make the pens look different e. g. make the Regular pen RED so it stands out 2. Set maximum dose on the Regular insulin pen, e. g. 12 units 3. Give patients clear instructions regarding what to do if they accidentally overdose Patient picked up wrong pen Missing Barrier: No mechanism to limit the dose of Regular insulin

Results • In a 2 -year period: 69 cases reported by 30 residents Each presented at least 1 case at ‘Safety Rounds’ • Types of incident reported: 49 Observed errors (33 with Harm) 4 anticipated errors 16 close calls • Settings: 24 Outpatient 39 Inpatient 6 Transition between settings

Results • Year 1: original format • 42 submissions (3 per resident vs. 6 expected) • 55% identified appropriate system-based solutions • Year 2: changed to visual format • 27 submissions (1. 8 per resident vs. 3 expected) • 81% identified appropriate system-based solutions • Significant improvement compared to year 1 (p=. 023)

Results Year 2: 27 submissions Identification and Categorization of Causative Factors Type: Situational Latent Active Mean per case 1. 6 1. 0 2. 1 % Identified 92% 58% 88% % Correctly categorized 84% 93% 67%

Results Year 2: 27 submissions • Solutions proposed % submissions that included Behavior changes by HC workers 70% System changes to prevent errors 81% Creation of Barriers to correct errors 33% Mean #/case 1. 2 1. 9 0. 6

Results Pre- and Post-Testing RCA Exercise Analysis of case Root causes identified System factors identified Improvement strategies proposed Education of healthcare workers System changes Steps to change the culture Pre (n=14) Post (n=15) Mean 6. 5 8. 5 1. 1 2. 8 0. 1 1. 6 0. 4 0. 3 2. 8 (p=0. 012) 0. 9 8. 6 5. 3 4. 5 9. 2 12. 8 (p=0. 006) 5. 5 4. 8 9. 1 FMEA Exercise Steps identified Human entities identified Other entities identified Failure modes identified

Conclusions Safety Journal: • Residents were able to appropriately identify and categorize causative agents including situational, latent and active failures and lack of safety barriers. • Most demonstrated the ability to devise system-based improvement strategies. The visual format appeared to facilitate this. • Pre- and Post-Testing of System Analysis Skills • There was evidence of improved understanding of the causes of errors, especially systemic causes, and the need for system change.

Future Plans This journal has received enthusiastic response (including at the ABMS/ACGME Conference in Sept 2004) and it is now being introduced at UB into: • Surgery Residency • Geriatric Center of Excellence • Pediatric Surgery Grand Rounds • Further evaluation is needed

M (b ic a rk Moving toward breaking the Cycle of ro at th e Fear m a w n ag g tr e ro n ee Kil l th (de nia em l; s hif ess e e b ng t th ) lam e) er a t a d tem) e ys h t s r the e lt e Fi (gam Scherkenback’s Cycle of Fear, 1991

Thanks & Safe Interventions! Thanks are due to HRSA for the Title VII Grant that made this work possible www. fammed. buffalo. edu/safety

References 1. Singh G, Singh A, Singh R, Taylor J, Singh S. Building a Learning Practice with Self. Empowered Teams. BMJ Quality 10 th European Forum April 2005. London. 2. Singh R, Singh A, et al. Computer Visualization of patient Safety in Primary Care: A Systems Approach adapted from Management Science and Engineering. Informatics in Primary care, Volume 13 No. 2. 2005. 3. Singh G, Singh R, Singh A. Practice-Based Failure Modes and Effects Analysis for System Enhancements. Invited Paper presented at: AAFP/RGC Safety Symposium, WONCA World Congress of Family Doctors. 2004. http: //fammed. uchsc. edu/taxonomy/resources. aspx (PDF accessed Feb. 5 2005) 4. Singh R, Taylor JS, Naughton B, Koenigsberg M, Wahler R, Anderson D, Mc. Causland L, Robinson A, and Singh G. A Comprehensive Collaborative Patient Safety Residency Curriculum to Address the Core Competencies. Medical Education. In Press (Accepted Feb. 2005). 5. Singh G, Singh R. Novel Use of Hazard Ratings from Safety Engineering to Target Error Reductions in Primary Care. Proceedings of the Annual Meeting of the North American Primary Care Research Group, October 2003. http: /www. Napcrg. org/confertence. html. (accessed Feb. 2005). 6. Reason J. Human Error. Cambridge University Press. 1990 Singh: April 2005