Comp 15 Usability Human Factors Unit 10 b

Comp 15 - Usability & Human Factors Unit 10 b - Designing for Safety This material was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1 U 24 OC 000003.

Woods and Colleagues: Resilience Engineering Challenger disaster an example of: • Drift toward failure as defenses erode in the face of production pressure. • An organization that takes past success as a reason for confidence instead of investing in anticipating the changing potential for failure. • Fragmented problem solving process that clouds the big picture. • Failure to revise assessments as new evidence accumulates. • Breakdowns at the boundaries of organizational units that impedes communication and coordination. Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 2

Woods and Colleagues: Challenger Analysis Interpretation of past “success”. The absence of failure is taken as positive indication that hazards are not present or that countermeasures are effective. An organization usually is unable to change its model of itself unless and until overwhelming evidence accumulates that demands revising the model. Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 3

‘Failure of Foresight’ Focus on differences-> people see no lessons for own operations, narrow well bounded responses Crux is to notice info that changes past models, without clearcut evidence Provide ‘fresh’ view by: • New people, interactions across diverse groups, knowledge, tools, new visualizations which capture big picture, reorganize data into different perspectives Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 4

Woods and Colleagues: Detecting Danger Cross-checking, collaborations Display safety margin indicators “Errors will always be there” – can’t have 100% perfection, but anticipate and avoid risk situations, or handle appropriately Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 5

Woods – Resilience Engineering (cont. ) Based on insights from above 5 patterns Assessing organization risk, i. e. that holes in decision making will produce unrecognized drift to failure boundary Assessing technical hazards, but goal is to monitor decision making Balance production pressures with protection pressures Management commitment to above Truly open and encouraged reporting Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 6

Resilience Engineering (cont. ) Learning culture v. culture of denial reflected in incident response Preparedness/Anticipation Opacity/Observability Flexibility/Stiffness Revise/Fixated Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 7

Resilience Engineering – 3 Basics (1) detecting signs of increasing organizational risk, especially when production pressures are intense or increasing; (2) having the resources and authority to make extra investments in safety at precisely these times when it appears least affordable; (3) having a means to recognize when and where to make targeted investments to control rising signs of organizational risk and re-balance the safety and production tradeoff. Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 8

Failure Factors and Recovery Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 9

Patel, Cohen – Error in Critical Care Distribution of cognition can result in vulnerabilities Component 15/Unit 10 b Increased complexity, interruptions (Laxmisan: every 9 min for attendings, 14 min for residents) Health IT Workforce Curriculum Version 2. 0/Spring 2011 10

Time Course of Medical Error Death Normal Routine Near Miss Boundary Adverse Event Report Boundary Violation of consensual bounds of safe practice After Patel, 2007 Component 15/Unit 10 b Error recovery: Detection and correction of violation Health IT Workforce Curriculum Version 2. 0/Spring 2011 11

Error Detection and Correction Poorly understood, but integral to all cognitive work Bounds of acceptable practice violated, usually corrected (e. g. prescribe neglected med) Failure to detect this > cross another boundary > adverse event Maximal productivity > strains system, shifts balance between error commission and correction (due to cognitive capacity strain) Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 12

Workflow Analysis and Modeling (Malhotra and Colleagues: 2006) Detailed characterization of individual workflows ID critical events Reconstruct collective workflow from events connected in space or time, done collaboratively Delineate the workflow, role players, devices, protocols, and communications so that we can identify and focus on areas where cognitive aids, technology or interventions may be of assistance. Develop a generalizable cognitive model to represent the intricate workflow applicable to other health care settings Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 13

Schematic Layout of the Cardio Thoracic Intensive Care Unit (CTICU) & Key Activities (Malhotra et al 2007) A, attending; R, resident; F, clinical fellow; PA, physicians assistant; N, nurse. Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 14

CTICU Critical Zones - Examples Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 15

Intensive Care Unit (ICU) and Critical Care Almost all patients admitted to ICU suffer adverse events 10% average mortality 5 milllion ICU admissions/year (US) 30% of hospital costs, $180 B / year Priority for JCAHO and Leapfrog Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 16

Factors in ICU Care Staffing: Intensivists reduce mortality (3 x reduction, Pronovost, JAMA 1999) Fewer ICU nurses -> increased LOS, pulmonary complications (Pronovost ECP 2001) Pharmacists: daily rounds: 66% reduction in ADEs (10. 4/1000 pt days -> 3. 5 Principles: staff accountable, reduce complexity, independent redundancies for key processes Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 17

Care Goal Sheet (Pronovost) Instituting care goals dramatically reduced length of stay from 2. 2 to 1, increasing revenue from new admissions • From Pronovost, 2005 Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 18

Critical Care Environments Methods include ethnographic data collection, observations, surveys and questionnaires coupled with cognitive task analysis of the processes Vankipuram: RID tags: dual method, like ‘black box’ of aviation Data trained analyzed, visualized using virtual world replay Israeli study: avg pt has 178 actions/day; errors 1% (from Gawande, the checklist); 2/day/pt Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 19

Virtual World Replay (from Vankipuram and Colleague: 2010) Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 20

Cognitive Taxonomy of Error (Zhang and Colleagues: 2004) Systematically categorize medical errors (individual level) Better understand cognitive mechanisms of medical error Framework to guide future studies Interventions to decrease errors Foundation for reporting system (to categorize, ID and generate solutions Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 21

Errors Reason: “an error is a failure of achieving the intended outcome in a planned sequence of mental or physical activities when that failure is not due to Chance” Not all adverse events caused by error E. g. device malfunction, System problems (e. g. delays in care caused by organizational policies); not caused by individuals Non-preventable: e. g. unpredictable drug reaction Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 22

Cognitive Taxonomy of Error Chain of events leading to error From: Zhang and Colleague: 2004 Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 23

Example of an Error and Questions It Raises (from Zhang, 2004) Nurse tries to program infusion pump to deliver 130. 1 ml/h, presses “ 1 3 0. 1” Nurse is unaware that decimal point only works for numbers up to 99. 9 Pump ignores decimal point key press and is programmed to deliver 1301 ml/h Error blamed on user, ‘solved’ by more training Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 24

Error Example (cont. ) Questions: why did decimal point only work up to 99. 9? (design flaw)? Why does device just ignore decimal key press, rather than alerting user? Why is nurse unaware of this flaw? Was this problem covered in training? Why was order written for 130. 1, which pump cannot deliver? Why did nurse not see 1301 on display? Tiredness? Display hard to read? Understaffing? Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 25

Error Taxonomy 2 types: • slips that result from the incorrect execution of a correct action sequence and • mistakes that result from the correct execution of an incorrect action sequence. • Zhang et al: based on Norman’s 7 -stage theory of action; errors can be on evaluation side as well as execution side Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 26

Taxonomy From Zhang and Colleague: 2004 Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 27

Examples From Zhang, 2004 Slip Stage in Action cycle Examples Execution slip Goal slips Doctor was called out of the room to answer an urgent call and afterwards went to the room of a different patient who was next in the queue. (Loss of activation) Intention slip A nurse intended to enter the rate of infusion using the updown arrow keys, because this is the technique required on the pump she most frequently uses; however, on this pump the arrow keys move the selection region instead of changing the selected number (capture) Action specification slips A nurse intends to decrease a value using the decrement function, but pushes the down arrow key (which moves to the next field) instead of the minus key. (Associative activation) Action “I meant to turn off the antibiotics IV only, but turned off the execution slips infusion pump completely” (Double capture) Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 28

Examples From Zhang, 2004 (cont. ) Slip Stage in Examples (From Zhang, 2004) Action cycle Executi Goal slips on slip Intention slip Doctor was called out of the room to answer an urgent call and afterwards went to the room of a different patient who was next in the queue. (Loss of activation) A nurse intended to enter the rate of infusion using the up-down arrow keys, because this is the technique required on the pump she most frequently uses; however, on this pump the arrow keys move the selection region instead of changing the selected number (capture) Action A nurse intends to decrease a value using the decrement specification function, but pushes the down arrow key (which moves to the slips next field) instead of the minus key. (Associative activation) Action execution slips Component 15/Unit 10 b “I meant to turn off the antibiotics iv only, but turned off the infusion pump completely” (Double capture) Health IT Workforce Curriculum Version 2. 0/Spring 2011 29

Examples From Zhang, 2004 (cont. ) Mistakes Stage in action cycle Examples (From Zhang, 2004) Execution mistakes Goal mistakes Incorrect diagnosis due to neglect of base rate information (Biases) Intention mistakes A physician treating a patient with oxygen set the flow control knob between 1 and 2 liters per minute, not realizing that the scale numbers represented discrete, rather than continuous settings (Incomplete knowledge) Action specification mistakes Strange burn scars appear in post-operative patients in a hospital. The problem was caused by electric discharge of a device that was not grounded. The device has a blinking red light to signal the problem, but the device operators did not know the meaning of the signal. (Incomplete knowledge) Action specification mistakes For example, a perfect knowledge of a surgical procedure may not lead to a successful surgical operation if the operator has not extensively practiced the procedure. (Dissociation between knowledge and rules) Component 15/Unit 10 b Health IT Workforce Curriculum Version 2. 0/Spring 2011 30

Examples From Zhang, 2004 (cont. ) Evaluation Mistakes Perception mistakes A pharmacists filling prescription for Lamisil (an antifungal) mistakenly perceived Lamictal (an anticonvulsant) as Lamisil because he mistakenly expected it since he was looking for Lamisil. (Misperception) Interpretatio A steady green light on an infusion pump means the device is n mistakes ready, and a flashing green light indicates an infusion is in progress. The device user did not know the meaning of the steady green light, and correctly interpreted it as an indication that the infusion had begun. (Incorrect knowledge) Action evaluation mistakes Component 15/Unit 10 b In the infusion pump example the user may not know that the device has accepted the volume, and may then assume that the goal (‘set volume to be infused at 1000 cc’) has not been accomplished, leading to a search for additional buttons (such as ‘enter’) to complete the goal (Incomplete knowledge) Health IT Workforce Curriculum Version 2. 0/Spring 2011 31

Cognitive Interventions Depend on type of slip or mistake • e. g. education, decision support, representational aid, information reduction, display design, device redesign Component 15/Unit 10 b Aids for perceptual systems • Example: if intention slip due to loss of activation in memory, give memory aid (‘Press Volume to enter volume to be infused’). Health IT Workforce Curriculum Version 2. 0/Spring 2011 32

Errors - Context Contrary to expectation, most errors can happen at times of low productivity Component 15/Unit 10 b Personnel will arrange items for maximal functioning during peak times (Xiao, 1995) Health IT Workforce Curriculum Version 2. 0/Spring 2011 33