Human Factors and Usability Engineering in Healthcare Jessica
Human Factors and Usability Engineering in Healthcare Jessica S. Ancker, Ph. D, MPH Associate Professor Division of Health Informatics Department of Healthcare Policy & Research Weill Cornell Medical College jsa 7002@med. cornell. edu 1
A fun idea for a football promotion – Dress the Buffalo Bills in red and the NY Jets in green
This is how it looked to the 8% of male fans with red-green color-blindness Whose fault was it that these male fans couldn’t follow the football game? https: //www. nytimes. com/2015/11/14/sports/football/when-red-bills-met-green-jets-colorblind-fans-lost. html? _r=0
A resident in a hospital opens a patient’s chart to place an order for insulin. He is interrupted by the attending physician, who asks about a second patient. Then an intern asks about a third patient. After dealing with these questions, the resident finally finishes placing the order for the insulin. But he has the wrong chart open, so the order is attached to the wrong patient chart. 4
The wrong patient gets the insulin. The patient has a severe reaction, and narrowly escapes death. Who’s to blame for this medical error? 5
The human factors perspective A system is a team of people, processes, and tools that operate together within some environment to take input and produce output. Human factors engineering is the discipline that seeks to design entire systems that function well. The human factors engineer takes into account human strengths and limitations in the design of interactive systems that involve people, tools and technology, and the design of work environments to ensure safety, effectiveness, and ease of use. 6
Examples of systems University Professors Students Curriculum Tuition Grading rules Classrooms Curriculum requirements Graduates Course software and texts Hospital Doctors Sick people Trainee doctors (residents) Required treatment protocols Electronic health record Nurses Supply of medications Operating rooms and beds Healthy people Fully trained doctors 7
A resident in a hospital opens a patient’s chart to place an order for insulin. He is interrupted by the attending physician, whoperspective, asks From a systems doesathe errorpatient. could about a second patient. Then an intern asks about third originate in: After dealing with these questions, the resident finally finishes The person? The process? placing the order for the insulin. But he has the wrong chart open, The technology? so the order is attached to the wrong patient chart. The environment? The relationships? From a systems perspective, what could be done to prevent this type of medical error in the future? 8
From a human factors perspective: Most errors occur because the system is not set up to support the person 9
In health informatics, we frequently focus most carefully on the technology part of the system (But never forget that the technology is only one part of any system) https: //cqpi. wisc. edu/seips/ 10
Broadly speaking, usability is the quality of the technology that renders it easy and pleasant to use. What are some examples of technologies that you think are highly usable? Process Consulting for Health IT 11
Jakob Nielsen’s definition of usability focuses on the relationship between the individual user and the technology 1. Learnability: How easy is it to accomplish basic tasks the first time? 2. Efficiency: How quickly can users perform tasks? 3. Memorability: When users return to the design after a period of not using it, how easily can they reestablish proficiency? 4. Errors: How many errors do users make, how severe are the errors, and how easily can they recover? 5. Satisfaction: How pleasant is it to use? 12
https: //bmcmedinformdecismak. biomedcent ral. com/articles/10. 1186/1472 -6947 -6 -3 Task-technology fit perspective suggests that we also examine the work that specific people are trying to do with the technology “Usability is the effectiveness, efficiency and satisfaction with which specific users can achieve a specific set of tasks in a particular environment” HIMSS Usability Task Force, 2009 13
The Cedars-Sinai case • • This major medical center in LA invested millions to build a state-of-the-art clinical decision support system Physicians refused to use it and forced hospital to drop project • poor usability (too many clicks) • poor fit with workflow (interfered with normal procedures) • physicians perceived it as a threat to patient safety, not a help Morrissey 2004 14
“[Usability] has a strong, often direct relationship with clinical productivity, error rate, user fatigue and user satisfaction. ” HIMSS EHR Usability Task Force, June 2009 http: //www. himss. org/content/files/himss_definingandtestingemrusability. pdf 15
Let’s visit www. medicare. gov Who are the intended users? What are some tasks they might need to do? What is the environment in which they will try to accomplish those tasks? Does the technology do a good job of supporting them? 16
How about https: //www. health. ny. gov/health_care/medicaid/ Who are the intended users? What are some tasks they might need to do? What is the environment in which they will try to accomplish those tasks? Does the technology do a good job of supporting them? 17
Formal methods for assessing usability and improving it 18
Four approaches to usability evaluation and improvement Rapid, catches lots of elementary problems especially at early stages A. Analyst-driven 1. Heuristic evaluation 2. Cognitive walk-through Potential lack of fit to user experience B. User-driven 3. User testing with observation 4. Questionnaires Rich source of information, closely mapped to user needs Slow, expensive, may fail to reach consensus 19
1. Analyst-driven usability evaluation through heuristic evaluation The usability analyst applies a checklist developed by experienced experts • Please open the Zhang checklist article: https: //www. ncbi. nlm. nih. gov/pubmed/14552844 “Heuristic” = “rule of thumb” or reminder 20
2. Analyst-driven usability evaluation through cognitive walkthrough Analyst uses the system to walk through the tasks and analyzes all interactions with the system, focusing on the user's cognitive processes 1. Select and describe the task(s) 2. Determine the correct or best sequence of actions required to complete task 3. Select and describe the goals of user groups 4. Employ system to perform the task, analyzing each interaction in light of Norman’s action cycle 21
Using the Norman action cycle in cognitive walkthrough Determine the function? Tell what actions are possible? Identify the right action(s)? Perform the action? Tell whether the system is in the desired state? Compare the system state with your expectations? Tell what state the system is in? 22
3. User-driven evaluation: User testing with observation Invite representative users to test the system under controlled conditions • • Take notes Use screen capture videorecording software such as Morae or Camtasia Make audiorecordings of users “thinking out loud” (verbal protocol) Make videorecordings of users’ faces and body language 23
Task-based user testing gives the users specific tasks to complete • Examples: • staff member registers a patient • provider places a medication order in an order entry system • patient logs into secure portal and messages physician • • Highly scripted and narrowly focused Tester provides as little help as possible because the goal to discover usability problems, not to help user accomplish task 24
4. User-driven usability research employing usability questionnaires Take a look at 4 questionnaires discussed here: https: //www. researchgate. net/profile/Thomas_Tullis/publication/228609327 _A_Comparison_of_Questionnaires_for_Assessing_Website_Usability/links/0 912 f 50 e 4 fe 12066 fd 000000/A-Comparison-of-Questionnaires-for-Assessing. Website-Usability. pdf 1. What do you think the advantages of using a questionnaire would be? 2. Disadvantages? 3. Which questionnaire did you like best? 25
Why is usability hard to achieve? 1. Designers are not users • • 2. High expertise levels mean highly sophisticated mental models of technology Have an unreasonable belief that they represent users Users are notoriously bad at defining what they want and need • • Limited ability to explain their own mental models and assumptions Tend to explain small tasks, not goals Good at identifying what they do not like! “few users are able to clearly articulate their needs…” Cooper 2007 3. In the design of a technology, the user may not be the only or even the most important stakeholder! 26
Great reads!! • • The Design of Everyday Things/The Psychology of Everyday Things, Donald Norman About Face: The Essentials of Interaction Design, Alan Cooper, Robert Reimann, David Cronin 27
Learning objectives 1. 2. 3. Define system and human factors engineering Define usability Compare 4 methods for studying usability A. Analyst-driven Method 1: Heuristic Method 2: Cognitive walkthrough B. User-driven Method 3: User testing and observation Method 4: Questionnaires 28
- Slides: 28