User Interface Evaluation Usability Testing Methods Usability Testing
User Interface Evaluation Usability Testing Methods
Usability Testing Methods ® Conduct experiments to find specific information about a design and/or product. ® Basis comes from experimental psychology. ® Uses statistical data methods ® Quantitative and Qualitative
Usability Testing Methods ® During usability testing, users work on specific tasks using the interface/product and evaluators use the results to evaluate and modify the interface/product. ® Widely used in practice, but not appropriately ® Often abused by developers that consider themselves to be usability experts. ® Can be very expensive and time consuming.
Usability Testing Methods ® Performance Measurement ® Thinking-aloud Protocol ® Question-asking ® Coaching Protocol Method
Usability Testing Methods ® Co-discovery ® Teaching Learning Method ® Retrospective ® Remote Testing
Performance Measurement ® Applicable ® Stages: Design, Code, Test & Deployment ® Personnel Usability Experts, approximately 1. ® Developers, 0. ® Users, 6. ®
Performance Measurement ® Usability Issues Covered Effectiveness: Yes ® Efficiency: Yes ® Satisfaction: No ® ® Quantitative Data is collected. ® Can NOT be conducted remotely. ® Can be used on any system.
Performance Measurement ® What is it? ® Used to collect quantitative data. ® Typically, you will be looking for benchmark data. ® Objectives ® MUST be quantifiable 75% of users shall be able to complete the basic task in less than 30 minutes.
Performance Measurement ® How can I do it? ® Define the goals that you expect users to perform
Performance Measurement ® How can I do it? ® Quantify the goals ® The time users take to complete a specific task. ® The Ratio between successful interactions and errors. ® The time spent recovering from errors. The number of user errors. ® The number of commands or other features that were never used by the user. ® The number of system features the user can remember during a debriefing after the test. ® The proportion of users who say that they would prefer using the system over some specified competitor. ®
Performance Measurement ® How can I do it? ® Get participants for the experiments ® Conduct very controlled experiments ® All variables must remain consistent across users
Performance Measurement ® Problems ® No With Performance Measurement qualitative data.
Thinking-aloud Protocol ® Applicable ® Stages: Design, Code, Test & Deployment ® Personnel Usability Experts, approximately 1. ® Developers, 0. ® Users, 4. ®
Thinking-aloud Protocol ® Usability Issues Covered Effectiveness: Yes ® Efficiency: No ® Satisfaction: Yes ® ® Quantitative Data is NOT collected. ® Can NOT be conducted remotely. ® Can be used on any system.
Thinking-aloud Protocol ® What is it? ® Technique where the participant is asked to vocalize his or her thoughts, feelings, and opinions while interacting with the product. .
Thinking-aloud Protocol ® How can I do it? ® Select the participants, who will be involved? ® Select the tasks and design scenarios. ® Ask the participant to perform a task using the software.
Thinking-aloud Protocol ® How can I do it? ® During the task, ask the user to ® Thoughts, opinions, feelings, etc. vocalize ® http: //www. cs. umd. edu/~zzj/Think. Alo. htm
Thinking-aloud Protocol ® Problem With Thinking-Aloud Protocol ® Cognitive Overload Can you walk & chew gum at the same time? ® Asking the participants to do too much. ®
Question-asking Protocol ® Applicable ® Stages: Design, Code, Test & Deployment ® Personnel Usability Experts, approximately 1. ® Developers, 0. ® Users, 4. ®
Question-asking Protocol ® Usability Issues Covered Effectiveness: Yes ® Efficiency: No ® Satisfaction: Yes ® ® Quantitative Data is NOT collected. ® Can NOT be conducted remotely. ® Can be used on any system.
Question-asking Protocol ® What is it? ® Similar ® Instead to Thinking-aloud protocol. of participant saying what they are thinking, the evaluator prompts the participant with questions while using the system.
Question-asking Protocol ® How can I do it? ® Select the participants, who will be involved? ® Select the tasks and design scenarios. ® Ask the participant to perform a task using the software.
Question-asking Protocol ® How can I do it? ® During the task, ask the user to questions about the product ® Thoughts, opinions, feelings, etc. ® http: //www. cs. umd. edu/~zzj/Question. htm
Question-asking Protocol ® Problem With Thinking-Aloud Protocol ® Cognitive Overload++ Can you walk, chew gum & talk at the same time? ® Asking the participants to do too much. ® Added pressure when the evaluator asks questions. ® Can be frustrating on novice users. ®
Coaching Method ® Applicable ® Stages: Design, Code, Test & Deployment ® Personnel Usability Experts, approximately 1. ® Developers, 0. ® Users, 4. ®
Coaching Method ® Usability Issues Covered Effectiveness: Yes ® Efficiency: No ® Satisfaction: Yes ® ® Quantitative Data is NOT collected. ® Can NOT be conducted remotely. ® Can be used on any system.
Coaching Method ® What is it? ®A system expert sits with the participant and acts as a coach. ® Expert ® The answers the participant’s questions. evaluator observes their interaction.
Coaching Method ® How can I do it? ® Select the participants, who will be involved? ® Select the tasks and design scenarios. ® Ask the participant to perform a task using the software in the presence of a coach/expert.
Coaching Method ® How can I do it? ® During the task, the user will ask the expert questions about the product ® http: //www. cs. umd. edu/~zzj/Coaching. htm
Coaching Method ® Problem ® In With Coaching Method reality, there will not be a coach present. ® This is good for creating a coaching system, but not for evaluating the interface.
Co-Discovery Learning ® Applicable ® Stages: Design, Code, Test & Deployment ® Personnel Usability Experts, approximately 1. ® Developers, 0. ® Users, 6. ®
Co-Discovery Learning ® Usability Issues Covered Effectiveness: Yes ® Efficiency: No ® Satisfaction: Yes ® ® Quantitative Data is NOT collected. ® Can NOT be conducted remotely. ® Can be used on any system.
Co-Discovery Learning ® What is it? ® Two test users attempt to perform tasks together while being observed. ® They are to help each other in the same manner as they would if they were working together to accomplish a common goal using the product. ® They are encouraged to explain what they are thinking about while working on the tasks. ® Thinking Aloud, but more natural because of partner.
Co-Discovery Learning ® How can I do it? ® Select the participants, who will be involved? ® Select the tasks and design scenarios. ® Ask the participants to perform a task using the software.
Co-Discovery Learning ® How can I do it? ® During the task, the users will help each other and voice their thoughts by talking to each other. ® http: //www. cs. umd. edu/~zzj/Codiscov. htm
Co-Discovery Learning ® Problem ® Neither ® With Co-Discovery Learning is an expert The blind leading the blind.
Teaching Method ® Applicable ® Stages: Design, Code, Test & Deployment ® Personnel Usability Experts, approximately 1. ® Developers, 0. ® Users, 4. ®
Teaching Method ® Usability Issues Covered Effectiveness: Yes ® Efficiency: No ® Satisfaction: Yes ® ® Quantitative Data is NOT collected. ® Can NOT be conducted remotely. ® Can be used on any system.
Teaching Method ® What ® You ® Ask is it? have 1 participant use the system. the participant to teach a new novice participant how to use the system.
Teaching Method ® How can I do it? ® Select the participants, who will be involved? ® Select the tasks and design scenarios. ® Ask the 1 st participant to perform a task using the software. ® Ask the 1 st participant to teach a new participant.
Teaching Method ® How can I do it? ® Observe their interactions. ® http: //www. cs. umd. edu/~zzj/Teaching. htm
Teaching Method ® Problem ® Neither With Teaching Method is an expert The blind leading the blind. ® Possible to discover some interesting things about the learn-ability of your interfaces. ®
Retrospective Testing ® Applicable ® Stages: Design, Code, Test & Deployment ® Personnel Usability Experts, approximately 1. ® Developers, 0. ® Users, 4. ®
Retrospective Testing ® Usability Issues Covered Effectiveness: Yes ® Efficiency: Yes ® Satisfaction: Yes ® ® Quantitative Data can be collected. ® Can NOT be conducted remotely. ® Can be used on any system.
Retrospective Testing ® What ®A is it? videotape of the session is observed by the usability expert and the participants.
Retrospective Testing ® How can I do it? ® Select the participants, who will be involved? ® Select the tasks and design scenarios. ® Use one of the usability testing methods that we have discussed. ® Videotape the session.
Retrospective Testing ® How can I do it? ® Review the videotape with the users. ® http: //www. cs. umd. edu/~zzj/Retrospe. htm
Retrospective Testing ® Problem With Retrospective Testing ® Extremely time consuming!
Remote Testing ® Applicable ® Stages: Design, Code, Test & Deployment ® Personnel Usability Experts, approximately 1. ® Developers, 0. ® Users, 5. ®
Remote Testing ® Usability Issues Covered Effectiveness: Yes ® Efficiency: Yes ® Satisfaction: Yes ® ® Quantitative Data can be collected. ® Can be conducted remotely. ® Can be used on any system.
Remote Testing ® What is it? ® The participants are separated from the evaluators. ® No formal observation. ® No usability lab.
Remote Testing ® How can I do it? ® Give the product/software to participants. ® Collect information about how they use your software/product. ® Methods Same-Time Different Place ® Different-Time Different Place ®
Remote Testing ® How can I do it? ® Camtasia, Look@Me, Snag. It ® Usability Logger ® http: //www. usabletools. com/ ® Journaled Sessions ® http: //www. cs. umd. edu/~zzj/Retrospe. htm
Remote Testing ® Problem ® The With Remote Testing evaluator is not there. ® Can’t observe facial expressions. ® Great for Web based systems.
Usability Testing Methods ® Select the method that works best for you. ® Select the method that fits your implementation. ® Be thorough during your experiments. ® The more data, the better.
Usability Testing Methods ® Hawthorne ® The Effect tendency for people to improve their performance after any change when they know their performance is being studied.
- Slides: 56