Chapter 8 Usability Studies and Usability Testing Human

Chapter 8 Usability Studies and Usability Testing Human Performance Engineering Robert W. Bailey, Ph. D. Third Edition

Usability Studies= help to make good initial decisions and may be collected any time in the design process. Usability Tests = used to evaluate design decisions after they are implemented.

• The earlier in the system development process that data is collected, the more valuable the information will be. • Cost justifying data collection decisions • Using someone else’s data • Sampling 1. Random Sampling 2. Subjective Sampling

Usability Studies - Observations - Questionnaires - Interviews Face to Face Telephone

Usability Testing: Should evaluate human performance and user preference.

Usability Engineering = combining prototyping and usability testing • Set acceptable usability levels (goals) • Conduct usability testing • Make required changes Repeat until goals are met

Study: stimulus –response compatibility v Group 1= pressed a left key for a left stimulus and a right key for a right stimulus (direct mapping) v Group 2= pressed a left key for a right stimulus and a right key for a left stimulus (indirect mapping) After 2400 trials, group 1 still showed an advantage in both speed and accuracy over group 2.

Measuring Usability What is the relationship between Performance and Preference? 71% perform well and like a system or perform poorly and dislike a system. 29% perform well and dislike a system or perform poorly and like a system. (Nielsen & Levy, 1994) *** Preferences of system designers do not always lead to the best performance-related design decisions.

Table 8 -1 Preferences and Performance Scores for Alternative User Interfaces Designer Average Preferences (%) per list (seconds) ___________________________________ Typed entry 5 percent 9. 9 One-level menu 18 percent 11. 1 Two-level menus 40 percent 12. 2 Multiple menus 37 percent 14. 7 ___________________________________ A follow-up study reported similar results (Bailey, 1993). A study was conducted to determine the best screen-based control for a new application. Forty-six computer professionals practiced using four different methods and then selected the one that they thought would elicit the fastest performance. The same people then participated in a performance test. The results are shown in Table 8 -2. Again, a relatively large percentage of the designers preferences did not elicit the fastest performance.

Table 8 -2 Preferences and Performance Scores for Alternative Widgets Percent Preferring Average Time to Each Widget Perform (seconds) __________________________ Open selection lists 76 percent 58 Drop-down selection lists 13 percent 90 Radio buttons 7 percent 53 Typed entry 4 percent 72 __________________________

Givon & Goldman, 1987: Designer: Is window A different from window B? User: I can’t tell. Designer: Which do you prefer? User: Window B

Types of Usability Tests: Formal = using theories to make predictions. Inspection= walkthroughs, questionnaires Performance = well-controlled simulations of people performing activities using system prototypes. Operational = methods used after a product is delivered.

Predicting Human Performance * Is the Principle purpose of usability testing. * Fidelity= variable that largely determines the predictive power of a usability test. * The fidelity of test conditions is directly related to cost and inversely related to the ease with which meaningful changes can be made to the system.

System Performance • Accuracy Objective • Cost-effective performance • User acceptance