Testing through user observations User Observation Guidelines for

Testing through user observations User Observation: Guidelines for Apple Developers, Kathleen Gomoll & Anne Nicol, January 1990 Notes based on: http: //pages. cpsc. ucalgary. ca/~sheelagh/courses/481/assig nments/2 Usability/apple_guidelines_usability. html

User testing n User testing methods include: n Focus groups and interviews n n Surveys with paper or web-based forms n n n Pros and cons? Timed performance tests or keystroke protocols Controlled laboratory experiments n n Pros and cons? User observation…

User observation n n Watching and listening carefully to users as they work with a product Why do it? n n Gathers objective data about users Relatively quick compared to other methods

Preparing for an observation n Set objectives: figure out what you're testing and what you're not. n n n Limiting scope of test makes it more likely you’ll get information that helps you solve a specific problem What might be the objectives for your project? Design the tasks: specific tasks that you want real users to accomplish n n Write them out as short, simple instructions What tasks would your user accomplish?

Preparing an observation (2) n n n Plan recording: video or audiotape? Why? Determine setting: ideally, a quiet, enclosed room with a desk, appropriate hardware and software, video camera, and two microphones Find representative users: people who have the same experience level as the typical user for your product n n Pairs of users is sometimes good: they usually talk more than people working alone Who might you get as your test users?

Conducting an observation (1) n n Introduce yourself Describe purpose of observation: n n n Set participant at ease You're helping us by trying out this product in its early stages We're looking for places where the product may be difficult to use. If you have trouble with some of the tasks, it's the product's fault, not yours. Don't feel bad; that's exactly what we're looking for. If we can locate the trouble spots, then we can go back and improve the product. Remember, we're testing the product, not you

Conducting an observation (2) n n n Tell the user it’s OK to quit at any time? Why? Explain any physical equipment. Such as? Explain how to “think aloud” Why? n n n Help discover expectations for your product, intentions and problem solving strategies We get a great deal of information from informal tests if we ask people to think aloud as they work through the exercises It may be a bit awkward at first, but it's really very easy once you get used to it If you forget to think aloud, I'll remind you Would you like me to demonstrate?

Conducting an observation (3) n Explain that you cannot provide help. Why not? n n Describe the task and introduce the product n n Because we want to create a most realistic situation Go ahead and ask questions for recording on tape I’ll answer any questions you may have afterward Give the participant written instructions for the tasks Don’t demonstrate what you’re trying to test! What tasks might you want to observe? Ask if there any questions before you start

Conducting an observation (4) n Conclude the observation n n Explain what you were trying to find out during the test. Set the participant at ease Answer any remaining questions Discuss any interesting behaviors you would like the participant to explain Use the results! n n n Don’t blame users for mistakes. Why not? Document your results and revision plans When should you do your observation?