Usability Testing Week 12 What is usability l

Usability Testing Week 12

What is usability? l Usability means focusing on users l People use products to be productive l Users are busy people trying to accomplish tasks l Users decide when a product is easy to use

What is Usability? l “A function of particular users performing particular tasks in a particular environment” (Smith et al. 68) l The “people who use the product can do so quickly and easily to accomplish their own tasks” (Dumas and Redish 4) l User-centered design, not “user-friendly”

What is Usability Testing? l An empirical study of a product’s usability by observing actual users do real tasks with the product l Involves: Real users l Real tasks l Specific usability goals/concerns l Observing and recording the testing l Data analysis l

Why Usability Testing? l To ensure that the product is usable for your user l Usability testing should be an iterative practice l Completed several times during the design and development life-cycle l End result is an improved product and a better understanding of the users that we’re designing for (Kaufmann 2006)

Usability Testing Components l Pilot Test l Usability Test

Usability Testing Components l Pilot testing Designed to identify hidden issues and eliminate surprises. l Pilot testing tests the methods of your actual study as a trial l Not with actual users l During pilot testing you can make modifications to your final study l

Usability Testing Components l Usability testing (Dumas & Reddish) Goal to improve usability of a product l Participants represent real users l Participants do real tasks l You observe and record what participants do and day l You analyze the data, diagnose the real problems, and recommend changes to fix those problems l

How many people should I test? l Discount l usability methods (Nielson) Suggests 5 Users (we will use one more 6 just to be on the safe side)

Pretest Questionnaire l To Obtain user demographics so that you are able to out biases in experiment that may be caused by age or experience

Task List l Contains multiple tasks or items (for our purposes 6, 12 or 18 tasks) that the user has to find or perform.

Post test Questionnaire l Ben Shneiderman, Design the User Interface

Post test Questionnaire l Written user surveys are a familiar, inexpensive and generally acceptable companion for usability tests and expert reviews. l Acceptance Tests

The Design Continuum System-Centered • Users dumb • Users same as us • Will use regardless • Bells and whistles • Do what they can, not what they should User-Friendly • Consider the audiences • Users will like this • Often draw on stereotypes • Reasoning not necessarily supported • Based on untested profiles and assumptions User-Centered • Users valuable • Users part of the design process • Early focus on users • Iterative • Involves research of/with users • Includes participatory design, contextual inquiry, ethnography, and usability testing

Step 1: User Analysis & Profiles l Who are your actual users? You may need to break your users into typical user categories. Consider: l l l Demographics: age, sex, race, education level, cultural background, socioeconomic status, … Experience level with the product, with products of the same genre, with required technology, . . . Other things: l l l motivation learning style subject matter knowledge location of use physical characteristics people with disabilities or impairments (from color blindness and learning disabilities to more severe disabilities)

Step 1: User Analysis & Profiles Con. l Create user profiles: l l l Break users into clear subgroups Profile/Define the characteristics of each subgroup Choose user profiles to test: l l Ideally users from all major profiles will be tested If limited testing: Choose profiles based on highest number of users in that profile or profiles that you think may have the greatest usability issues

Step 2: Decide what to Test 1. Choose an overall purpose l 2. Determine objectives or what you are testing for. Examples: l l 3. Example: How useable is our new website? Does our search engine provide usable results in the first 5 links returned? Are search results clear to the users? Choose type of test: 1. 2. 3. Performance: Can they do it? Understandability: Can they understand it? Read-and-locate: Can they find it?

Step 2: Decide what to Test con. Select tasks: 4. l l l Consider tasks with a high chance of user failure (complex tasks, one-of-akind tasks, highly abstract or technical tasks) Consider tasks with a high cost of user failure (tasks that require support, like help or support calls, to complete; tasks where data could be damaged or lost) Consider: l First impressions (look and feel) l First tasks l Tasks most performed l Critical tasks l Specific problem areas l New task for the product Select performance objectives (should be individualized for each task) 5. l l Time: How long to complete tasks, to find things, to performance procedures Error/Success: user errors, attempts to do/find something, numbers of times section re-read, if the task was completed successfully

Step 3: Preparing for the Testing 1. 2. Choose order of tasks: start easy, go sequential, or be random Create written test materials: l l l 3. 4. Recruit participants & determine “payment” Define team member’s roles: l l l 5. 6. 7. Task list for users Written welcome speech/ Intro to be read to user Consent forms Observation forms Pre-task and post task questionnaires & interview questions Other materials Facilitator/Briefer (necessary): Often only team member to interact with users Observation recorder/note taker (necessary) Camera operator (optional) Help desk operator (optional) Test administer (optional) Create written test plan Practice: conduct walkthroughs of the testing and if possible pilot test (the pilot test users could even be a team member) Prepare test environment (day of test)

Step 4: Conducting the test 1. Greet & Brief participant: a. b. c. d. e. 2. 3. 4. 5. Read/say welcome Emphasize that you are not testing them, but the product and that they should act as natural as possible Explain think-aloud protocol (if using) Emphasize how user tells you she has completed a task Stress that the testing is “anonymous” Be unbiased (especially the Facilitator/Briefer) Intervene carefully (avoid as much as possible) Observe and record data Debrief user

Step 5: Analyzing the Data 1. Collate data into findings: a. Choose an approach: l l b. Top-down approach: predetermine categories of findings (like navigation, design, terminology) and go through data looking for “hits” Bottom-up approach: put each observation on a sticky note/note card, sort into categories and label categories Determine time and errors/success l l Examine findings for each user, user profile, and task Use analysis techniques such as statistics (even averages help)

Step 5: Analyzing the Data con. 1. Analyze data: a. b. c. 2. Determine cause of problems Determine scope/severity of problems Make recommendations/changes Report Findings

Good Luck & Have Fun! Where to find out more: l Barnum Usability Testing and Research l Barker Writing Software Documentation, Chapter 6 “Conducting Usability Tests” l Hom “General Concepts of Usability Testing” http: //jthom. best. vwh. net/usability/general. htm l