A Comparison of Heuristic Evaluation Cognitive Walkthroughs and

A Comparison of Heuristic Evaluation, Cognitive Walkthroughs and Usability Testing Alfred Kobsa University of California, Irvine

Heuristic Evaluation Normally performed by a team of 3 -5 experts • • Original designers are do not see their own mistakes easily Experts go through guidelines (heuristics) Rate individually, form consensus Can be performed on early designs, mockups and prototypes Pros: • Cheaper than usability experiments • Reasonably effective: Rule of thumb: 1 expert ca. 40% of errors, 2: 50%, 3: 60%, 5: 80% Cons: • HCI experts find mostly local problems (when domain expertise is also present, more and more global problems will be found)

Cognitive Walkthrough Performed by original designers or experts (individuals or groups) • Analyst(s) imagine performing a task and walk through design documents • Analysts try to determine whether users will be able figure out where to go and how to do things in the design • Can be performed based on early designs, mockups and prototypes Pros: • Cheaper than usability experiments • Reveals global errors more readily than heuristic evaluation Cons: • Original designers do not see their own mistakes easily • Experts may need long to understand the task at hand

Usability testing Performed by original designers (or experts) • with users who are representative for the target population • planned in teams, carried out by (parallel) subteams of 1 -4 people Pros: • Reveals more usability problems than other methods • Finds more global problems • Finds more unique problems (? ) Cons: • Time-intensive: several weeks to a few months • Does not detect local/minor problems very well ☛ perform heuristic evaluation and/or cognitive walkthrough first