Evaluation approaches Text p 340 347 Text p

Evaluation approaches Text p 340 - 347 Text p 590 - 595

Evaluation approaches • Usability testing • Field studies • Analytical evaluation • Combining approaches • Opportunistic evaluations

Usability testing • Dominant approach in the 80’s • Involves measuring typical users performance on typical tasks • Note the number and types of errors that users make and record the time it takes them to complete the task • Recorded on video/audio with interactions often recorded • User satisfaction questionnaires and interviews are also used to elicit user opinions • Test environment and format of testing is controlled by the valuator

Field studies • Done in natural setting with the aim of understanding what people do naturally and how products mediate their activities • Help identify opportunities for new technology • Establish the requirements for design • Facilitate the introduction of technology or how to deploy existing technology in new contexts • Evaluate technology

Analytical evaluation • 2 categories: – inspections • heuristic evaluation • Walkthroughs – theoretical based models • keystroke model • Fitt’s Law

Analytical evaluation Heuristics • Based on common sense knowledge and usability guidelines • Originally developed for screen based applications • Adapted to make new sets for evaluation web based products, mobile devices and computerised toys • Care needs to be taken as designers are sometimes led astray by findings from heuristic evaluation that are mot as accurate as they first thought

Analytical evaluation Cognitive Walkthroughs • Involve simulating a users problem solving process at each step in the human computer dialog and checking to see how users progress from step to step in these interactions • Focus on evaluation designs for ease of learning • Other walkthroughs developed: pluralistic

Analytical evaluation Predictive models • Have been primarily used for comparing the efficacy of different interfaces for the same application • Eg the optimal arrangement and location of features on the interface base • The keystroke model provides numerical predictions of user performance • Fitt’s Law predicts the time it takes to reach a target using a pointing device

Characteristics of approaches Usability testing Field studies Analytical Users do task natural not involved Location controlled natural anywhere When prototype early prototype Data quantitative qualitative problems Feed back measures & errors descriptions problems Type applied naturalistic expert

Evaluation approaches and methods Method Usability testing Field studies Observing x x Asking users x x x Asking experts Testing Modeling Analytical x x x