Lecture 3 Discovering what people cant tell you

Lecture 3: Discovering what people can't tell you: Contextual Inquiry and Design Methodology* Brad Myers 20 -790 Human-Computer Interaction in e. Commerce *These lecture notes based in part on notes created by Professors Bonnie John and Ken Koedinger 1

Notes n Turn in Homework 1 today n n n Hardcopy and email to: bam+hw@cs. cmu. edu Start on Homework 2 today Note distribution of number of points for the different homeworks n Can tell what is most important: studies, not implementation 2

Contextual Design n n One method for organizing the development process We teach it to our MS and BS students Seems to be very successful Described in book: n n H. Beyer and K. Holtzblatt. 1998. Contextual Design: Defining Customer-Centered Systems. San Francisco, CA: Morgan Kaufmann Publishers, Inc. ISBN: 1558604111. http: //www. incent. com/ 3

Common HCI methods in the software lifecycle System Formulation e. g. , interviews, Requirements questionnaires, Contextual Inquiry Architectural Design Detailed Design e. g. MHP, GOMS, Heuristic Evaluation, Cognitive Walkthrough, Rapid prototyping + Think-aloud testing, Controlled experiments. Implementation System Test and Deployment e. g. , Think-aloud Usability Testing, Log analysis, Contextual Inquiry, Controlled Experiments 4

HCI methods in the design process Design Ideas Contextual Inquiry Tasks Heuristic Evaluation Prototyping GOMS Cognitive Walkthrough Analytic Methods Think-Aloud Usability Studies Empirical Methods New Design Ideas Contextual Inquiry is used in beginning of design process n 5

User Study Methods & the different fields they come from n Questionnaires, Interviews n n Focus Groups n n Experimental Psychology Think-aloud protocols n n Business, marketing technique Laboratory studies n n Social Psychology Cognitive Psychology Participant/observer ethnographic studies n Anthropology 6

Contextual Inquiry & Design n Contextual Inquiry n n An evolving method A kind of “enthnographic” or “participatory design” method Combines aspects of other methods: n Interviewing, think-aloud protocols, participant/observer in the context of the work Part of “Contextual Design” n Also includes models to describe results 7

“Contextual Inquiry” n n Interpretive field research method Depends on conversations with users in the context of their work n n Recommends “direct observation” when possible When not possible n n cued recall of past experience, or recreation of related experience Used to define requirements, plans and designs. Drives the creative process: n n In original design In considering new features or functionality 8

Why Context? n Design complete work process n n n Integration! n n Fits into “fabric” of entire operations Not just “point solutions” to specific problems Consistency, effectiveness, efficiency, coherent Design from data n n Not just opinions, negotiation Not just a list of features 9

Who? n Interviewers: “Cross-functional” team n n n Designers UI specialists Product managers Marketing Technical people Customers n n Between 6 – 20 Representative of different roles 10

Where? n Design is a group activity n n n Shared across different groups Useful to have a designated, long-term space for the project team Interviews at customer site 11

Key Concepts in Contextual Inquiry n Context n n Partnership n n Work with users as co-investigators Interpretation n n Understand users' needs in their work or living environment Assigning meaning to the observations Focus n Listen and probe from a clearly defined set of concerns 12

Context n n Definition: n The interrelated conditions within which something occurs or exists Understand work in its natural environment n Go to the user n Observe real work n Use real examples and artifacts n “Artifact”: An object created by human workmanship n Interview while she/he is working 13

Key distinctions about context Interviews, Surveys, Focus Groups Summary data & abstractions Contextual Inquiry Subjective Objective Limited by reliability of human memory Spontaneous, as it happens What customers think & say they want What customers actually need Ongoing experience & concrete data 14

Elements of User's Context: Pay Attention to all of these n n n n User's work space User's work intentions User's words Tools used How people work together Business goals Organizational and cultural structure 15

Standard Contextual Inquiry: Work-based Interview Use when: n Product or process already exists n Or a near competitor’s n User is able to complete a task while you observe n Work can be interrupted 16

What to Record n n n Work flow and tasks Work opportunities and problems Tool opportunities and problems Design ideas and validations User's words n n Ask for elaboration, explanation Your observations 17

Interview Note-Taking n When to take notes? n n n Any observations not being recorded Note taking can help you pay closer attention Notes lead to faster turn-around Do not let it interfere with interviewing How to record? n n What the user says – in quotes What the user does – plain text Your interpretation – in parentheses Write fast! 18

Reasons for variation on the standard work-based interview n Different goals n n Designing a known product n Know the competition Addressing a new work domain n Study what replacing Designing for a new technology Types of tasks that make work-based inquiry impractical n n n Intermittent – instrument or keep logs Uninterruptible – video and review later Extremely long – point sample and review 19

Some Alternative Contextual Inquiry Interview Methods n For intermittent tasks n n n For uninterruptible tasks n n Artifact walkthrough New technology within current work n n Post-observation inquiry For extremely long or multi-person tasks n n In-context cued recall Activity logs Future Scenario Prototype or prior version exists n Prototype/Test drive 20

Partnership n n Definition: n A relationship characterized by close cooperation Build an equitable relationship with the user Suspend your assumptions and beliefs Invite the user into the inquiry process 21

Why is Partnership Important? n Information is obtained through a dialog The user is the expert. Not a conventional interview n Alternative way to view the relationship: n n Master/Apprentice n n The user is the “master craftsman” at his/her work You are the apprentice trying to learn 22

Establishing Partnership n n n Share control Use open-ended questions that invite users to talk: n "What are you doing? " n "Is that what you expect? " n "Why are you doing. . . ? " Let the user lead the conversation Listen! Pay attention to communication that is non-verbal 23

Making your interpretations explicit Procedure we recommend (not in Beyer & Holtzblatt’s writings) n n n Label “facts” with the line number of the transcript or time on the tape Interpretations are then anything not labeled that way If you do this all the way through, the links back to facts are explicit and the intermediate hypotheses and ideas can always be challenged 24

Analysis n n In the moment: Simultaneous data collection and analysis during interview Post interview: n n Using notes, tapes, and transcripts Analysis by a group: n n n Integrates multiple perspectives Creates shared vision Creates shared focus Builds teams Saves time 25

Focus n n Focus is a perspective We always have an entering focus Better to make it explicit Characteristics of focus: n n n set of pre-conceived assumptions and beliefs reveals and conceals “Show me how you do …” n n Decide what to ask about Still use general questions n Not an interview! 26

Setting Focus § § Form a team of stakeholders Brainstorm: questions, assumptions, design ideas. § § § Each group member brainstorms individually Group meets and brainstorms Delay evaluation during brainstorming Record the items generated Prune questions § Defer Qs that participants cannot answer § Conclusions about other peoples’ experiences § How large is the market? § Would you buy this product? 27

Setting and using focus Focus setting meeting Brainstorm questions Organize Practice for interview Select focus Generate new openended questions Practice staying focused Don’t reuse old, closed questions Test your understanding of focus 28

Example of CI n Video of sample session with a e. Commerce site: http: //www. cs. cmu. edu/~bam/uicourse/EHCIcontexualinquiry. mpg n Issues to observe n n Interview of work in progress, in “context” Actual session of doing a task n n n Not an interview asking about possible tasks, etc. Questions to clarify about routine, motivations n Why do certain actions: need intent for actions n Notice problems (“breakdowns”) Notice what happens that causes users to do something (“triggers”) n E. g. appearance of error messages, other feedback, external events (phone ringing), etc. 29

Screen shots of important points in video http: //www. cs. cmu. edu/~bam/uicourse/EHCIcontexualinquiry. Screens. ppt 30