Usercentered design n n Theme know thy user

User-centered design n n Theme - “know thy user, ” “honor thy user” Involves four major components: ¨ Early focus on user and task ¨ Empirical measurement ¨ Iterative design ¨ Participatory design ISE 412 - 2 1

Norman’s Theory of Action ISE 412 - 2 2

“Gulfs” of Evaluation and Execution ISE 412 - 2 3

Design to “bridge the gulfs” ISE 412 - 2 4

The key n Understanding human behavior, cognitive processes, capabilities, and limitations n Understanding what people are doing, why, where, and under what conditions ¨ To n ISE 412 - 2 explore further … Norman, D. A. (2005) Human-Centered Design Considered Harmful, http: //www. jnd. org/dn. mss/ human-centered_desig. html 5

The Information Processing Model from: Wickens, Lee, Liu, & Becker (2004) An Introduction to Human Factors Engineering (2 nd ed. ). New Jersey: Pearson Prentice Hall. pg. 122 ISE 412 - 2 6

HUMAN FACTORS RESEARCH n What can we measure? ¨ Attitude and motivation ¨ Preferences ¨ Knowledge ¨ Skills and abilities (physical and mental) ¨ Aptitude and potential to perform/succeed ¨ Emotion (happy, sad, mad, excited, tense) ¨ Physiological states ¨ Behavior ¨ Performance ISE 412 - 2 7

RESEARCH METHODS IN HUMAN FACTORS ENGINEERING n Basic vs Applied ¨ Definitions: ISE 412 - 2 8

Dimensions of Research Methodologies: n Tradeoff: Experimental Control vs Real-World Relevance ¨ Degree of Control: n over conditions of observation n over "treatment" (independent) variables ¨ Degree to which behavioral (dependent) variables are representative of some larger population ¨ Will the results "scale up" to complex real-life situations? ISE 412 - 2 9

Research Methodology “In a Nutshell” n Define problem and hypotheses What questions do you want to answer? ¨ What cause/effect relationships do you expect to see? ¨ n Specify the experimental plan ISE 412 - 2 ¨ Independent variables (Treatment): ¨ Dependent variables (Effect): ¨ Experimental design: ¨ Specifics: 10

Research Methodology “In a Nutshell” (cont. ) n Conduct the study ¨ follow the plan you defined ¨ use a “pilot study” if you are unsure of your plan n Analyze the data ¨ Quantitative/objective ¨ Subjective ¨ Protocol ISE 412 - 2 data analyses 11

Research Methodology “In a Nutshell” (cont. ) n Draw conclusions ¨ Were the hypotheses supported by the results of the experiment? n n ISE 412 - 2 Statistically significant? (Usually use p = 0. 05 or 0. 01, but these are not carved in stone!) Recall Type I and Type II errors … If not significant, are there interesting trends in the data? Interaction effects? (e. g. , mouse vs keyboard confounded with type of input required when filling out online forms) ¨ Can you say why? n Will effect the degree to which you can generalize your results. n Can you explain interaction effects, trends that are not statistically significant? 12

An overview of experimental types: Laboratory Experiment n Example: Carter's 1979 study of the effects of various display characteristics on search time using college students in a dark quiet room. Experimental Control Relevance H i g h L o w ISE 412 - 2 13

An overview of experimental types: Field Experiment / Quasi-Experiment n Examples: (1) Comparing new ATC interface with existing using real air traffic controllers in real-life settings (field experiment). (2)Comparing "New Math" to conventional instruction by assigning programs to different classrooms (quasiexperiment). Experimental Control Relevance M o d e r a te ISE 412 - 2 14

An overview of experimental types: Simulation Study n Examples: Flight simulators, power plant control simulators. Experimental Control Relevance H i g h L o w ISE 412 - 2 15

An overview of experimental types: Field Study (Case study) n n Examples: (1) flight crew task analyses conducted in-flight to determine task allocation, workload, etc. prior to specification and design of automated systems; (2) investigators observation and modeling of satellite ground operations Investigator observes and records real-life activities without any experimental manipulations. ¨ n Typically, such a study seeks to articulate how and why some behavioral phenomenon occurs. Generally, should be the first step in investigating human interaction with complex systems. ISE 412 - 2 16

Descriptive Methods n Survey/Questionnaire ¨ Used to gauge attitudes, preferences, etc. , often on a numerical scale n n ¨ ISE 412 - 2 (e. g. , 1=strongly disagree, . . . , 5=strongly agree). Amenable to statistical analyses. Problems: n how to avoid biased questions n Respondents may not be totally truthful 17

Descriptive Methods (cont. ) n Interview ¨ Also used to gauge attitudes, preferences, etc. n ISE 412 - 2 Can delve into the reasons behind those attitudes. ¨ Can be used with a questionnaire to clarify or expand on responses. ¨ Problems: n Same as with survey/questionnaire. n Tricky to analyze. 18

Descriptive Methods n Concurrent Verbal Protocols ¨ n While the subject performs a task (in a laboratory, field, or simulation setting), he/she verbalizes- thoughts, reasons for actions, etc. This is often very illuminating in terms of identifying subject strategies, difficulties, etc. But difficult to analyze and may affect task performance itself. Retrospective Verbal Protocols After the subject has performed a task (in a laboratory, field, or simulation setting), he/she verbalizes thoughts, reasons for actions, etc. This may also be quite interesting, but again is difficult to analyze. Also this relies on the subject's memory, which is prone to error. ¨ Also: critical incident technique, accident analysis. ¨ ISE 412 - 2 19

General Remarks: n n Before you do any experiment, you should have one or more hypotheses in mind that you want to test or explore. If possible, use multiple methods (e. g. , field study and interviews and questionnaires). ¨ n ISE 412 - 2 If you obtain converging evidence from a number of separate sources, you can be more confident that the hypothesis is correct. Consider reliability and validity issues: ¨ Reliability: ¨ Construct Validity: 20

General Remarks (cont. ): n Reliability and validity issues (concl. ): ¨ Internal Validity (for causal or explanatory studies): ¨ External n Validity: Ethical Issues: ¨ Ethical treatment of human subjects n protection from mental and physical harm n privacy n informed consent ¨ Code ISE 412 - 2 of Engineering Ethics 21

Contextual Inquiry (Field Study +) n n A fact-based approach to understanding the reality of users’ goals, processes, and tasks. Puts systems designers directly in the “world” of the user to see, hear, feel, taste, smell, and better understand: ¨ ¨ ¨ n the goals of the user the activities and specific behaviors users engage in to achieve those goals the places in which they occur the tools, supplies, etc. that are used the artifacts in the environment and their meanings etc. Provides rich, context-specific data from which to develop a shared understanding of users that will guide design ISE 412 - 2 22

C. I. vs Field Study Contextual Inquiry Typical Focus (Traditionally) operators, their activities, tools, information requirements, decision making, work environment, etc. , with emphasis on specific tasks or goals. Interactions of people, environments, tools, and artifacts in specific workplaces or doing specific types of work, with emphasis on communication & cultural aspects. Typical Approach Researcher is an observer who tries not to interfere with task performance. Follow-up interviews are used to gather more info. Researcher is a participant and the operator is considered a partner in the process. Questions may be asked during observations, with follow-up interviews as well. Typical Product Task analyses, physical models, information flow models. ISE 412 - 2 Flow models, cultural models, physical models, sequence models, and artifact models. 23

The C. I. Process (Source: Beyer, H. & Holtzblatt, K. (1998) Contextual Design: Defining Customer-Centered Systems. San Diego: Academic Press) n n n Design the research Recruit participants Collect data interview (video taping optional) n introduction n initial interview n contextual interview n wrap-up ¨ research notes ¨ artifacts ¨ “homework”, prework, etc. if applicable ¨ n n ISE 412 - 2 Analyze video, notes, visuals, artifacts, etc. Synthesize results 24

Designing Contextual Studies: General Approach n What will you be investigating? n notepad and pencil/pen are minimum requirement, but usually sufficient ¨ video or audio tape optional ¨ pros and cons … ¨ specific activities, problems, etc. ¨ environments ¨ target population ¨ n Script what questions will you ask in the interview(s)? ¨ are there specific tasks or problems you want to be sure to observe? ¨ what follow-up do you anticipate? ¨ ISE 412 - 2 Equipment & supplies n Prework / Homework ‘depth’ questions you’d like participants to answer before or after the interview ¨ not always necessary, but can give broader perspective of the workplace, goals, attitudes, etc. ¨ NOTE: This applies to all field studies, not just CI. 25

To discuss … n n n What are you investigating? (e. g. , what types of market/customers? , doing what? , etc. ) What kind of questions? Do you want to “shadow” one or more individuals or observe a random sample? ¨ n n Are you going to try to video or audiotape? (If so, you’ll need permission!) REMEMBER! … ¨ ISE 412 - 2 if individuals, do you want them to do any kind of “pre-work”? This is your baseline research, so the broader and more inclusive you can be at this point, the better you will be able to narrow the scope of your project later. 26

Recruiting for a Study n Number depends on the process you’re studying, the target population, etc. ¨ typically 10 -20, but can be less for smaller, more focused design problems ¨ recruit at the high end of your target sample size to cover ‘dropouts’ ¨ n Diverse, representative sample, considering (as appropriate for the process you are supporting): age range ¨ male/female mix ¨ type of work, workplace, etc. ¨ Ethnic heritage ¨ n Once removed from interviewer (friend of a friend) or more ¨ ISE 412 - 2 familiarity breeds too much shared understanding 27

Interview Techniques: Getting Started (5 -10 min. ) DO: n. Come to the interview prepared: ¨video equipment ready to go ¨notes, pen, other materials accessible ¨right hand free to shake hands at the door! DON’T: n n get distracted by side conversations ¨ ask questions about personal matters not related to the study ¨ tell “your” story instead of hearing his/hers ¨ n. Set the participant at ease: ¨introduce yourselves and remind them why you’re there ¨use equipment setup time to “break the ice” n. Select initial interview site ¨in proximity to “work location” or Show up at the door with video “rolling” Fall into the “buddy trap” n Presume or demand starting point, if possible ¨room for participant and interviewer to sit comfortably ¨ask participant where they are most comfortable n. Maintain a polite, professional demeanor ¨“set ISE 412 - 2 the tone” for the next 2 hours 28

Interview Techniques: The Initial Interview (15 min. ) DO: n. Think “broad part of the funnel” ¨start with broad understanding of goals, motives, priorities, etc. ¨get “big picture” understanding of their viewpoint ¨ask for opinions about “tools” and equipment typically used n. Listen for and note things you will want to observe and ask about during the contextual interview n. Ask open-ended questions n. Follow up on interesting points, but maintain pace and direction of the interview n. At the end of this phase, move the interview to the “workspace” ISE 412 - 2 DON’T: n. Lose sight of the purpose and direction of the interview ¨by allowing broad motivational discussions to veer into side issues ¨“gently but firmly” guide the conversation back on course n. Delve into details of what and how ¨these should be addressed in the contextual interview n. Ask leading questions n“Suggest an answer” or finish the sentence for them n. Suggest judgement of the goals, motivations, activities, etc. (either of the participant or anyone else. ) 29

Interview Techniques: Contextual Interview (60 -90 min. ) DO: n. Watch and listen more than talk and ask. ¨Provide occasional feedback (“okay, ” “I see”, etc. ) to let the participant know you’re listening ¨Use questions sparingly to gain clarification, understand “why” something is done the way it is, or to guide the participant’s comments n. Questions, when warranted, should be open-ended and clear ¨“I noticed you ___ - why is/was that? ” ¨“ What are your thoughts (or feelings) about that? ” ¨“You said you sometimes ____ - when would you do that and why? ” ¨“What do you mean by ‘___’? ” “Say some more about that. ” ¨“Tell me what that is for. ” “Tell me what you’re doing now. ” (sparingly) ISE 412 - 2 DON’T: n. Interrupt the flow unnecessarily. ¨Ask yourself, “Can this question wait until she/he’s finished with this? ” ¨On the other hand, some questions HAVE to be asked in context - it’s a judgement call! n. Ask leading questions. ¨“Is the reason you’re doing that because ___? ” ¨“Many people ___. Do you feel the same way? ” ¨“When you say ‘sometimes’, you mean about once a month, right? ” 30

Interview Techniques: Contextual Interview (cont. ) DO: n Take notes, make sketches, write down questions you want to delve into in detail at the end note what is happening, your interpretation, your immediate thoughts, etc. ¨ ALWAYS keep your attention on the participant and her/his activities DON’T: n ¨ ¨ n Pay close attention to key artifacts in the environment: Finish the sentence or suggest an answer n It is not necessary to fill all silences Imply judgement in questions or responses “You don’t do THAT, do you? ” ¨ “Aren’t you afraid of ___? ” ¨ “Don’t you hate it when ___? ” ¨ videotape or photograph individual and groups of items ¨ if there are “disposable” items, ask if you can take them with you ¨ ISE 412 - 2 31

Interview Techniques: The Wrap-Up (15 min. ) DO: n Ask any remaining questions you have about the goals, activities, methods, tools, etc. n Summarize key points you heard during the interview, giving the participant a chance to respond, elaborate, correct your interpretation, etc. n Thank the participant for taking the time and allowing you to learn about his/her activities. n Be sure to take all notes, supplies, equipment, etc. , when you leave. ISE 412 - 2 DON’T: n Overstay your welcome. n Leave any trash behind when you leave. 32

Your turn … n n Write one or two questions you will want to ask during your interview. Test your questions with me as your “participant. ” ¨ Does the participant understand the question? ¨ Does the answer help you to better understand the situation you are investigating? ¨ What follow-up or depth questions are suggested by the participant’s answer? n n Revise your questions accordingly. Use this method to help you refine your interview script. ISE 412 - 2 33