CS 594 Empirical Methods in HCC Structuring the

CS 594: Empirical Methods in HCC Structuring the Research Inquiry Dr. Debaleena Chattopadhyay Department of Computer Science debchatt@uic. edu debaleena. com hci. cs. uic. edu

1 Research Ethics in HCI 2 Research Design

1 Research Ethics in HCI

Before we jump in… • Observing classes occurring in Continuum to design a new tool for classroom instruction • Sentiment analysis from public Instagram posts during Nov 2016 US elections • For an insurance company funded project, studying Fitbit use of insured users • Analyze type of and frequency of users’ posts after showing negative news stories on their timeline. Would you need IRB approval?

What is Ethics? (vs. Morals and Values) • A person who knows the difference between right and wrong and chooses right is moral. • Ethics are moral values in action. • A process that guides (a researcher) in making decisions in a situation with conflicting goals • Ethics is always context dependent and therefore universal guidelines cannot be given

Law, Policy, and Ethics Legal Institutional Ethical • What is codified in institutional policies and procedures is typically a higher standard than merely legal, and what is ethical is a higher standard than what is allowed by policy or law. • Laws shift at political boundaries, policies shift at institutional boundaries, and ethics shift at cultural boundaries.

Some Different Approaches to Ethics Consequence ethics What will result from decisions? On which time scale, implications to whom. . ? Principle ethics Decisions are guided by generally approved principles Good scientific practice Virtue ethics What is good research, what kind of motives and goals does it have? What is a good graduate student like?

Why should we care about ethics? • The Nuremburg War Trials • The Belmont Report • The Milgram Experiment We show, via a massive (N = 689, 003) experiment on Facebook, that emotional states can be transferred to others via emotional contagion, leading people to experience the same emotions without their awareness. We provide experimental evidence that emotional contagion occurs without direct interaction between people (exposure to a friend expressing an emotion is sufficient), and in the complete absence of nonverbal cues.

Type of Ethical Issues in HCC • Human subject research • Design research • Internet research

Ethics in Conducting Human-Subject Studies What is human subject research? – Human behavior is examined (e. g. experiments) – Personal information is collected (e. g. , questionnaires & interviews) – Products of human behavior are examined (e. g. , web sites, source code)

Different types of Human-Subject Studies • Observational study – Online – Offline • Experimental study Participatory design in partnership with professional engineers seems like a far cry from anthropologists studying a tribal culture or medical researchers trying out a new antibiotic or medical device. Privacy depends on the idea of “reasonable expectations”

Basic Ethical Principles • Respect for persons • Beneficence • Justice The ethics of a study and the research value of a study are inseparable concepts.

Basic Ethical Principles (cont. . ) Research ethics is about balancing risk versus benefit. Research means “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge. ”

Basics of the IRB process • • CITI training Exempt review Expedited review Full review • What is your target population? • What kind of interventions will you be conducting? • Keep careful research records. IRBs may audit a study to verify compliance.

Ethics in Conducting Online Studies • If information is published online, you are not interacting with its author and the information is not private. • You may not quote online posting of participants that can be found using a search engine. • However, should you decide to ask the author a question about their online posting, then you have interacted and are indeed engaged in human subjects research.

Ethics in Conducting Online Studies (cont. . ) • Problems with internet-based recruiting – Disturbing the environment – Sense of belonging for exploring sensitive topics • Amazon Mechanical Turk – Workers often receive compensation less than the legal minimum wage in the USA – Basic principle of respect?

Design Ethics • HCI research involves creating new artifacts that reshape the human experience. • Design is intervention, an intentional effort to create change. • HCI is increasingly engaging with matters of social change that go beyond the immediate qualities of interaction. In doing so, HCI takes on scientific and moral concerns.

Take Away • Ethical research on human subjects in any field involves balancing the potential benefits with potential harms. • Traditional human subjects research seeks to protect vulnerable research participants by anonymizing them in written accounts. • Analyzing data available on the Internet without a password does not technically constitute human subjects research. • And emerging technologies will inevitably create new challenges that have not yet been named.

2 Research Design

Research Design • • Purpose of research Correlation ≠ Causation Units of Analysis How to design a research project What to measure? Conceptualization Operationalization Reliability and validity of metrics

Before we jump in… • • Write down a research question. What is the unit of analysis? What do you want to measure? How will you measure it? we will revisit this at the end of this section

Purpose of Research • Exploration • Description • Explanation

Nomothetic Causality • When doing research for explanation: – Correlation ≠ Causation • Criteria for Nomothetic Causality – The variables must be correlated (correlation) – The cause takes place before the effect (time order) – The variables are nonspurious. ( the effect cannot be explained in terms of some third variable)

Nomothetic Causality (cont. . )

Units of Analysis • What are you studying? – – Individuals Groups Organizations Family • Units of analysis in a study are usually also the units of observation. • Important for the generalizability of your research results

Types of Studies (as in time) • Cross-Sectional Studies • Longitudinal Studies

How to Design a Research Project • Find a research topic that interests you; the narrower the better • Conceptualize; form a research question • Choose a research method • Operationalize; what to measure • Measure • Analyze • What conclusions can be drawn?

Elements of a Research Proposal • • Problem or Objective Literature Review Identify Subjects for Study What are the key variables in your study? Data-Collection Methods Analysis Timeline

What will you be measuring? • Direct observables (e. g. , height) • Indirect observables (e. g. , workload using a NASA TLX scale) • Construct (combining one or more direct or indirect observables)

Conceptualization • Conceptualization, then, produces a specific, agreed-on meaning for a concept, for the purposes of research. • This process of specifying exact meaning involves describing the indicators we’ll be using to measure our concept and the different aspects of the concept, called dimensions.

Conceptualization (cont. . ) • Indicator: An observation that we choose to consider as a reflection of a variable we wish to study. E. g. , attending religious services might be considered an indicator of religiosity. • Dimension: A specifiable aspect of a concept. E. g. , “Religiosity, ” for example, might be specified in terms of a belief dimension, a ritual dimension, a devotional dimension, a knowledge dimension, and so forth.

Operationalization An operational definition specifies precisely how a concept will be measured—that is, the operations we choose to perform. E. g. , , efficiency ~ time on task, accuracy ~ error count Conceptualization Nominal definition Operational definition Measurements in the real world

Operationalization: level of measurmerent • • Nominal (e. g. , sex) Ordinal (e. g. , socioeconomic status) Interval (e. g. , Fahrenheit) Ratio (e. g. , age)

Criteria of Measurement Quality • Reliability: That quality of measurement methods that suggests that the same data would have been collected each time in repeated observations of the same phenomenon. For a survey, the question “Did you attend religious services last week? ” would have higher reliability than the question “About how many times have you attended religious services in your life? ” • Validity: A term describing a measure that accurately reflects the concept it is intended to measure. For example, your IQ would seem a more valid measure of your intelligence than would the number of hours you spend in the library.

Valid and Reliable

Let’s revisit… • • Write down a research question. What is the unit of analysis? What do you want to measure? How will you measure it? how would you modify your answers?

Upcoming: • Post your research introductions on Piazza. – DUE Aug 31, 11: 59 pm CST • CITI IRB training – DUE Sep 5, 11: 59 pm CST • Start thinking about your course projects.