Experiment COMM 420 8 fall 2007 Nan Yu

Experiment COMM 420. 8 fall 2007 Nan Yu 1

Experiment How do we know that IV is truly related to DV? An experiment provides the strongest possible evidence that two variables are related – stronger than field studies or surveys. 2

Examples l Imaging that our research question is What is the relationship between “modality of the ads (copy v. visual)” and “liking of the ad” 3

How do we know? – experiment How do you like the ad? 1 2 3 4 5 6 7 Not at all very much Copy only Liking: 3. 5 35 people were random assigned to read a copy-only ad of a product Picture only 35 people read were random assigned to a picture-only of a ad of the same product Liking: 6. 3 significant higher 4

True Experiment A true experiment has 4 characteristics: ¡Manipulated independent variables ¡Random assignment ¡Experimental conditions (other than manipulated IVs) are held constant or as similar as possible. ¡DVs are systematically measured. 5

Manipulated IVs l Variables of interest (causes) are directly manipulated. l Different groups of participants will be exposed to different levels of the IV(s). l In true experiments, the investigator administers or manipulates the hypothesized cause as a treatment directly observe its effects. 6

Example of manipulated IV IV: Brand image (manipulated) DV: Purchase intention (measured) 7

Example of manipulated IV IV: Exposure to different body size (manipulated) DV: Perceived body type (measured) 8

Random Assignment l Each person must have an equal probability of being assigned to any of the experimental conditions. 9

Random Assignment (cont. ) Why we need random assignment? l Because we want to make sure that the differences between groups after they were exposed to different levels of the IV are due to the IV effect on the DV, and not to the differences between groups. 10

Random Assignment (cont. ) experimenter Treatment A • Flipping a coin • Pull names from a thoroughly shaken box. • Random numbers table Treatment B Participants come in random orders, and experimenter randomly assign them to treatment A or treatment B. 11

Establishing causation l A change in the value of IV must result in a change in the value of DV. l IV comes in before DV l Third variables are ruled out. There are no alternative explanations for the established relationship. 12

What is a third variable? Third variable refers to a variable that causes changes of both IV and DV. 13

Example of third variable Town size Number of churches Number of bars 14

Example of third variable (cont. ) Economic condition Amount of usage of computer grade 15

Example of third variable (cont. ) Heat Ice-cream consumption Crime rate 16

Third variable l When a third variable is discovered, one can not establish the causation between two variables. l But you can still say that the IV is related (or correlated) to DV. 17

Experimental Design 18

Imaging that our research question is What is the relationship between “exposure to violent TV programs” and “aggression” 19

How do we know? –two-group experiment Randomly assign participants to the two conditions. Control group no exposure to violent TV programs Scales measuring aggression intention 1. 5 Treatment group 1 -hour exposure to violent TV programs 3. 5 higher 20

More Complex Design True experimental design All of them are treatment groups Randomly assign participants to the two conditions. 1 -hour exposure to violent TV programs Scales measuring aggression intention 2. 5 low 3 -hour exposure to violent TV programs 3. 5 medium 2 -hour, for three times exposure to violent TV programs high 6. 5 highest 21

Factorial Experimental Design (more than one IV) Randomly assign participants to the six conditions. Independent Variables Dependant variables In a dark room aggression brightness In a bright room low Exposure to violent TV programs medium high This is a 2 X 3 factorial design 22

Factorial design (cont. ) dark bright Low exposure 2 1 Medium exposure 3 2 High exposure 5 3 Value of measured aggression intention 23

In-class Demo 1 If a researcher designed a study like this, what kind of experiment design is it? How many groups (conditions) does the study have? DV: Liking toward the picture A 2 X 2 factorial design 4 conditions 24

Steps in a true (full) experiment study l Formulate a hypothesis or a research question. l Manipulate IV(s) (stimulus). l Pretest the stimulus l Recruit participants l Present to the participants a cover story l Randomly assign participants to a treatment group or to the control group. l Introduce treatment or intervention l Measure the dependent variable (s) l Debriefing 25

Internal vs. External Validity of Experimental Studies l When a true experiment (with full control by the researcher) is carried out properly, the study has high internal validity. l Changes in the DV were probably caused by (not merely related to or correlated with) the treatment. l However, experiments have lower external validity. ¡What is the reason for this? 26

Threats to Internal Validity l History ¡ History confound refers to any IV, other than the treatment, that: l Occurs between pretest and posttest in an experiment. l Affects the experimental groups differently. l Maturation ¡ Maturation confound refers to the fact that people in an experiment grow older, or get more experienced, while you are trying to conduct an experiment. 27

Threats to Internal Validity (cont. ) l Mortality (attrition) ¡ Mortality confound refers to the fact that individuals might not complete their participation in an experiment. l Testing ¡ Testing confound occurs in laboratory and field experiments when subjects get used to being tested for indicators on DVs. l Instrumentation ¡ Instrumentation confound results from changing measurement instruments or observers. 28

Threats to Internal Validity (cont. ) l Selection bias ¡Choosing participants is a major threat to validity in quasi-experiments, where random assignment doesn’t work. l Regression to the Mean ¡Is a confound that can occur when one studies groups that have extreme scores on a DV. ¡No matter what the treatment is, over time one would expect the extreme scores to become more moderate, just because there’s nowhere else for them to go. 29

Threats to Internal Validity (cont. ) l Demand Characteristics ¡Confounds that occur as reactions to experimental conditions ¡Awareness of the research purpose l Diffusion of treatments ¡Occurs especially in field experiments, when a control group cannot be prevented from receiving the treatment in an experiment. 30

Other types of experimental design (not true experiment) 31

One-Shot Case Study Design l Single group of individuals measured on DV after a treatment has taken place l No causation effects can be inferred TIME 1 Random Assignment Treatment group none TIME 2 Pretest Treatment Posttest IV DV 32

One-Group Pretest-Posttest Design l Only one group involved and the DV is measured before and after the treatment. l In this case, the pretest almost acts like a control group. However, the experimenter has to deal with maturation, testing and instrumentation, mortality effect, etc. TIME 1 Treatment group TIME 2 Random Assignment Pretest Treatment Posttest none Y 1 MX Y 2 33

Nonequivalent Control (Static) Group Comparison (Classic Design without Randomization) l No random assignment l Group difference could be the reason for the posttest measures, not because of the treatment. TIME 1 Assignment Treatment group Control group TIME 2 Pretest Treatment Posttest None IV DV None 34

Nonequivalent Group Pretest-Posttest Design l No random assignment l Can establish initial equivalence if the pretest of the two groups generates similar results TIME 1 Assignment Treatment group Control group TIME 2 Pretest Treatment Posttest DV IV DV DV None 35

Quasi-Experimental Design Time Series Design l Involves getting data from a series of points before and after a treatment l Evaluating statistically whether the treatment had an effect or not TIME 1 Assignm ent GROUP 1 TIME 2 Pretest Treatment Posttest DVs are measured 4 times in different time points before exposed to the treatment IV DV are measured 4 times in different time points 36