Experimental Designs Psych 231 Research Methods in Psychology

Experimental Designs Psych 231: Research Methods in Psychology

n Exam 2 coming up: Monday Oct 27 th n n Review session Thursday 6: 30 De. Garmo 463 Piloting experiments in lab this week Announcements

n 1 Factor - two levels n n n Advantages: • Simple, relatively easy to interpret the results, good first step, sometimes all you need Disadvantages: • “True” shape of the function is hard to see 1 Factor - more than two levels n Advantages • Better picture of the function • Less worry about your range of the independent variable n Disadvantages • Needs more resources (participants and/or stimuli) • Requires more complex statistical analysis (analysis of variance and pair-wise comparisons) Experimental Designs

n Factorial Designs: Two or more factors n Advantages • Interaction effects • Adding factors decreases the variability • Because you’re controlling more of the variables that influence the dependent variable • This increases the statistical Power of the statistical tests • Increases generalizability of the results • Because you have a situation closer to the real world (where all sorts of variables are interacting) n Disadvantages • Experiments become very large, and can become unwieldy • The statistical analyses get much more complex • Interpretation of the results can get hard • Need more resources Factorial experiments

n What is the effect of presenting words in color on memory for those words? n So you present lists of words for recall either in color or in black-and-white. Clock Chair Cab n Clock Chair Cab Two different designs to examine this question Example

n Between-Groups Factor § 2 -levels § Each of the participants is in only one level of the IV levels Colored words Clock Chair Cab participants Test BW words Clock Chair Cab

n Within-Groups Factor § Sometimes called “repeated measures” design § 2 -levels, All of the participants are in both levels of the IV levels participants Colored words Clock Chair Cab Test BW words Clock Chair Cab Test

n Between-subjects designs n Each participant participates in one and only one condition of the experiment. n Within-subjects designs n All participants participate in all of the conditions of the experiment. Colored words Test participants BW words participants Colored words Test BW words Test Between vs. Within Subjects Designs

n Between-subjects designs n Each participant participates in one and only one condition of the experiment. n Within-subjects designs n All participants participate in all of the conditions of the experiment. Colored words Test participants BW words participants Colored words Test BW words Test Between vs. Within Subjects Designs

n Clock Colored words Chair Cab Advantages: Test participants BW Clock words Chair Cab n Independence of groups (levels of the IV) • Harder to guess what the experiment is about without experiencing the other levels of IV • Exposure to different levels of the independent variable(s) cannot “contaminate” the dependent variable • Sometimes this is a ‘must, ’ because you can’t reverse the effects of prior exposure to other levels of the IV • No order effects to worry about • Counterbalancing is not required Between subjects designs

n Disadvantages Clock Colored words Chair Cab Test participants BW Clock words Chair Cab n Individual differences between the people in the groups • Excessive variability • Non-Equivalent groups Between subjects designs

n The groups are composed of different individuals participants Colored words BW words Individual differences Test

n The groups are composed of different individuals participants n Colored words BW words Excessive variability due to individual differences n Test Harder to detect the effect of the IV if there is one Individual differences NR R R

n The groups are composed of different individuals participants n Colored words Test BW words Non-Equivalent groups (possible confound) n The groups may differ not only because of the IV, but also because the groups are composed of different individuals Individual differences

n Strive for Equivalent groups n n n Created equally - use the same process to create both groups Treated equally - keep the experience as similar as possible for the two groups Composed of equivalent individuals • Random assignment to groups - eliminate bias • Matching groups - match each individuals in one group to an individual in the other group on relevant characteristics Dealing with Individual Differences

Group A Red Short 21 yrs Blue tall 23 yrs Green average 22 yrs Brown tall 22 yrs Group B matched Matching groups Red Short 21 yrs Blue tall 23 yrs Green average 22 yrs Brown tall 22 yrs n Matched groups n n Trying to create equivalent groups Also trying to reduce some of the overall variability • Eliminating variability from the variables that you matched people on Color Height Age

n Between-subjects designs n Each participant participates in one and only one condition of the experiment. n Within-subjects designs n All participants participate in all of the conditions of the experiment. Colored words Test participants Colored words Test BW words Between vs. Within Subjects Designs

n Advantages: n Don’t have to worry about individual differences • Same people in all the conditions • Variability between conditions is smaller (statistical advantage) n Fewer participants are required Within subjects designs

n Disadvantages n Order effects: • Carry-over effects • Progressive error Counterbalancing is probably necessary to address these order effects Within subjects designs

n Carry-over effects n n Transfer between conditions is possible Effects may persist from one condition into another • e. g. Alcohol vs no alcohol experiment on the effects on hand-eye coordination. Hard to know how long the effects of alcohol may persist. Condition 1 Condition 2 test Order effects How long do we wait for the effects to wear off? test

n Progressive error n n Practice effects – improvement due to repeated practice Fatigue effects – performance deteriorates as participants get bored, tired, distracted Order effects

n Counterbalancing is probably necessary n This is used to control for “order effects” • Ideally, use every possible order • n! (e. g. , AB = 2! = 2 orders; ABC = 3! = 6 orders, ABCD = 4! = 24 orders, etc ). n All counterbalancing assumes Symmetrical Transfer • The assumption that AB and BA have reverse effects and thus cancel out in a counterbalanced design Dealing with order effects

n Simple case n n Two conditions A & B Two counterbalanced orders: • AB • BA Colored words Test BW words Test Colored words Test participants n Not so simple analysis: need to analyze as a factorial design, with 2 factors (word color & order) Counterbalancing

n n Simple case More than two conditions: n Often it is not practical to use every possible ordering • n! (e. g. , AB = 2! = 2 orders; ABC = 3! = 6 orders, ABCD = 4! = 24 orders, etc ) • Common Solution: Partial counterbalancing • Latin square designs – a form of partial counterbalancing, so that each group of trials occur in each position an equal number of times Counterbalancing

n Example: consider four conditions Recall: ABCD = 4! = 24 possible orders 1) Unbalanced Latin square: each condition appears in each position (4 orders) n Order 1 A B C D Order 2 B C D A Order 3 C D A B Order 4 D A B C Partial counterbalancing

n Example: consider four conditions Recall: ABCD = 4! = 24 possible orders 2) Balanced Latin square: each condition appears before and after all others (8 orders) n A B C D A B D C B C D A B C A D C D A B C D B A D A B C D A C B Partial counterbalancing

n Mixed designs n n Treat some factors as within-subjects (participants get all levels of that factor) and others as between-subjects (each level of this factor gets a different group of participants). This only works with factorial (multi-factor) designs Mixed factorial designs

Source Word Type Depth of Proc Word * Depth Error (within) Error (btwn) SS 22. 4 120. 3 10. 5 357. 1 456. 4 df 1 148 148 Mean Square 22. 4 120. 3 10. 5 2. 4 3. 1 F 9. 3 39. 0 4. 3 Sig. . 003 <. 001. 039 • Main effect of both variables • An interaction Class experiment results

n You need to describe: n n n How many factors How many levels of each factor Whether the factors are within or between groups • e. g. , 2 (shallow/deep processing) x 2 (abstract/concrete) mixed groups factorial design abstract concrete Shallow 4. 0 4. 2 Deep 4. 9 5. 8 Describing your design