Experiment Basics Designs Psych 231 Research Methods in
Experiment Basics: Designs Psych 231: Research Methods in Psychology
n n n Quiz 7 due Friday Don’t forget that Exam 2 is coming up (Mon. Oct 21) Check the Syllabus for change in assigned chapters (the new edition moved a chapter, so most chapters #’s have changed from this point forward). Announcements
n How do we introduce control? n Methods of Experimental Control • Constancy/Randomization • Comparison • Production Controlling Variability
n Constancy/Randomization n If there is a variable that may be related to the DV that you can’t (or don’t want to) manipulate • Control variable: hold it constant (so there isn’t any variability from that variable, no R weight from that variable) • Random variable: let it vary randomly across all of the experimental conditions (so the R weight from that variable is the same for all conditions) Methods of Controlling Variability
n Comparison n An experiment always makes a comparison, so it must have at least two groups (2 sides of our scale in the weight analogy) • Sometimes there are control groups • This is often the absence of the treatment Training group • • No training (Control) group Without control groups if is harder to see what is really happening in the experiment • It is easier to be swayed by plausibility or inappropriate comparisons (see diet crystal example) Useful for eliminating potential confounds (think about our list of threats to internal validity) Methods of Controlling Variability
n Comparison n An experiment always makes a comparison, so it must have at least two groups • Sometimes there are control groups • This is often the absence of the treatment • Sometimes there a range of values of the IV 1 week of Training group 2 weeks of Training group 3 weeks of Training group Methods of Controlling Variability
n Production n The experimenter selects the specific values of the Independent Variables 1 week of Training group 2 weeks of Training group 3 weeks of Training group n selects the specific values variability 1 weeks 2 weeks 3 weeks Duration taking the training program Methods of Controlling Variability
n Production n The experimenter selects the specific values of the Independent Variables 1 week of Training group 2 weeks of Training group 3 weeks of Training group • Need to do this carefully • Suppose that you don’t find a difference in the DV across your different groups • Is this because the IV and DV aren’t related? • Or is it because your levels of IV weren’t different enough Methods of Controlling Variability
n n So far we’ve covered a lot of the general details of experiments Now let’s consider some specific experimental designs. n n Some bad (but not uncommon) designs (and potential fixes) Some good designs • • 1 Factor, two levels 1 Factor, multi-levels Factorial (more than 1 factor) Between & within factors Experimental designs
n Bad design example 1: Does standing close to somebody cause them to move? n n (theory of personal space) “hmm… that’s an empirical question. Let’s see what happens if …” Design: you stand closely to people and see how long before they move Problem: no control group to establish the comparison group (this design is sometimes called “one-shot case study design”) Fix: introduce a (or some) comparison group(s) Very Close (. 1 m) Close (. 5 m) Not Close (1. 0 m) Poorly designed experiments
n Bad design example 2: n Does a relaxation program decrease the urge to smoke? n 2 groups • relaxation training group • no relaxation training group n Training group No training (Control) group The participants choose which group to be in Poorly designed experiments
n Bad design example 2: Non-equivalent control groups Self Assignment Independent Variable Dependent Variable Training group Measure No training (Control) group Measure participants Random Assignment Problem: selection bias for the two groups Fix: need to do random assignment to groups Poorly designed experiments
n Bad design example 3: n Does a relaxation program decrease the urge to smoke? n Pre-test desire to smoke Give relaxation training program Post-test desire to smoke n n Poorly designed experiments
n Bad design example 3: One group pretest-posttest design Dependent Independent Variable Dependent Variable Pre vs. Post Variable participants Pre-test Training group Post-test Measure Post-test No Training Fix: Add Pre-test Measure group another factor Problems include: history, maturation, testing, and more Poorly designed experiments
n n So far we’ve covered a lot of the general details of experiments Now let’s consider some specific experimental designs. n n Some bad (but not uncommon) designs Some good designs • • 1 Factor, two levels 1 Factor, multi-levels Factorial (more than 1 factor) Between & within factors Experimental designs
n Good design example n What are our IV and DV? How does anxiety level affect test performance? • Two groups take the same test • Grp 1(low anxiety group): 5 min lecture on how good grades don’t matter, just trying is good enough • Grp 2 (moderate anxiety group): 5 min lecture on the importance of good grades for success n 1 Factor (Independent variable), two levels • Basically you want to compare two treatments (conditions) • The statistics are pretty easy, a t-test 1 factor - 2 levels
n Good design example n How does anxiety level affect test performance? Random Assignment IV: Anxiety Dependent Variable Low Test Moderate Test participants 1 factor - 2 levels
n Good design example n How does anxiety level affect test performance? anxiety low moderate 60 80 test performance One factor Use a t-test to see if these points are statistically different 80 - 60 = 20 Observed difference between conditions T-test = Difference expected by chance low Two levels 1 factor - 2 levels moderate anxiety Based on estimate of error R
n Advantages: n n Simple, relatively easy to interpret the results Is the independent variable worth studying? • If no effect, then may don’t bother with a more complex design n Sometimes two levels is all you need • One theory predicts one pattern and another predicts a different pattern 1 factor - 2 levels
n Disadvantages: n “True” shape of the function is hard to see • Interpolation and Extrapolation are not a good idea Interpolation test performance What happens within of the ranges that you test? low 1 factor - 2 levels moderate anxiety
n Disadvantages: n “True” shape of the function is hard to see • Interpolation and Extrapolation are not a good idea Extrapolation test performance What happens outside of the ranges that you test? low moderate anxiety 1 factor - 2 levels high
n n So far we’ve covered a lot of the general details of experiments Now let’s consider some specific experimental designs. n n Some bad (but not uncommon) designs Some good designs • • 1 Factor, two levels 1 Factor, multi-levels Factorial (more than 1 factor) Between & within factors Experimental designs
n n For more complex theories you will typically need more complex designs (more than two levels of one IV) 1 factor - more than two levels n n Basically you want to compare more than two conditions The statistics are a little more difficult, an ANOVA (Analysis of Variance) 1 Factor - multilevel experiments
n Good design example (similar to earlier ex. ) n How does anxiety level affect test performance? • Groups take the same test • Grp 1(low anxiety group): 5 min lecture on how good grades don’t matter, just trying is good enough • Grp 2 (moderate anxiety group): 5 min lecture on the importance of good grades for success • Grp 3 (high anxiety group): 5 min lecture on how the students must pass this test to pass the course 1 Factor - multilevel experiments
Random Assignment participants IV: Anxiety Dependent Variable Low Test Moderate Test High Test 1 factor - 3 levels
low mod high 60 80 60 test performance anxiety low mod high anxiety 1 Factor - multilevel experiments
n Advantages n Gives a better picture of the relationship (functions other than just straight lines) 2 levels test performance low moderate anxiety n 3 levels low mod high anxiety Generally, the more levels you have, the less you have to worry about your range of the independent variable 1 Factor - multilevel experiments
n Disadvantages n n Needs more resources (participants and/or stimuli) Requires more complex statistical analysis (ANOVA [Analysis of Variance] & follow-up pair -wise comparisons) 1 Factor - multilevel experiments
n The ANOVA just tells you that not all of the groups are equal. Tests: High = Moderate = Low n n If your conclusion is that not all groups are equal (you get a “significant ANOVA”), then you should do further tests to see where the differences are: Pair-wise comparisons (think of these as doing what a t-test would do): • High vs. Low • High vs. Moderate • Low vs. Moderate Pair-wise comparisons
- Slides: 29