Experimental Control cont Psych 231 Research Methods in

Experimental Control cont. Psych 231: Research Methods in Psychology

Announcements n Re-writes of group project intros due this week in lab – Please attach the original intro draft n Group Project Methods section due next week in lab – Please bring your textbook to lab n Remember to download and read the articles for class exp soon (e. g. , before next week’s labs)

Sources of variability (noise) n Sources of Total (T) Variability: T = NRexp + NRother +R Our goal is to reduce R and NRother so that we can detect NRexp. n That is, so we can see the changes in the DV that are due to the changes in the independent variable(s). n

Weight analogy n Imagine the different sources of variability as weights NR other NR exp R Treatment group NR other R control group

Weight analogy n If NRother and R are large relative to NRexp then detecting a difference may be difficult NR other NR exp R NR other R

Weight analogy n But if we reduce the size of NRother and R relative to NRexp then detecting gets easier NR other NR exp R R

Using control to reduce problems n Potential Problems – Excessive random variability – Confounding – Dissimulation

Potential Problems n Excessive random variability – If control procedures are not applied, then R component of data will be excessively large, and may make NR undetectable – So try to minimize this by using good measures of DV, good manipulations of IV, etc.

Excessive random variability Hard to detect the effect of NRexp R NR NR other exp NR other R

Potential Problems n Confounding – If relevant EV co-varies with IV, then NR component of data will be "significantly" large, and may lead to misattribution of effect to IV IV DV Co-vary together EV

Confounding Hard to detect the effect of NRexp because the effect looks like it could be from NRexp but is really (mostly) due to the NRother R NR other NR exp R

Potential Problems n Potential problem caused by experimental control – Dissimulation • If EV which interacts with IV is held constant, then effect of IV is known only for that level of EV, and may lead to overgeneralization of IV effect n This is a potential problem that affects the external validity

Methods of Controlling Variability Comparison n Production n Constancy/Randomization n

Methods of Controlling Variability n Comparison – An experiment always makes a comparison, so it must have at least two groups • Sometimes there are control groups • This is typically the absence of the treatment – Without control groups if is harder to see what is really happening in the experiment – it is easier to be swayed by plausibility or inappropriate comparisons • Sometimes there are just a range of values of the IV

Methods of Controlling Variability n Production – The experimenter selects the specific values of the Independent Variables • (as opposed to allowing the levels to freely vary as in observational studies) – Need to do this carefully • Suppose that you don’t find a difference in the DV across your different groups – Is this because the IV and DV aren’t related? – Or is it because your levels of IV weren’t different enough

Methods of Controlling Variability n Constancy/Randomization – If there is a variable that may be related to the DV that you can’t (or don’t want to) manipulate • you should either hold it constant (control variable) • let it vary randomly across all of the experimental conditions (random variable) – But beware confounds, variables that are related to both the IV and DV but aren’t controlled

Poorly designed experiments n Example: Does standing close to somebody cause them to move? – So you stand closely to people and see how long before they move – Problem: no control group to establish the comparison group (this design is sometimes called “one-shot case study design”)

Poorly designed experiments n Does a relaxation program decrease the urge to smoke? – One group pretest-posttest design – Pretest desire level – give relaxation program – posttest desire to smoke

Poorly designed experiments n One group pretest-posttest design Dependent Variable participants Pre-test Independent Variable Training group Dependent Variable Post-test Measure – Problems include: history, maturation, testing, instrument decay, statistical regression, and more

Poorly designed experiments Example: Smoking example again, but with two groups. The subjects get to choose which group (relaxation or no program) to be in n Non-equivalent control groups n – Problem: selection bias for the two groups, need to do random assignment to groups

Poorly designed experiments n Non-equivalent control groups Self Assignment Independent Variable Dependent Variable Training group Measure No training (Control) group Measure participants

“Well designed” experiments n Post-test only designs Random Assignment Independent Variable Dependent Variable Experimental group Measure Control group Measure participants

“Well designed” experiments n Pretest-posttest design Random Dependent Independent Assignment Variable Dependent Variable Measure Experimental group Measure Control group Measure participants

Next time n Read chapters 8 & 10