Internal Validity Accuracy of your conclusions concerning the

Internal Validity Ø Accuracy of your conclusions concerning the cause and effect relationship between the IV and DV. Ø Could something else have caused the change in the DV?

Threats to Internal Validity

History Events that occur between the DV measurements in a repeated measures design. Ø Example: l l l DV: Pretest and Posttest measure of school safety. IV: Metal detectors installed Threat: school shooting between pre and posttest

Maturation Ø Changes in participants that occur over time during an experiment. Ø More of a threat for repeated measures studies. l Study skills improve over the semester

Testing or Practice Effects Ø A threat to repeated measures studies in which, measuring the DV (pretest) causes a change in the DV (posttest). l Retaking the SAT Ø Reactive measures l Measuring employee satisfaction may actually raise satisfaction levels.

Instrumentation Ø Changes in the measurement of the DV due to the measuring “device” Ø Could be equipment failure or human error. l Different set of observers

Statistical Regression The tendency for extreme scores to move closer to the mean when retested. l Pretest for Type A personality; interventions may appear to work well only because these extremely uptight people could only get less uptight.

Mortality Ø Differential dropout rates between the groups. Ø Particularly true of unpleasant or lengthy studies. l Study on stress and immune response in rats.

Diffusion of Treatment Ø Participants in one group become aware of the treatment given to another group. l One team becomes aware of special treatment that another is getting.

Selection Ø Unequal groups due primarily to nonrandom assignment or using existing groups. l Comparing Kathy’s team to Scott’s team

Interactions with Selection Systematic differences between or among groups based on each of the other threats. l Selection X History: The control group experiences something the experimental group does not.

Protecting Internal Validity 1. Random Assignment 2. Experimental design e. g. , adding a control group

Threats to External Validity

External Validity Ø Refers to the extent to which findings from a study can be generalized to situations outside of the study.

Internal Validity Ø External What is the cause-and Ø Does this relationship -effect relationship? hold in other situations?

Three Types of Generalizability Ø Population: from our sample to the population Ø Environmental: from the lab to the “real world” Ø Temporal: from the present to the future.

Four Threats to External Validity

1. Pretesting Ø May actually change the subject in a way that we didn’t want. In the real world we would only manipulate the IV, not pretest. Ø

Solomon Four Group Design Ø Allows us to see what would happen without pretesting. R R O 1 O 3 X X O 2 O 4 O 5 O 6

2. Interaction of Selection with Treatment When the treatment effect only occurs with a specific sample.

3. Reactive Arrangements Conditions of the experimental setting that alter Ss behavior. e. g. , being observed in a lab vs. the real world.

4. Multiple Treatment Interference A problem for repeated measures studies because in the real world people might only receive one treatment.

Other Threats to Generalizability Using Convenience Samples -White Rats -College students -White -American

When do we need external validity? Only when we are trying to predict real-life behavior in the real world.

Will your group project have external validity? Does it matter?

Will your group project have internal validity? Does it matter?

What are the most serious threats to each? Evaluating your own research and that of others

Programmatic Research A series of research experiments that deal with a related topic or question. Systematically rules our threats to internal and external validity.